Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Contingency pearson reduction: Only the largest cell discrepancy is retained. · case 01
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: Only the largest cell discrepancy is retained. · case 02
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: Only the largest cell discrepancy is retained. · case 03
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: Only the largest cell discrepancy is retained. · case 04
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: Only the largest cell discrepancy is retained. · case 05
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: A zero expected cell is assigned a unit discrepancy. · case 01
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: A zero expected cell is assigned a unit discrepancy. · case 02
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: A zero expected cell is assigned a unit discrepancy. · case 03
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: A zero expected cell is assigned a unit discrepancy. · case 04
The reduction disagrees with its explicit aggregation oracle.
Contingency pearson reduction: A zero expected cell is assigned a unit discrepancy. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pairs tied in a coordinate are counted as concordant. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pairs tied in a coordinate are counted as concordant. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pairs tied in a coordinate are counted as concordant. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pairs tied in a coordinate are counted as concordant. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pairs tied in a coordinate are counted as concordant. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Discordant pairs are added instead of subtracted. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Discordant pairs are added instead of subtracted. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Discordant pairs are added instead of subtracted. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Discordant pairs are added instead of subtracted. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Discordant pairs are added instead of subtracted. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Squaring the numerator loses negative association direction. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Squaring the numerator loses negative association direction. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Squaring the numerator loses negative association direction. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Squaring the numerator loses negative association direction. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Squaring the numerator loses negative association direction. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Tie-adjusted pair counts are replaced by all observation pairs. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Tie-adjusted pair counts are replaced by all observation pairs. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Tie-adjusted pair counts are replaced by all observation pairs. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Tie-adjusted pair counts are replaced by all observation pairs. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Tie-adjusted pair counts are replaced by all observation pairs. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: X tie correction counts tied pairs rather than varying pairs. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: X tie correction counts tied pairs rather than varying pairs. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: X tie correction counts tied pairs rather than varying pairs. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: X tie correction counts tied pairs rather than varying pairs. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: X tie correction counts tied pairs rather than varying pairs. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Y pair variability is tested using x differences. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Y pair variability is tested using x differences. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Y pair variability is tested using x differences. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Y pair variability is tested using x differences. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Y pair variability is tested using x differences. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Only adjacent observations are compared. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Only adjacent observations are compared. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Only adjacent observations are compared. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Only adjacent observations are compared. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Only adjacent observations are compared. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Separate marginal varying-pair counts are added instead of multiplied. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Separate marginal varying-pair counts are added instead of multiplied. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Separate marginal varying-pair counts are added instead of multiplied. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Separate marginal varying-pair counts are added instead of multiplied. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Separate marginal varying-pair counts are added instead of multiplied. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pair contributions depend on difference magnitudes instead of signs. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pair contributions depend on difference magnitudes instead of signs. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pair contributions depend on difference magnitudes instead of signs. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pair contributions depend on difference magnitudes instead of signs. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: Pair contributions depend on difference magnitudes instead of signs. · case 05
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: A constant coordinate is assigned a measured perfect association. · case 01
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: A constant coordinate is assigned a measured perfect association. · case 02
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: A constant coordinate is assigned a measured perfect association. · case 03
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: A constant coordinate is assigned a measured perfect association. · case 04
The reduction disagrees with its explicit aggregation oracle.
Tie adjusted concordance squared: A constant coordinate is assigned a measured perfect association. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement compares label names instead of co-membership. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement compares label names instead of co-membership. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement compares label names instead of co-membership. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement compares label names instead of co-membership. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement compares label names instead of co-membership. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Joint cell pair counts include self-pairs. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Joint cell pair counts include self-pairs. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Joint cell pair counts include self-pairs. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Joint cell pair counts include self-pairs. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Joint cell pair counts include self-pairs. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Chance agreement uses a sum of marginal pair totals. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Chance agreement uses a sum of marginal pair totals. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Chance agreement uses a sum of marginal pair totals. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Chance agreement uses a sum of marginal pair totals. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Chance agreement uses a sum of marginal pair totals. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The maximum marginal pair count replaces their average normalization ceiling. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The maximum marginal pair count replaces their average normalization ceiling. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The maximum marginal pair count replaces their average normalization ceiling. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The maximum marginal pair count replaces their average normalization ceiling. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The maximum marginal pair count replaces their average normalization ceiling. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement is not adjusted for expected chance agreement. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement is not adjusted for expected chance agreement. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement is not adjusted for expected chance agreement. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement is not adjusted for expected chance agreement. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Observed agreement is not adjusted for expected chance agreement. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Worse-than-chance association is clipped to zero. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Worse-than-chance association is clipped to zero. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Worse-than-chance association is clipped to zero. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Worse-than-chance association is clipped to zero. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Worse-than-chance association is clipped to zero. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Both marginal pair counts are taken from joint cells. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Both marginal pair counts are taken from joint cells. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Both marginal pair counts are taken from joint cells. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Both marginal pair counts are taken from joint cells. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: Both marginal pair counts are taken from joint cells. · case 05
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The number of observation pairs is replaced by observation count. · case 01
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The number of observation pairs is replaced by observation count. · case 02
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The number of observation pairs is replaced by observation count. · case 03
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The number of observation pairs is replaced by observation count. · case 04
The reduction disagrees with its explicit aggregation oracle.
Adjusted partition pair agreement: The number of observation pairs is replaced by observation count. · case 05
The reduction disagrees with its explicit aggregation oracle.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗