Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Binary calibration decomposition: Binary uncertainty uses the square of prevalence. · case 01
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Binary uncertainty uses the square of prevalence. · case 02
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Binary uncertainty uses the square of prevalence. · case 03
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Binary uncertainty uses the square of prevalence. · case 04
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Binary uncertainty uses the square of prevalence. · case 05
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Uncertainty is reported as total squared variation rather than per-observation variance. · case 01
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Uncertainty is reported as total squared variation rather than per-observation variance. · case 02
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Uncertainty is reported as total squared variation rather than per-observation variance. · case 03
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Uncertainty is reported as total squared variation rather than per-observation variance. · case 04
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Uncertainty is reported as total squared variation rather than per-observation variance. · case 05
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: A zero-size forecast bin receives synthetic outcomes. · case 01
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: A zero-size forecast bin receives synthetic outcomes. · case 02
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: A zero-size forecast bin receives synthetic outcomes. · case 03
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: A zero-size forecast bin receives synthetic outcomes. · case 04
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: A zero-size forecast bin receives synthetic outcomes. · case 05
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Forecast probabilities are rounded to integers before grouping. · case 01
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Forecast probabilities are rounded to integers before grouping. · case 02
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Forecast probabilities are rounded to integers before grouping. · case 03
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Forecast probabilities are rounded to integers before grouping. · case 04
The reduction disagrees with its explicit aggregation oracle.
Binary calibration decomposition: Forecast probabilities are rounded to integers before grouping. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected extrema are emitted by value rather than original position. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected extrema are emitted by value rather than original position. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected extrema are emitted by value rather than original position. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected extrema are emitted by value rather than original position. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected extrema are emitted by value rather than original position. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal minima select the last occurrence. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal minima select the last occurrence. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal minima select the last occurrence. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal minima select the last occurrence. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal minima select the last occurrence. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal maxima select the last occurrence. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal maxima select the last occurrence. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal maxima select the last occurrence. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal maxima select the last occurrence. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Equal maxima select the last occurrence. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: A final incomplete bucket is discarded. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: A final incomplete bucket is discarded. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: A final incomplete bucket is discarded. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: A final incomplete bucket is discarded. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: A final incomplete bucket is discarded. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected points report bucket-local rather than global indices. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected points report bucket-local rather than global indices. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected points report bucket-local rather than global indices. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected points report bucket-local rather than global indices. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Selected points report bucket-local rather than global indices. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Flat buckets emit the same point twice. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Flat buckets emit the same point twice. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Flat buckets emit the same point twice. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Flat buckets emit the same point twice. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Flat buckets emit the same point twice. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket includes the first observation of the next bucket. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket includes the first observation of the next bucket. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket includes the first observation of the next bucket. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket includes the first observation of the next bucket. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket includes the first observation of the next bucket. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: The next bucket starts one sample too late. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: The next bucket starts one sample too late. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: The next bucket starts one sample too late. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: The next bucket starts one sample too late. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: The next bucket starts one sample too late. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket uses extrema of the entire series. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket uses extrema of the entire series. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket uses extrema of the entire series. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket uses extrema of the entire series. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Each bucket uses extrema of the entire series. · case 05
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Both selected indices come from minimum reduction. · case 01
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Both selected indices come from minimum reduction. · case 02
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Both selected indices come from minimum reduction. · case 03
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Both selected indices come from minimum reduction. · case 04
The reduction disagrees with its explicit aggregation oracle.
Bucket extrema downsampling: Both selected indices come from minimum reduction. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Shared labels contribute one instead of their shared multiplicity. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Shared labels contribute one instead of their shared multiplicity. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Shared labels contribute one instead of their shared multiplicity. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Shared labels contribute one instead of their shared multiplicity. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Shared labels contribute one instead of their shared multiplicity. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The denominator adds both frequencies, double-counting shared mass. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The denominator adds both frequencies, double-counting shared mass. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The denominator adds both frequencies, double-counting shared mass. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The denominator adds both frequencies, double-counting shared mass. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The denominator adds both frequencies, double-counting shared mass. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Union mass is calculated only on shared labels. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Union mass is calculated only on shared labels. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Union mass is calculated only on shared labels. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Union mass is calculated only on shared labels. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Union mass is calculated only on shared labels. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Per-label ratios are averaged instead of pooling frequency mass. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Per-label ratios are averaged instead of pooling frequency mass. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Per-label ratios are averaged instead of pooling frequency mass. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Per-label ratios are averaged instead of pooling frequency mass. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Per-label ratios are averaged instead of pooling frequency mass. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: One empty sample is treated like two empty samples. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: One empty sample is treated like two empty samples. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: One empty sample is treated like two empty samples. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: One empty sample is treated like two empty samples. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: One empty sample is treated like two empty samples. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Frequency mass is multiplied by nominal label magnitude. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Frequency mass is multiplied by nominal label magnitude. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Frequency mass is multiplied by nominal label magnitude. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Frequency mass is multiplied by nominal label magnitude. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: Frequency mass is multiplied by nominal label magnitude. · case 05
The reduction disagrees with its explicit aggregation oracle.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗