Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
All modes sorted · case 01
Only one mode is returned when frequencies tie.
All modes sorted · case 02
Only one mode is returned when frequencies tie.
All modes sorted · case 03
Only one mode is returned when frequencies tie.
All modes sorted · case 04
Only one mode is returned when frequencies tie.
All modes sorted · case 05
Only one mode is returned when frequencies tie.
Median absolute deviation · case 01
Deviation from zero replaces deviation from the sample median.
Median absolute deviation · case 02
Deviation from zero replaces deviation from the sample median.
Median absolute deviation · case 03
Deviation from zero replaces deviation from the sample median.
Median absolute deviation · case 04
Deviation from zero replaces deviation from the sample median.
Median absolute deviation · case 05
Deviation from zero replaces deviation from the sample median.
Mean absolute deviation about mean · case 01
The absolute mean is substituted for average absolute residual.
Mean absolute deviation about mean · case 02
The absolute mean is substituted for average absolute residual.
Mean absolute deviation about mean · case 03
The absolute mean is substituted for average absolute residual.
Mean absolute deviation about mean · case 04
The absolute mean is substituted for average absolute residual.
Mean absolute deviation about mean · case 05
The absolute mean is substituted for average absolute residual.
Root mean square squared · case 01
Squaring the mean allows opposite-sign observations to cancel.
Root mean square squared · case 02
Squaring the mean allows opposite-sign observations to cancel.
Root mean square squared · case 03
Squaring the mean allows opposite-sign observations to cancel.
Root mean square squared · case 04
Squaring the mean allows opposite-sign observations to cancel.
Root mean square squared · case 05
Squaring the mean allows opposite-sign observations to cancel.
Harmonic mean rational · case 01
The arithmetic mean replaces a reciprocal-domain mean.
Harmonic mean rational · case 02
The arithmetic mean replaces a reciprocal-domain mean.
Harmonic mean rational · case 03
The arithmetic mean replaces a reciprocal-domain mean.
Harmonic mean rational · case 04
The arithmetic mean replaces a reciprocal-domain mean.
Harmonic mean rational · case 05
The arithmetic mean replaces a reciprocal-domain mean.
Trimmed mean drop each tail · case 01
Trimming arrival order does not remove the extremes.
Trimmed mean drop each tail · case 02
Trimming arrival order does not remove the extremes.
Trimmed mean drop each tail · case 03
Trimming arrival order does not remove the extremes.
Trimmed mean drop each tail · case 04
Trimming arrival order does not remove the extremes.
Trimmed mean drop each tail · case 05
Trimming arrival order does not remove the extremes.
Winsorized sample · case 01
Trimming removes observations instead of replacing extreme values.
Winsorized sample · case 02
Trimming removes observations instead of replacing extreme values.
Winsorized sample · case 03
Trimming removes observations instead of replacing extreme values.
Winsorized sample · case 04
Trimming removes observations instead of replacing extreme values.
Winsorized sample · case 05
Trimming removes observations instead of replacing extreme values.
Population covariance rational · case 01
The cross moment is not centered by both means.
Population covariance rational · case 02
The cross moment is not centered by both means.
Population covariance rational · case 03
The cross moment is not centered by both means.
Population covariance rational · case 04
The cross moment is not centered by both means.
Population covariance rational · case 05
The cross moment is not centered by both means.
Sample covariance rational · case 01
The population divisor omits Bessel correction.
Sample covariance rational · case 02
The population divisor omits Bessel correction.
Sample covariance rational · case 03
The population divisor omits Bessel correction.
Sample covariance rational · case 04
The population divisor omits Bessel correction.
Sample covariance rational · case 05
The population divisor omits Bessel correction.
Linear regression slope rational · case 01
The ratio of means incorrectly forces the fitted line through zero.
Linear regression slope rational · case 02
The ratio of means incorrectly forces the fitted line through zero.
Linear regression slope rational · case 03
The ratio of means incorrectly forces the fitted line through zero.
Linear regression slope rational · case 04
The ratio of means incorrectly forces the fitted line through zero.
Linear regression slope rational · case 05
The ratio of means incorrectly forces the fitted line through zero.
Regression intercept given slope · case 01
The sample mean is returned without removing the slope contribution.
Regression intercept given slope · case 02
The sample mean is returned without removing the slope contribution.
Regression intercept given slope · case 03
The sample mean is returned without removing the slope contribution.
Regression intercept given slope · case 04
The sample mean is returned without removing the slope contribution.
Regression intercept given slope · case 05
The sample mean is returned without removing the slope contribution.
Coefficient of determination rational · case 01
Residual fraction is returned instead of explained fraction.
Coefficient of determination rational · case 02
Residual fraction is returned instead of explained fraction.
Coefficient of determination rational · case 03
Residual fraction is returned instead of explained fraction.
Coefficient of determination rational · case 04
Residual fraction is returned instead of explained fraction.
Coefficient of determination rational · case 05
Residual fraction is returned instead of explained fraction.
Mean absolute error · case 01
Opposite errors cancel before absolute value is applied.
Mean absolute error · case 02
Opposite errors cancel before absolute value is applied.
Mean absolute error · case 03
Opposite errors cancel before absolute value is applied.
Mean absolute error · case 04
Opposite errors cancel before absolute value is applied.
Mean absolute error · case 05
Opposite errors cancel before absolute value is applied.
Mean squared error · case 01
Averaging before squaring permits residual cancellation.
Mean squared error · case 02
Averaging before squaring permits residual cancellation.
Mean squared error · case 03
Averaging before squaring permits residual cancellation.
Mean squared error · case 04
Averaging before squaring permits residual cancellation.
Mean squared error · case 05
Averaging before squaring permits residual cancellation.
Binary precision rational · case 01
Recall denominator replaces predicted-positive count.
Binary precision rational · case 02
Recall denominator replaces predicted-positive count.
Binary precision rational · case 03
Recall denominator replaces predicted-positive count.
Binary precision rational · case 04
Recall denominator replaces predicted-positive count.
Binary precision rational · case 05
Recall denominator replaces predicted-positive count.
Binary recall rational · case 01
Predicted positives replace actual positives.
Binary recall rational · case 02
Predicted positives replace actual positives.
Binary recall rational · case 03
Predicted positives replace actual positives.
Binary recall rational · case 04
Predicted positives replace actual positives.
Binary recall rational · case 05
Predicted positives replace actual positives.
Binary f one score · case 01
Intersection-over-union replaces the F1 overlap normalization.
Binary f one score · case 02
Intersection-over-union replaces the F1 overlap normalization.
Binary f one score · case 03
Intersection-over-union replaces the F1 overlap normalization.
Binary f one score · case 04
Intersection-over-union replaces the F1 overlap normalization.
Binary f one score · case 05
Intersection-over-union replaces the F1 overlap normalization.
Binary specificity · case 01
False negatives replace false positives in actual-negative count.
Binary specificity · case 02
False negatives replace false positives in actual-negative count.
Binary specificity · case 03
False negatives replace false positives in actual-negative count.
Binary specificity · case 04
False negatives replace false positives in actual-negative count.
Binary specificity · case 05
False negatives replace false positives in actual-negative count.
Jaccard set similarity · case 01
Adding input lengths double-counts the intersection and duplicates.
Jaccard set similarity · case 02
Adding input lengths double-counts the intersection and duplicates.
Jaccard set similarity · case 03
Adding input lengths double-counts the intersection and duplicates.
Jaccard set similarity · case 04
Adding input lengths double-counts the intersection and duplicates.
Jaccard set similarity · case 05
Adding input lengths double-counts the intersection and duplicates.
Dice set similarity · case 01
The intersection is not doubled in Dice normalization.
Dice set similarity · case 02
The intersection is not doubled in Dice normalization.
Dice set similarity · case 03
The intersection is not doubled in Dice normalization.
Dice set similarity · case 04
The intersection is not doubled in Dice normalization.
Dice set similarity · case 05
The intersection is not doubled in Dice normalization.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗