Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Multiset overlap similarity: The ratio of minimum to maximum mass is inverted. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The ratio of minimum to maximum mass is inverted. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The ratio of minimum to maximum mass is inverted. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The ratio of minimum to maximum mass is inverted. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The ratio of minimum to maximum mass is inverted. · case 05
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The largest single-label overlap replaces total shared mass. · case 01
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The largest single-label overlap replaces total shared mass. · case 02
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The largest single-label overlap replaces total shared mass. · case 03
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The largest single-label overlap replaces total shared mass. · case 04
The reduction disagrees with its explicit aggregation oracle.
Multiset overlap similarity: The largest single-label overlap replaces total shared mass. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Dot product uses minimum matched frequencies instead of products. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Dot product uses minimum matched frequencies instead of products. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Dot product uses minimum matched frequencies instead of products. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Dot product uses minimum matched frequencies instead of products. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Dot product uses minimum matched frequencies instead of products. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The first squared norm uses total frequency rather than squared counts. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The first squared norm uses total frequency rather than squared counts. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The first squared norm uses total frequency rather than squared counts. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The first squared norm uses total frequency rather than squared counts. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The first squared norm uses total frequency rather than squared counts. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Both norms are computed from the first sample. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Both norms are computed from the first sample. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Both norms are computed from the first sample. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Both norms are computed from the first sample. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Both norms are computed from the first sample. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The dot product is not squared despite squared-norm normalization. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The dot product is not squared despite squared-norm normalization. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The dot product is not squared despite squared-norm normalization. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The dot product is not squared despite squared-norm normalization. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: The dot product is not squared despite squared-norm normalization. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Squared norms are added instead of multiplied. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Squared norms are added instead of multiplied. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Squared norms are added instead of multiplied. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Squared norms are added instead of multiplied. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Squared norms are added instead of multiplied. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Norms discard coordinates absent from the other sample. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Norms discard coordinates absent from the other sample. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Norms discard coordinates absent from the other sample. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Norms discard coordinates absent from the other sample. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Norms discard coordinates absent from the other sample. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: A zero vector is assigned perfect cosine similarity. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: A zero vector is assigned perfect cosine similarity. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: A zero vector is assigned perfect cosine similarity. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: A zero vector is assigned perfect cosine similarity. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: A zero vector is assigned perfect cosine similarity. · case 05
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Frequency vectors are dotted by insertion order rather than matching labels. · case 01
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Frequency vectors are dotted by insertion order rather than matching labels. · case 02
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Frequency vectors are dotted by insertion order rather than matching labels. · case 03
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Frequency vectors are dotted by insertion order rather than matching labels. · case 04
The reduction disagrees with its explicit aggregation oracle.
Frequency cosine squared: Frequency vectors are dotted by insertion order rather than matching labels. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Truthy filtering discards legitimate zero observations. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Truthy filtering discards legitimate zero observations. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Truthy filtering discards legitimate zero observations. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Truthy filtering discards legitimate zero observations. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Truthy filtering discards legitimate zero observations. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing observations are replaced by zero and admitted as complete pairs. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing observations are replaced by zero and admitted as complete pairs. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing observations are replaced by zero and admitted as complete pairs. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing observations are replaced by zero and admitted as complete pairs. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing observations are replaced by zero and admitted as complete pairs. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Paired count uses all original rows. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Paired count uses all original rows. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Paired count uses all original rows. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Paired count uses all original rows. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Paired count uses all original rows. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: X subtotal includes rows with missing y. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: X subtotal includes rows with missing y. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: X subtotal includes rows with missing y. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: X subtotal includes rows with missing y. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: X subtotal includes rows with missing y. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing values are removed from each coordinate separately before pairing. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing values are removed from each coordinate separately before pairing. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing values are removed from each coordinate separately before pairing. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing values are removed from each coordinate separately before pairing. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Missing values are removed from each coordinate separately before pairing. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Cross-product sum is replaced by the product of coordinate sums. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Cross-product sum is replaced by the product of coordinate sums. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Cross-product sum is replaced by the product of coordinate sums. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Cross-product sum is replaced by the product of coordinate sums. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Cross-product sum is replaced by the product of coordinate sums. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Squared distance is computed after summing residuals. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Squared distance is computed after summing residuals. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Squared distance is computed after summing residuals. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Squared distance is computed after summing residuals. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Squared distance is computed after summing residuals. · case 05
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Repeated complete rows are collapsed. · case 01
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Repeated complete rows are collapsed. · case 02
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Repeated complete rows are collapsed. · case 03
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Repeated complete rows are collapsed. · case 04
The reduction disagrees with its explicit aggregation oracle.
Jointly observed pair summary: Repeated complete rows are collapsed. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: All observations receive equal weight instead of equal total group weight. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: All observations receive equal weight instead of equal total group weight. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: All observations receive equal weight instead of equal total group weight. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: All observations receive equal weight instead of equal total group weight. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: All observations receive equal weight instead of equal total group weight. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Residuals cancel within each group before squaring. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Residuals cancel within each group before squaring. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Residuals cancel within each group before squaring. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Residuals cancel within each group before squaring. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Residuals cancel within each group before squaring. · case 05
The reduction disagrees with its explicit aggregation oracle.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗