Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Dataset split: equal event times follow input order instead of record ID order · case 01
equal event times follow input order instead of record ID order.
Dataset split: equal event times follow input order instead of record ID order · case 02
equal event times follow input order instead of record ID order.
Dataset split: equal event times follow input order instead of record ID order · case 03
equal event times follow input order instead of record ID order.
Dataset split: equal event times follow input order instead of record ID order · case 04
equal event times follow input order instead of record ID order.
Dataset split: equal event times follow input order instead of record ID order · case 05
equal event times follow input order instead of record ID order.
Dataset split: training data includes observations from validation period · case 01
training data includes observations from validation period.
Dataset split: training data includes observations from validation period · case 02
training data includes observations from validation period.
Dataset split: training data includes observations from validation period · case 03
training data includes observations from validation period.
Dataset split: training data includes observations from validation period · case 04
training data includes observations from validation period.
Dataset split: training data includes observations from validation period · case 05
training data includes observations from validation period.
Dataset split: per-class allocation rounds each quota down independently · case 01
per-class allocation rounds each quota down independently.
Dataset split: per-class allocation rounds each quota down independently · case 02
per-class allocation rounds each quota down independently.
Dataset split: per-class allocation rounds each quota down independently · case 03
per-class allocation rounds each quota down independently.
Dataset split: per-class allocation rounds each quota down independently · case 04
per-class allocation rounds each quota down independently.
Dataset split: per-class allocation rounds each quota down independently · case 05
per-class allocation rounds each quota down independently.
Dataset split: exact multiples emit an empty trailing block · case 01
exact multiples emit an empty trailing block.
Dataset split: exact multiples emit an empty trailing block · case 02
exact multiples emit an empty trailing block.
Dataset split: exact multiples emit an empty trailing block · case 03
exact multiples emit an empty trailing block.
Dataset split: exact multiples emit an empty trailing block · case 04
exact multiples emit an empty trailing block.
Dataset split: exact multiples emit an empty trailing block · case 05
exact multiples emit an empty trailing block.
Dataset split: impossible fold requests silently produce an incomplete fold map · case 01
impossible fold requests silently produce an incomplete fold map.
Dataset split: impossible fold requests silently produce an incomplete fold map · case 02
impossible fold requests silently produce an incomplete fold map.
Dataset split: impossible fold requests silently produce an incomplete fold map · case 03
impossible fold requests silently produce an incomplete fold map.
Dataset split: impossible fold requests silently produce an incomplete fold map · case 04
impossible fold requests silently produce an incomplete fold map.
Dataset split: impossible fold requests silently produce an incomplete fold map · case 05
impossible fold requests silently produce an incomplete fold map.
Dataset split: IDs from later partition manifests disappear · case 01
IDs from later partition manifests disappear.
Dataset split: IDs from later partition manifests disappear · case 02
IDs from later partition manifests disappear.
Dataset split: IDs from later partition manifests disappear · case 03
IDs from later partition manifests disappear.
Dataset split: IDs from later partition manifests disappear · case 04
IDs from later partition manifests disappear.
Dataset split: IDs from later partition manifests disappear · case 05
IDs from later partition manifests disappear.
Dataset split: purging uses row positions instead of timestamps · case 01
purging uses row positions instead of timestamps.
Dataset split: purging uses row positions instead of timestamps · case 02
purging uses row positions instead of timestamps.
Dataset split: purging uses row positions instead of timestamps · case 03
purging uses row positions instead of timestamps.
Dataset split: purging uses row positions instead of timestamps · case 04
purging uses row positions instead of timestamps.
Dataset split: purging uses row positions instead of timestamps · case 05
purging uses row positions instead of timestamps.
Dataset split: quota fill stops before an indivisible group that crosses the target · case 01
quota fill stops before an indivisible group that crosses the target.
Dataset split: quota fill stops before an indivisible group that crosses the target · case 02
quota fill stops before an indivisible group that crosses the target.
Dataset split: quota fill stops before an indivisible group that crosses the target · case 03
quota fill stops before an indivisible group that crosses the target.
Dataset split: quota fill stops before an indivisible group that crosses the target · case 04
quota fill stops before an indivisible group that crosses the target.
Dataset split: quota fill stops before an indivisible group that crosses the target · case 05
quota fill stops before an indivisible group that crosses the target.
Labels attach by row position after examples are reordered · case 01
Labels attach by row position after examples are reordered.
Labels attach by row position after examples are reordered · case 02
Labels attach by row position after examples are reordered.
Labels attach by row position after examples are reordered · case 03
Labels attach by row position after examples are reordered.
Labels attach by row position after examples are reordered · case 04
Labels attach by row position after examples are reordered.
Labels attach by row position after examples are reordered · case 05
Labels attach by row position after examples are reordered.
Class IDs shift when the vocabulary is sorted differently · case 01
Class IDs shift when the vocabulary is sorted differently.
Class IDs shift when the vocabulary is sorted differently · case 02
Class IDs shift when the vocabulary is sorted differently.
Class IDs shift when the vocabulary is sorted differently · case 03
Class IDs shift when the vocabulary is sorted differently.
Class IDs shift when the vocabulary is sorted differently · case 04
Class IDs shift when the vocabulary is sorted differently.
Class IDs shift when the vocabulary is sorted differently · case 05
Class IDs shift when the vocabulary is sorted differently.
One-hot targets omit classes absent from the current batch · case 01
One-hot targets omit classes absent from the current batch.
One-hot targets omit classes absent from the current batch · case 02
One-hot targets omit classes absent from the current batch.
One-hot targets omit classes absent from the current batch · case 03
One-hot targets omit classes absent from the current batch.
One-hot targets omit classes absent from the current batch · case 04
One-hot targets omit classes absent from the current batch.
One-hot targets omit classes absent from the current batch · case 05
One-hot targets omit classes absent from the current batch.
Filtering examples leaves the original label list attached · case 01
Filtering examples leaves the original label list attached.
Filtering examples leaves the original label list attached · case 02
Filtering examples leaves the original label list attached.
Filtering examples leaves the original label list attached · case 03
Filtering examples leaves the original label list attached.
Filtering examples leaves the original label list attached · case 04
Filtering examples leaves the original label list attached.
Filtering examples leaves the original label list attached · case 05
Filtering examples leaves the original label list attached.
A class-column reorder silently relabels target vectors · case 01
A class-column reorder silently relabels target vectors.
A class-column reorder silently relabels target vectors · case 02
A class-column reorder silently relabels target vectors.
A class-column reorder silently relabels target vectors · case 03
A class-column reorder silently relabels target vectors.
A class-column reorder silently relabels target vectors · case 04
A class-column reorder silently relabels target vectors.
A class-column reorder silently relabels target vectors · case 05
A class-column reorder silently relabels target vectors.
Soft targets are renormalized over only nonzero entries · case 01
Soft targets are renormalized over only nonzero entries.
Soft targets are renormalized over only nonzero entries · case 02
Soft targets are renormalized over only nonzero entries.
Soft targets are renormalized over only nonzero entries · case 03
Soft targets are renormalized over only nonzero entries.
Soft targets are renormalized over only nonzero entries · case 04
Soft targets are renormalized over only nonzero entries.
Soft targets are renormalized over only nonzero entries · case 05
Soft targets are renormalized over only nonzero entries.
Class weights are emitted in class order instead of example order · case 01
Class weights are emitted in class order instead of example order.
Class weights are emitted in class order instead of example order · case 02
Class weights are emitted in class order instead of example order.
Class weights are emitted in class order instead of example order · case 03
Class weights are emitted in class order instead of example order.
Class weights are emitted in class order instead of example order · case 04
Class weights are emitted in class order instead of example order.
Class weights are emitted in class order instead of example order · case 05
Class weights are emitted in class order instead of example order.
Multi-label annotations are returned without vocabulary encoding · case 01
Multi-label annotations are returned without vocabulary encoding.
Multi-label annotations are returned without vocabulary encoding · case 02
Multi-label annotations are returned without vocabulary encoding.
Multi-label annotations are returned without vocabulary encoding · case 03
Multi-label annotations are returned without vocabulary encoding.
Multi-label annotations are returned without vocabulary encoding · case 04
Multi-label annotations are returned without vocabulary encoding.
Multi-label annotations are returned without vocabulary encoding · case 05
Multi-label annotations are returned without vocabulary encoding.
Conflicting duplicate labels are resolved by last-write order · case 01
Conflicting duplicate labels are resolved by last-write order.
Conflicting duplicate labels are resolved by last-write order · case 02
Conflicting duplicate labels are resolved by last-write order.
Conflicting duplicate labels are resolved by last-write order · case 03
Conflicting duplicate labels are resolved by last-write order.
Conflicting duplicate labels are resolved by last-write order · case 04
Conflicting duplicate labels are resolved by last-write order.
Conflicting duplicate labels are resolved by last-write order · case 05
Conflicting duplicate labels are resolved by last-write order.
Ignored examples remain in the supervised loss mask · case 01
Ignored examples remain in the supervised loss mask.
Ignored examples remain in the supervised loss mask · case 02
Ignored examples remain in the supervised loss mask.
Ignored examples remain in the supervised loss mask · case 03
Ignored examples remain in the supervised loss mask.
Ignored examples remain in the supervised loss mask · case 04
Ignored examples remain in the supervised loss mask.
Ignored examples remain in the supervised loss mask · case 05
Ignored examples remain in the supervised loss mask.
A class permutation is applied in the wrong direction · case 01
A class permutation is applied in the wrong direction.
A class permutation is applied in the wrong direction · case 02
A class permutation is applied in the wrong direction.
A class permutation is applied in the wrong direction · case 03
A class permutation is applied in the wrong direction.
A class permutation is applied in the wrong direction · case 04
A class permutation is applied in the wrong direction.
A class permutation is applied in the wrong direction · case 05
A class permutation is applied in the wrong direction.
Expanded views lose the source example label · case 01
Expanded views lose the source example label.
Expanded views lose the source example label · case 02
Expanded views lose the source example label.
Expanded views lose the source example label · case 03
Expanded views lose the source example label.
Expanded views lose the source example label · case 04
Expanded views lose the source example label.
Expanded views lose the source example label · case 05
Expanded views lose the source example label.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗