Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Checkpoint mapping: requested axis permutation is ignored · case 01
requested axis permutation is ignored.
Checkpoint mapping: requested axis permutation is ignored · case 02
requested axis permutation is ignored.
Checkpoint mapping: requested axis permutation is ignored · case 03
requested axis permutation is ignored.
Checkpoint mapping: requested axis permutation is ignored · case 04
requested axis permutation is ignored.
Checkpoint mapping: requested axis permutation is ignored · case 05
requested axis permutation is ignored.
Checkpoint mapping: slot suffix matching associates the wrong parameter · case 01
slot suffix matching associates the wrong parameter.
Checkpoint mapping: slot suffix matching associates the wrong parameter · case 02
slot suffix matching associates the wrong parameter.
Checkpoint mapping: slot suffix matching associates the wrong parameter · case 03
slot suffix matching associates the wrong parameter.
Checkpoint mapping: slot suffix matching associates the wrong parameter · case 04
slot suffix matching associates the wrong parameter.
Checkpoint mapping: slot suffix matching associates the wrong parameter · case 05
slot suffix matching associates the wrong parameter.
Checkpoint mapping: interior marker occurrence is mistaken for shadow suffix · case 01
interior marker occurrence is mistaken for shadow suffix.
Checkpoint mapping: interior marker occurrence is mistaken for shadow suffix · case 02
interior marker occurrence is mistaken for shadow suffix.
Checkpoint mapping: interior marker occurrence is mistaken for shadow suffix · case 03
interior marker occurrence is mistaken for shadow suffix.
Checkpoint mapping: interior marker occurrence is mistaken for shadow suffix · case 04
interior marker occurrence is mistaken for shadow suffix.
Checkpoint mapping: interior marker occurrence is mistaken for shadow suffix · case 05
interior marker occurrence is mistaken for shadow suffix.
Checkpoint mapping: unlisted numeric conversions are accepted · case 01
unlisted numeric conversions are accepted.
Checkpoint mapping: unlisted numeric conversions are accepted · case 02
unlisted numeric conversions are accepted.
Checkpoint mapping: unlisted numeric conversions are accepted · case 03
unlisted numeric conversions are accepted.
Checkpoint mapping: unlisted numeric conversions are accepted · case 04
unlisted numeric conversions are accepted.
Checkpoint mapping: unlisted numeric conversions are accepted · case 05
unlisted numeric conversions are accepted.
Checkpoint mapping: tied parameter aliases load conflicting values · case 01
tied parameter aliases load conflicting values.
Checkpoint mapping: tied parameter aliases load conflicting values · case 02
tied parameter aliases load conflicting values.
Checkpoint mapping: tied parameter aliases load conflicting values · case 03
tied parameter aliases load conflicting values.
Checkpoint mapping: tied parameter aliases load conflicting values · case 04
tied parameter aliases load conflicting values.
Checkpoint mapping: tied parameter aliases load conflicting values · case 05
tied parameter aliases load conflicting values.
Checkpoint mapping: partial tensor restoration copies the wrong interval · case 01
partial tensor restoration copies the wrong interval.
Checkpoint mapping: partial tensor restoration copies the wrong interval · case 02
partial tensor restoration copies the wrong interval.
Checkpoint mapping: partial tensor restoration copies the wrong interval · case 03
partial tensor restoration copies the wrong interval.
Checkpoint mapping: partial tensor restoration copies the wrong interval · case 04
partial tensor restoration copies the wrong interval.
Checkpoint mapping: partial tensor restoration copies the wrong interval · case 05
partial tensor restoration copies the wrong interval.
Checkpoint mapping: tensor positions use insertion order instead of declared indices · case 01
tensor positions use insertion order instead of declared indices.
Checkpoint mapping: tensor positions use insertion order instead of declared indices · case 02
tensor positions use insertion order instead of declared indices.
Checkpoint mapping: tensor positions use insertion order instead of declared indices · case 03
tensor positions use insertion order instead of declared indices.
Checkpoint mapping: tensor positions use insertion order instead of declared indices · case 04
tensor positions use insertion order instead of declared indices.
Checkpoint mapping: tensor positions use insertion order instead of declared indices · case 05
tensor positions use insertion order instead of declared indices.
Checkpoint mapping: restoration plan ignores dependencies between aliases · case 01
restoration plan ignores dependencies between aliases.
Checkpoint mapping: restoration plan ignores dependencies between aliases · case 02
restoration plan ignores dependencies between aliases.
Checkpoint mapping: restoration plan ignores dependencies between aliases · case 03
restoration plan ignores dependencies between aliases.
Checkpoint mapping: restoration plan ignores dependencies between aliases · case 04
restoration plan ignores dependencies between aliases.
Checkpoint mapping: restoration plan ignores dependencies between aliases · case 05
restoration plan ignores dependencies between aliases.
Replica gradients ignore local sample counts · case 01
Replica gradients ignore local sample counts.
Replica gradients ignore local sample counts · case 02
Replica gradients ignore local sample counts.
Replica gradients ignore local sample counts · case 03
Replica gradients ignore local sample counts.
Replica gradients ignore local sample counts · case 04
Replica gradients ignore local sample counts.
Replica gradients ignore local sample counts · case 05
Replica gradients ignore local sample counts.
Disjoint gradient contributions are averaged instead of summed · case 01
Disjoint gradient contributions are averaged instead of summed.
Disjoint gradient contributions are averaged instead of summed · case 02
Disjoint gradient contributions are averaged instead of summed.
Disjoint gradient contributions are averaged instead of summed · case 03
Disjoint gradient contributions are averaged instead of summed.
Disjoint gradient contributions are averaged instead of summed · case 04
Disjoint gradient contributions are averaged instead of summed.
Disjoint gradient contributions are averaged instead of summed · case 05
Disjoint gradient contributions are averaged instead of summed.
Replica mean omits one received contribution from its denominator · case 01
Replica mean omits one received contribution from its denominator.
Replica mean omits one received contribution from its denominator · case 02
Replica mean omits one received contribution from its denominator.
Replica mean omits one received contribution from its denominator · case 03
Replica mean omits one received contribution from its denominator.
Replica mean omits one received contribution from its denominator · case 04
Replica mean omits one received contribution from its denominator.
Replica mean omits one received contribution from its denominator · case 05
Replica mean omits one received contribution from its denominator.
Per-worker loss means receive equal weight despite different token counts · case 01
Per-worker loss means receive equal weight despite different token counts.
Per-worker loss means receive equal weight despite different token counts · case 02
Per-worker loss means receive equal weight despite different token counts.
Per-worker loss means receive equal weight despite different token counts · case 03
Per-worker loss means receive equal weight despite different token counts.
Per-worker loss means receive equal weight despite different token counts · case 04
Per-worker loss means receive equal weight despite different token counts.
Per-worker loss means receive equal weight despite different token counts · case 05
Per-worker loss means receive equal weight despite different token counts.
All-gather output follows arrival order · case 01
All-gather output follows arrival order.
All-gather output follows arrival order · case 02
All-gather output follows arrival order.
All-gather output follows arrival order · case 03
All-gather output follows arrival order.
All-gather output follows arrival order · case 04
All-gather output follows arrival order.
All-gather output follows arrival order · case 05
All-gather output follows arrival order.
Gradient shards are clipped independently rather than by the global norm · case 01
Gradient shards are clipped independently rather than by the global norm.
Gradient shards are clipped independently rather than by the global norm · case 02
Gradient shards are clipped independently rather than by the global norm.
Gradient shards are clipped independently rather than by the global norm · case 03
Gradient shards are clipped independently rather than by the global norm.
Gradient shards are clipped independently rather than by the global norm · case 04
Gradient shards are clipped independently rather than by the global norm.
Gradient shards are clipped independently rather than by the global norm · case 05
Gradient shards are clipped independently rather than by the global norm.
A missing parameter shard silently shortens reconstruction · case 01
A missing parameter shard silently shortens reconstruction.
A missing parameter shard silently shortens reconstruction · case 02
A missing parameter shard silently shortens reconstruction.
A missing parameter shard silently shortens reconstruction · case 03
A missing parameter shard silently shortens reconstruction.
A missing parameter shard silently shortens reconstruction · case 04
A missing parameter shard silently shortens reconstruction.
A missing parameter shard silently shortens reconstruction · case 05
A missing parameter shard silently shortens reconstruction.
Non-owner replicas contribute gradients for an owned shard · case 01
Non-owner replicas contribute gradients for an owned shard.
Non-owner replicas contribute gradients for an owned shard · case 02
Non-owner replicas contribute gradients for an owned shard.
Non-owner replicas contribute gradients for an owned shard · case 03
Non-owner replicas contribute gradients for an owned shard.
Non-owner replicas contribute gradients for an owned shard · case 04
Non-owner replicas contribute gradients for an owned shard.
Non-owner replicas contribute gradients for an owned shard · case 05
Non-owner replicas contribute gradients for an owned shard.
A late gradient from an older collective step is applied · case 01
A late gradient from an older collective step is applied.
A late gradient from an older collective step is applied · case 02
A late gradient from an older collective step is applied.
A late gradient from an older collective step is applied · case 03
A late gradient from an older collective step is applied.
A late gradient from an older collective step is applied · case 04
A late gradient from an older collective step is applied.
A late gradient from an older collective step is applied · case 05
A late gradient from an older collective step is applied.
Mean reduction trusts declared world size after dropout · case 01
Mean reduction trusts declared world size after dropout.
Mean reduction trusts declared world size after dropout · case 02
Mean reduction trusts declared world size after dropout.
Mean reduction trusts declared world size after dropout · case 03
Mean reduction trusts declared world size after dropout.
Mean reduction trusts declared world size after dropout · case 04
Mean reduction trusts declared world size after dropout.
Mean reduction trusts declared world size after dropout · case 05
Mean reduction trusts declared world size after dropout.
Sparse gradient coordinates are combined by list position · case 01
Sparse gradient coordinates are combined by list position.
Sparse gradient coordinates are combined by list position · case 02
Sparse gradient coordinates are combined by list position.
Sparse gradient coordinates are combined by list position · case 03
Sparse gradient coordinates are combined by list position.
Sparse gradient coordinates are combined by list position · case 04
Sparse gradient coordinates are combined by list position.
Sparse gradient coordinates are combined by list position · case 05
Sparse gradient coordinates are combined by list position.
Retransmitted replica updates contribute more than once · case 01
Retransmitted replica updates contribute more than once.
Retransmitted replica updates contribute more than once · case 02
Retransmitted replica updates contribute more than once.
Retransmitted replica updates contribute more than once · case 03
Retransmitted replica updates contribute more than once.
Retransmitted replica updates contribute more than once · case 04
Retransmitted replica updates contribute more than once.
Retransmitted replica updates contribute more than once · case 05
Retransmitted replica updates contribute more than once.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗