Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Reduce-scatter gives every rank the first slice · case 01
Reduce-scatter gives every rank the first slice.
Reduce-scatter gives every rank the first slice · case 02
Reduce-scatter gives every rank the first slice.
Reduce-scatter gives every rank the first slice · case 03
Reduce-scatter gives every rank the first slice.
Reduce-scatter gives every rank the first slice · case 04
Reduce-scatter gives every rank the first slice.
Reduce-scatter gives every rank the first slice · case 05
Reduce-scatter gives every rank the first slice.
All-reduce silently truncates vectors with mismatched shapes · case 01
All-reduce silently truncates vectors with mismatched shapes.
All-reduce silently truncates vectors with mismatched shapes · case 02
All-reduce silently truncates vectors with mismatched shapes.
All-reduce silently truncates vectors with mismatched shapes · case 03
All-reduce silently truncates vectors with mismatched shapes.
All-reduce silently truncates vectors with mismatched shapes · case 04
All-reduce silently truncates vectors with mismatched shapes.
All-reduce silently truncates vectors with mismatched shapes · case 05
All-reduce silently truncates vectors with mismatched shapes.
Shard values are placed by list position instead of declared offsets · case 01
Shard values are placed by list position instead of declared offsets.
Shard values are placed by list position instead of declared offsets · case 02
Shard values are placed by list position instead of declared offsets.
Shard values are placed by list position instead of declared offsets · case 03
Shard values are placed by list position instead of declared offsets.
Shard values are placed by list position instead of declared offsets · case 04
Shard values are placed by list position instead of declared offsets.
Shard values are placed by list position instead of declared offsets · case 05
Shard values are placed by list position instead of declared offsets.
Collective slot IDs are replaced by row positions · case 01
Collective slot IDs are replaced by row positions.
Collective slot IDs are replaced by row positions · case 02
Collective slot IDs are replaced by row positions.
Collective slot IDs are replaced by row positions · case 03
Collective slot IDs are replaced by row positions.
Collective slot IDs are replaced by row positions · case 04
Collective slot IDs are replaced by row positions.
Collective slot IDs are replaced by row positions · case 05
Collective slot IDs are replaced by row positions.
Worker variances omit between-worker spread · case 01
Worker variances omit between-worker spread.
Worker variances omit between-worker spread · case 02
Worker variances omit between-worker spread.
Worker variances omit between-worker spread · case 03
Worker variances omit between-worker spread.
Worker variances omit between-worker spread · case 04
Worker variances omit between-worker spread.
Worker variances omit between-worker spread · case 05
Worker variances omit between-worker spread.
Gradient bucket reconstruction discards global offsets · case 01
Gradient bucket reconstruction discards global offsets.
Gradient bucket reconstruction discards global offsets · case 02
Gradient bucket reconstruction discards global offsets.
Gradient bucket reconstruction discards global offsets · case 03
Gradient bucket reconstruction discards global offsets.
Gradient bucket reconstruction discards global offsets · case 04
Gradient bucket reconstruction discards global offsets.
Gradient bucket reconstruction discards global offsets · case 05
Gradient bucket reconstruction discards global offsets.
Equal-offset shard fragments retain arrival order · case 01
Equal-offset shard fragments retain arrival order.
Equal-offset shard fragments retain arrival order · case 02
Equal-offset shard fragments retain arrival order.
Equal-offset shard fragments retain arrival order · case 03
Equal-offset shard fragments retain arrival order.
Equal-offset shard fragments retain arrival order · case 04
Equal-offset shard fragments retain arrival order.
Equal-offset shard fragments retain arrival order · case 05
Equal-offset shard fragments retain arrival order.
Unknown worker IDs inflate a replica average · case 01
Unknown worker IDs inflate a replica average.
Unknown worker IDs inflate a replica average · case 02
Unknown worker IDs inflate a replica average.
Unknown worker IDs inflate a replica average · case 03
Unknown worker IDs inflate a replica average.
Unknown worker IDs inflate a replica average · case 04
Unknown worker IDs inflate a replica average.
Unknown worker IDs inflate a replica average · case 05
Unknown worker IDs inflate a replica average.
L2 normalization uses the squared magnitude as divisor · case 01
L2 normalization uses the squared magnitude as divisor.
L2 normalization uses the squared magnitude as divisor · case 02
L2 normalization uses the squared magnitude as divisor.
L2 normalization uses the squared magnitude as divisor · case 03
L2 normalization uses the squared magnitude as divisor.
L2 normalization uses the squared magnitude as divisor · case 04
L2 normalization uses the squared magnitude as divisor.
L2 normalization uses the squared magnitude as divisor · case 05
L2 normalization uses the squared magnitude as divisor.
L1 normalization divides by the largest coordinate · case 01
L1 normalization divides by the largest coordinate.
L1 normalization divides by the largest coordinate · case 02
L1 normalization divides by the largest coordinate.
L1 normalization divides by the largest coordinate · case 03
L1 normalization divides by the largest coordinate.
L1 normalization divides by the largest coordinate · case 04
L1 normalization divides by the largest coordinate.
L1 normalization divides by the largest coordinate · case 05
L1 normalization divides by the largest coordinate.
Infinity normalization divides by total magnitude · case 01
Infinity normalization divides by total magnitude.
Infinity normalization divides by total magnitude · case 02
Infinity normalization divides by total magnitude.
Infinity normalization divides by total magnitude · case 03
Infinity normalization divides by total magnitude.
Infinity normalization divides by total magnitude · case 04
Infinity normalization divides by total magnitude.
Infinity normalization divides by total magnitude · case 05
Infinity normalization divides by total magnitude.
Zero embeddings become a fabricated unit vector · case 01
Zero embeddings become a fabricated unit vector.
Zero embeddings become a fabricated unit vector · case 02
Zero embeddings become a fabricated unit vector.
Zero embeddings become a fabricated unit vector · case 03
Zero embeddings become a fabricated unit vector.
Zero embeddings become a fabricated unit vector · case 04
Zero embeddings become a fabricated unit vector.
Zero embeddings become a fabricated unit vector · case 05
Zero embeddings become a fabricated unit vector.
Vector centering divides by the signed coordinate sum · case 01
Vector centering divides by the signed coordinate sum.
Vector centering divides by the signed coordinate sum · case 02
Vector centering divides by the signed coordinate sum.
Vector centering divides by the signed coordinate sum · case 03
Vector centering divides by the signed coordinate sum.
Vector centering divides by the signed coordinate sum · case 04
Vector centering divides by the signed coordinate sum.
Vector centering divides by the signed coordinate sum · case 05
Vector centering divides by the signed coordinate sum.
Weighted pooling divides by token count · case 01
Weighted pooling divides by token count.
Weighted pooling divides by token count · case 02
Weighted pooling divides by token count.
Weighted pooling divides by token count · case 03
Weighted pooling divides by token count.
Weighted pooling divides by token count · case 04
Weighted pooling divides by token count.
Weighted pooling divides by token count · case 05
Weighted pooling divides by token count.
Masked mean pooling counts masked positions · case 01
Masked mean pooling counts masked positions.
Masked mean pooling counts masked positions · case 02
Masked mean pooling counts masked positions.
Masked mean pooling counts masked positions · case 03
Masked mean pooling counts masked positions.
Masked mean pooling counts masked positions · case 04
Masked mean pooling counts masked positions.
Masked mean pooling counts masked positions · case 05
Masked mean pooling counts masked positions.
Length-normalized pooling divides by squared token count · case 01
Length-normalized pooling divides by squared token count.
Length-normalized pooling divides by squared token count · case 02
Length-normalized pooling divides by squared token count.
Length-normalized pooling divides by squared token count · case 03
Length-normalized pooling divides by squared token count.
Length-normalized pooling divides by squared token count · case 04
Length-normalized pooling divides by squared token count.
Length-normalized pooling divides by squared token count · case 05
Length-normalized pooling divides by squared token count.
Square-root length pooling divides by squared token count · case 01
Square-root length pooling divides by squared token count.
Square-root length pooling divides by squared token count · case 02
Square-root length pooling divides by squared token count.
Square-root length pooling divides by squared token count · case 03
Square-root length pooling divides by squared token count.
Square-root length pooling divides by squared token count · case 04
Square-root length pooling divides by squared token count.
Square-root length pooling divides by squared token count · case 05
Square-root length pooling divides by squared token count.
Weighted pooling normalizes token vectors before combining them · case 01
Weighted pooling normalizes token vectors before combining them.
Weighted pooling normalizes token vectors before combining them · case 02
Weighted pooling normalizes token vectors before combining them.
Weighted pooling normalizes token vectors before combining them · case 03
Weighted pooling normalizes token vectors before combining them.
Weighted pooling normalizes token vectors before combining them · case 04
Weighted pooling normalizes token vectors before combining them.
Weighted pooling normalizes token vectors before combining them · case 05
Weighted pooling normalizes token vectors before combining them.
Gradient style clipping expands a vector already under the norm bound · case 01
Gradient style clipping expands a vector already under the norm bound.
Gradient style clipping expands a vector already under the norm bound · case 02
Gradient style clipping expands a vector already under the norm bound.
Gradient style clipping expands a vector already under the norm bound · case 03
Gradient style clipping expands a vector already under the norm bound.
Gradient style clipping expands a vector already under the norm bound · case 04
Gradient style clipping expands a vector already under the norm bound.
Gradient style clipping expands a vector already under the norm bound · case 05
Gradient style clipping expands a vector already under the norm bound.
Epsilon normalization adds epsilon to every coordinate divisor · case 01
Epsilon normalization adds epsilon to every coordinate divisor.
Epsilon normalization adds epsilon to every coordinate divisor · case 02
Epsilon normalization adds epsilon to every coordinate divisor.
Epsilon normalization adds epsilon to every coordinate divisor · case 03
Epsilon normalization adds epsilon to every coordinate divisor.
Epsilon normalization adds epsilon to every coordinate divisor · case 04
Epsilon normalization adds epsilon to every coordinate divisor.
Epsilon normalization adds epsilon to every coordinate divisor · case 05
Epsilon normalization adds epsilon to every coordinate divisor.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗