Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Causal position does not reset at segment boundaries · case 01
Causal position does not reset at segment boundaries.
Causal position does not reset at segment boundaries · case 02
Causal position does not reset at segment boundaries.
Causal position does not reset at segment boundaries · case 03
Causal position does not reset at segment boundaries.
Causal position does not reset at segment boundaries · case 04
Causal position does not reset at segment boundaries.
Causal position does not reset at segment boundaries · case 05
Causal position does not reset at segment boundaries.
Block-causal mask lets keys cross supplied block boundaries · case 01
Block-causal mask lets keys cross supplied block boundaries.
Block-causal mask lets keys cross supplied block boundaries · case 02
Block-causal mask lets keys cross supplied block boundaries.
Block-causal mask lets keys cross supplied block boundaries · case 03
Block-causal mask lets keys cross supplied block boundaries.
Block-causal mask lets keys cross supplied block boundaries · case 04
Block-causal mask lets keys cross supplied block boundaries.
Block-causal mask lets keys cross supplied block boundaries · case 05
Block-causal mask lets keys cross supplied block boundaries.
Sliding window stops after the initial query rows · case 01
Sliding window stops after the initial query rows.
Sliding window stops after the initial query rows · case 02
Sliding window stops after the initial query rows.
Sliding window stops after the initial query rows · case 03
Sliding window stops after the initial query rows.
Sliding window stops after the initial query rows · case 04
Sliding window stops after the initial query rows.
Sliding window stops after the initial query rows · case 05
Sliding window stops after the initial query rows.
Bidirectional local attention omits future neighbors · case 01
Bidirectional local attention omits future neighbors.
Bidirectional local attention omits future neighbors · case 02
Bidirectional local attention omits future neighbors.
Bidirectional local attention omits future neighbors · case 03
Bidirectional local attention omits future neighbors.
Bidirectional local attention omits future neighbors · case 04
Bidirectional local attention omits future neighbors.
Bidirectional local attention omits future neighbors · case 05
Bidirectional local attention omits future neighbors.
Causal stripe mask includes keys from every residue · case 01
Causal stripe mask includes keys from every residue.
Causal stripe mask includes keys from every residue · case 02
Causal stripe mask includes keys from every residue.
Causal stripe mask includes keys from every residue · case 03
Causal stripe mask includes keys from every residue.
Causal stripe mask includes keys from every residue · case 04
Causal stripe mask includes keys from every residue.
Causal stripe mask includes keys from every residue · case 05
Causal stripe mask includes keys from every residue.
Global keys remain hidden from earlier queries · case 01
Global keys remain hidden from earlier queries.
Global keys remain hidden from earlier queries · case 02
Global keys remain hidden from earlier queries.
Global keys remain hidden from earlier queries · case 03
Global keys remain hidden from earlier queries.
Global keys remain hidden from earlier queries · case 04
Global keys remain hidden from earlier queries.
Global keys remain hidden from earlier queries · case 05
Global keys remain hidden from earlier queries.
Block sparse mask attends to non-neighbor blocks · case 01
Block sparse mask attends to non-neighbor blocks.
Block sparse mask attends to non-neighbor blocks · case 02
Block sparse mask attends to non-neighbor blocks.
Block sparse mask attends to non-neighbor blocks · case 03
Block sparse mask attends to non-neighbor blocks.
Block sparse mask attends to non-neighbor blocks · case 04
Block sparse mask attends to non-neighbor blocks.
Block sparse mask attends to non-neighbor blocks · case 05
Block sparse mask attends to non-neighbor blocks.
Cross-attention mask omits the source interval limit · case 01
Cross-attention mask omits the source interval limit.
Cross-attention mask omits the source interval limit · case 02
Cross-attention mask omits the source interval limit.
Cross-attention mask omits the source interval limit · case 03
Cross-attention mask omits the source interval limit.
Cross-attention mask omits the source interval limit · case 04
Cross-attention mask omits the source interval limit.
Cross-attention mask omits the source interval limit · case 05
Cross-attention mask omits the source interval limit.
The final input token has no next-token target · case 01
The final input token has no next-token target.
The final input token has no next-token target · case 02
The final input token has no next-token target.
The final input token has no next-token target · case 03
The final input token has no next-token target.
The final input token has no next-token target · case 04
The final input token has no next-token target.
The final input token has no next-token target · case 05
The final input token has no next-token target.
Truncation removes a retained target at the cutoff · case 01
Truncation removes a retained target at the cutoff.
Truncation removes a retained target at the cutoff · case 02
Truncation removes a retained target at the cutoff.
Truncation removes a retained target at the cutoff · case 03
Truncation removes a retained target at the cutoff.
Truncation removes a retained target at the cutoff · case 04
Truncation removes a retained target at the cutoff.
Truncation removes a retained target at the cutoff · case 05
Truncation removes a retained target at the cutoff.
Start marker context is copied as its own target · case 01
Start marker context is copied as its own target.
Start marker context is copied as its own target · case 02
Start marker context is copied as its own target.
Start marker context is copied as its own target · case 03
Start marker context is copied as its own target.
Start marker context is copied as its own target · case 04
Start marker context is copied as its own target.
Start marker context is copied as its own target · case 05
Start marker context is copied as its own target.
The first observed token is paired with itself after start insertion · case 01
The first observed token is paired with itself after start insertion.
The first observed token is paired with itself after start insertion · case 02
The first observed token is paired with itself after start insertion.
The first observed token is paired with itself after start insertion · case 03
The first observed token is paired with itself after start insertion.
The first observed token is paired with itself after start insertion · case 04
The first observed token is paired with itself after start insertion.
The first observed token is paired with itself after start insertion · case 05
The first observed token is paired with itself after start insertion.
End marker is omitted from next-token supervision · case 01
End marker is omitted from next-token supervision.
End marker is omitted from next-token supervision · case 02
End marker is omitted from next-token supervision.
End marker is omitted from next-token supervision · case 03
End marker is omitted from next-token supervision.
End marker is omitted from next-token supervision · case 04
End marker is omitted from next-token supervision.
End marker is omitted from next-token supervision · case 05
End marker is omitted from next-token supervision.
The end marker receives a spurious outgoing prediction · case 01
The end marker receives a spurious outgoing prediction.
The end marker receives a spurious outgoing prediction · case 02
The end marker receives a spurious outgoing prediction.
The end marker receives a spurious outgoing prediction · case 03
The end marker receives a spurious outgoing prediction.
The end marker receives a spurious outgoing prediction · case 04
The end marker receives a spurious outgoing prediction.
The end marker receives a spurious outgoing prediction · case 05
The end marker receives a spurious outgoing prediction.
Ignored target labels mask the following position · case 01
Ignored target labels mask the following position.
Ignored target labels mask the following position · case 02
Ignored target labels mask the following position.
Ignored target labels mask the following position · case 03
Ignored target labels mask the following position.
Ignored target labels mask the following position · case 04
Ignored target labels mask the following position.
Ignored target labels mask the following position · case 05
Ignored target labels mask the following position.
The final target mask is dropped with the input slice · case 01
The final target mask is dropped with the input slice.
The final target mask is dropped with the input slice · case 02
The final target mask is dropped with the input slice.
The final target mask is dropped with the input slice · case 03
The final target mask is dropped with the input slice.
The final target mask is dropped with the input slice · case 04
The final target mask is dropped with the input slice.
The final target mask is dropped with the input slice · case 05
The final target mask is dropped with the input slice.
A context window predicts each token from itself · case 01
A context window predicts each token from itself.
A context window predicts each token from itself · case 02
A context window predicts each token from itself.
A context window predicts each token from itself · case 03
A context window predicts each token from itself.
A context window predicts each token from itself · case 04
A context window predicts each token from itself.
A context window predicts each token from itself · case 05
A context window predicts each token from itself.
The final in-window context token is omitted · case 01
The final in-window context token is omitted.
The final in-window context token is omitted · case 02
The final in-window context token is omitted.
The final in-window context token is omitted · case 03
The final in-window context token is omitted.
The final in-window context token is omitted · case 04
The final in-window context token is omitted.
The final in-window context token is omitted · case 05
The final in-window context token is omitted.
A supplied target prefix is aligned one token late · case 01
A supplied target prefix is aligned one token late.
A supplied target prefix is aligned one token late · case 02
A supplied target prefix is aligned one token late.
A supplied target prefix is aligned one token late · case 03
A supplied target prefix is aligned one token late.
A supplied target prefix is aligned one token late · case 04
A supplied target prefix is aligned one token late.
A supplied target prefix is aligned one token late · case 05
A supplied target prefix is aligned one token late.
A longer target prefix repeats the final context in extra pairs · case 01
A longer target prefix repeats the final context in extra pairs.
A longer target prefix repeats the final context in extra pairs · case 02
A longer target prefix repeats the final context in extra pairs.
A longer target prefix repeats the final context in extra pairs · case 03
A longer target prefix repeats the final context in extra pairs.
A longer target prefix repeats the final context in extra pairs · case 04
A longer target prefix repeats the final context in extra pairs.
A longer target prefix repeats the final context in extra pairs · case 05
A longer target prefix repeats the final context in extra pairs.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗