Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Output stopping: A stop token in the prompt prematurely ends generated output · case 01
A stop token in the prompt prematurely ends generated output.
Output stopping: A stop token in the prompt prematurely ends generated output · case 02
A stop token in the prompt prematurely ends generated output.
Output stopping: A stop token in the prompt prematurely ends generated output · case 03
A stop token in the prompt prematurely ends generated output.
Output stopping: A stop token in the prompt prematurely ends generated output · case 04
A stop token in the prompt prematurely ends generated output.
Output stopping: A stop token in the prompt prematurely ends generated output · case 05
A stop token in the prompt prematurely ends generated output.
Output stopping: Any satisfied stopping criterion ends output · case 01
Any satisfied stopping criterion ends output.
Output stopping: Any satisfied stopping criterion ends output · case 02
Any satisfied stopping criterion ends output.
Output stopping: Any satisfied stopping criterion ends output · case 03
Any satisfied stopping criterion ends output.
Output stopping: Any satisfied stopping criterion ends output · case 04
Any satisfied stopping criterion ends output.
Output stopping: Any satisfied stopping criterion ends output · case 05
Any satisfied stopping criterion ends output.
Output stopping: One sample’s finish state changes a neighboring sample · case 01
One sample’s finish state changes a neighboring sample.
Output stopping: One sample’s finish state changes a neighboring sample · case 02
One sample’s finish state changes a neighboring sample.
Output stopping: One sample’s finish state changes a neighboring sample · case 03
One sample’s finish state changes a neighboring sample.
Output stopping: One sample’s finish state changes a neighboring sample · case 04
One sample’s finish state changes a neighboring sample.
Output stopping: One sample’s finish state changes a neighboring sample · case 05
One sample’s finish state changes a neighboring sample.
Output stopping: An ordinary end marker is reported as a length finish · case 01
An ordinary end marker is reported as a length finish.
Output stopping: An ordinary end marker is reported as a length finish · case 02
An ordinary end marker is reported as a length finish.
Output stopping: An ordinary end marker is reported as a length finish · case 03
An ordinary end marker is reported as a length finish.
Output stopping: An ordinary end marker is reported as a length finish · case 04
An ordinary end marker is reported as a length finish.
Output stopping: An ordinary end marker is reported as a length finish · case 05
An ordinary end marker is reported as a length finish.
Output stopping: A full token budget is not reported as a length finish · case 01
A full token budget is not reported as a length finish.
Output stopping: A full token budget is not reported as a length finish · case 02
A full token budget is not reported as a length finish.
Output stopping: A full token budget is not reported as a length finish · case 03
A full token budget is not reported as a length finish.
Output stopping: A full token budget is not reported as a length finish · case 04
A full token budget is not reported as a length finish.
Output stopping: A full token budget is not reported as a length finish · case 05
A full token budget is not reported as a length finish.
Output stopping: Repeated stop matches return the final occurrence · case 01
Repeated stop matches return the final occurrence.
Output stopping: Repeated stop matches return the final occurrence · case 02
Repeated stop matches return the final occurrence.
Output stopping: Repeated stop matches return the final occurrence · case 03
Repeated stop matches return the final occurrence.
Output stopping: Repeated stop matches return the final occurrence · case 04
Repeated stop matches return the final occurrence.
Output stopping: Repeated stop matches return the final occurrence · case 05
Repeated stop matches return the final occurrence.
Output stopping: Empty stop list incorrectly stops every output · case 01
Empty stop list incorrectly stops every output.
Output stopping: Empty stop list incorrectly stops every output · case 02
Empty stop list incorrectly stops every output.
Output stopping: Empty stop list incorrectly stops every output · case 03
Empty stop list incorrectly stops every output.
Output stopping: Empty stop list incorrectly stops every output · case 04
Empty stop list incorrectly stops every output.
Output stopping: Empty stop list incorrectly stops every output · case 05
Empty stop list incorrectly stops every output.
Output stopping: Streaming suffix retains only one character of a longer possible stop prefix · case 01
Streaming suffix retains only one character of a longer possible stop prefix.
Output stopping: Streaming suffix retains only one character of a longer possible stop prefix · case 02
Streaming suffix retains only one character of a longer possible stop prefix.
Output stopping: Streaming suffix retains only one character of a longer possible stop prefix · case 03
Streaming suffix retains only one character of a longer possible stop prefix.
Output stopping: Streaming suffix retains only one character of a longer possible stop prefix · case 04
Streaming suffix retains only one character of a longer possible stop prefix.
Output stopping: Streaming suffix retains only one character of a longer possible stop prefix · case 05
Streaming suffix retains only one character of a longer possible stop prefix.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗