Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Arrival order wins capacity over stronger routes · case 01
Arrival order wins capacity over stronger routes.
Arrival order wins capacity over stronger routes · case 02
Arrival order wins capacity over stronger routes.
Arrival order wins capacity over stronger routes · case 03
Arrival order wins capacity over stronger routes.
Arrival order wins capacity over stronger routes · case 04
Arrival order wins capacity over stronger routes.
Arrival order wins capacity over stronger routes · case 05
Arrival order wins capacity over stronger routes.
Router scores are paired with the wrong expert IDs · case 01
Router scores are paired with the wrong expert IDs.
Router scores are paired with the wrong expert IDs · case 02
Router scores are paired with the wrong expert IDs.
Router scores are paired with the wrong expert IDs · case 03
Router scores are paired with the wrong expert IDs.
Router scores are paired with the wrong expert IDs · case 04
Router scores are paired with the wrong expert IDs.
Router scores are paired with the wrong expert IDs · case 05
Router scores are paired with the wrong expert IDs.
Dropped token IDs vanish from the dispatch result · case 01
Dropped token IDs vanish from the dispatch result.
Dropped token IDs vanish from the dispatch result · case 02
Dropped token IDs vanish from the dispatch result.
Dropped token IDs vanish from the dispatch result · case 03
Dropped token IDs vanish from the dispatch result.
Dropped token IDs vanish from the dispatch result · case 04
Dropped token IDs vanish from the dispatch result.
Dropped token IDs vanish from the dispatch result · case 05
Dropped token IDs vanish from the dispatch result.
Eligibility filtering runs after router top-k truncation · case 01
Eligibility filtering runs after router top-k truncation.
Eligibility filtering runs after router top-k truncation · case 02
Eligibility filtering runs after router top-k truncation.
Eligibility filtering runs after router top-k truncation · case 03
Eligibility filtering runs after router top-k truncation.
Eligibility filtering runs after router top-k truncation · case 04
Eligibility filtering runs after router top-k truncation.
Eligibility filtering runs after router top-k truncation · case 05
Eligibility filtering runs after router top-k truncation.
Combine weights are normalized across unrelated tokens · case 01
Combine weights are normalized across unrelated tokens.
Combine weights are normalized across unrelated tokens · case 02
Combine weights are normalized across unrelated tokens.
Combine weights are normalized across unrelated tokens · case 03
Combine weights are normalized across unrelated tokens.
Combine weights are normalized across unrelated tokens · case 04
Combine weights are normalized across unrelated tokens.
Combine weights are normalized across unrelated tokens · case 05
Combine weights are normalized across unrelated tokens.
Repeated expert contributions are not merged per token · case 01
Repeated expert contributions are not merged per token.
Repeated expert contributions are not merged per token · case 02
Repeated expert contributions are not merged per token.
Repeated expert contributions are not merged per token · case 03
Repeated expert contributions are not merged per token.
Repeated expert contributions are not merged per token · case 04
Repeated expert contributions are not merged per token.
Repeated expert contributions are not merged per token · case 05
Repeated expert contributions are not merged per token.
Tokens with too few available routes are partially dispatched · case 01
Tokens with too few available routes are partially dispatched.
Tokens with too few available routes are partially dispatched · case 02
Tokens with too few available routes are partially dispatched.
Tokens with too few available routes are partially dispatched · case 03
Tokens with too few available routes are partially dispatched.
Tokens with too few available routes are partially dispatched · case 04
Tokens with too few available routes are partially dispatched.
Tokens with too few available routes are partially dispatched · case 05
Tokens with too few available routes are partially dispatched.
Extreme router logits bypass the declared score bound · case 01
Extreme router logits bypass the declared score bound.
Extreme router logits bypass the declared score bound · case 02
Extreme router logits bypass the declared score bound.
Extreme router logits bypass the declared score bound · case 03
Extreme router logits bypass the declared score bound.
Extreme router logits bypass the declared score bound · case 04
Extreme router logits bypass the declared score bound.
Extreme router logits bypass the declared score bound · case 05
Extreme router logits bypass the declared score bound.
Decoding: Acceptance count includes tokens after the first mismatch · case 01
Acceptance count includes tokens after the first mismatch.
Decoding: Acceptance count includes tokens after the first mismatch · case 02
Acceptance count includes tokens after the first mismatch.
Decoding: Acceptance count includes tokens after the first mismatch · case 03
Acceptance count includes tokens after the first mismatch.
Decoding: Acceptance count includes tokens after the first mismatch · case 04
Acceptance count includes tokens after the first mismatch.
Decoding: Acceptance count includes tokens after the first mismatch · case 05
Acceptance count includes tokens after the first mismatch.
Decoding: Rejection location is reported as a one-based position · case 01
Rejection location is reported as a one-based position.
Decoding: Rejection location is reported as a one-based position · case 02
Rejection location is reported as a one-based position.
Decoding: Rejection location is reported as a one-based position · case 03
Rejection location is reported as a one-based position.
Decoding: Rejection location is reported as a one-based position · case 04
Rejection location is reported as a one-based position.
Decoding: Rejection location is reported as a one-based position · case 05
Rejection location is reported as a one-based position.
Decoding: Rejected draft tokens inflate the accepted token total · case 01
Rejected draft tokens inflate the accepted token total.
Decoding: Rejected draft tokens inflate the accepted token total · case 02
Rejected draft tokens inflate the accepted token total.
Decoding: Rejected draft tokens inflate the accepted token total · case 03
Rejected draft tokens inflate the accepted token total.
Decoding: Rejected draft tokens inflate the accepted token total · case 04
Rejected draft tokens inflate the accepted token total.
Decoding: Rejected draft tokens inflate the accepted token total · case 05
Rejected draft tokens inflate the accepted token total.
Decoding: Replacement token is inserted after the rejected draft token · case 01
Replacement token is inserted after the rejected draft token.
Decoding: Replacement token is inserted after the rejected draft token · case 02
Replacement token is inserted after the rejected draft token.
Decoding: Replacement token is inserted after the rejected draft token · case 03
Replacement token is inserted after the rejected draft token.
Decoding: Replacement token is inserted after the rejected draft token · case 04
Replacement token is inserted after the rejected draft token.
Decoding: Replacement token is inserted after the rejected draft token · case 05
Replacement token is inserted after the rejected draft token.
Decoding: Rollback removes only the final draft token · case 01
Rollback removes only the final draft token.
Decoding: Rollback removes only the final draft token · case 02
Rollback removes only the final draft token.
Decoding: Rollback removes only the final draft token · case 03
Rollback removes only the final draft token.
Decoding: Rollback removes only the final draft token · case 04
Rollback removes only the final draft token.
Decoding: Rollback removes only the final draft token · case 05
Rollback removes only the final draft token.
Decoding: Draft positions without verifier results are counted as covered · case 01
Draft positions without verifier results are counted as covered.
Decoding: Draft positions without verifier results are counted as covered · case 02
Draft positions without verifier results are counted as covered.
Decoding: Draft positions without verifier results are counted as covered · case 03
Draft positions without verifier results are counted as covered.
Decoding: Draft positions without verifier results are counted as covered · case 04
Draft positions without verifier results are counted as covered.
Decoding: Draft positions without verifier results are counted as covered · case 05
Draft positions without verifier results are counted as covered.
Decoding: Bonus token is emitted when only part of the draft was accepted · case 01
Bonus token is emitted when only part of the draft was accepted.
Decoding: Bonus token is emitted when only part of the draft was accepted · case 02
Bonus token is emitted when only part of the draft was accepted.
Decoding: Bonus token is emitted when only part of the draft was accepted · case 03
Bonus token is emitted when only part of the draft was accepted.
Decoding: Bonus token is emitted when only part of the draft was accepted · case 04
Bonus token is emitted when only part of the draft was accepted.
Decoding: Bonus token is emitted when only part of the draft was accepted · case 05
Bonus token is emitted when only part of the draft was accepted.
Decoding: Replacement token is missing from the emitted-token count · case 01
Replacement token is missing from the emitted-token count.
Decoding: Replacement token is missing from the emitted-token count · case 02
Replacement token is missing from the emitted-token count.
Decoding: Replacement token is missing from the emitted-token count · case 03
Replacement token is missing from the emitted-token count.
Decoding: Replacement token is missing from the emitted-token count · case 04
Replacement token is missing from the emitted-token count.
Decoding: Replacement token is missing from the emitted-token count · case 05
Replacement token is missing from the emitted-token count.
Decoding: Proposal contains one token beyond the verifier window · case 01
Proposal contains one token beyond the verifier window.
Decoding: Proposal contains one token beyond the verifier window · case 02
Proposal contains one token beyond the verifier window.
Decoding: Proposal contains one token beyond the verifier window · case 03
Proposal contains one token beyond the verifier window.
Decoding: Proposal contains one token beyond the verifier window · case 04
Proposal contains one token beyond the verifier window.
Decoding: Proposal contains one token beyond the verifier window · case 05
Proposal contains one token beyond the verifier window.
Decoding: Target evaluation count omits the rejected token comparison · case 01
Target evaluation count omits the rejected token comparison.
Decoding: Target evaluation count omits the rejected token comparison · case 02
Target evaluation count omits the rejected token comparison.
Decoding: Target evaluation count omits the rejected token comparison · case 03
Target evaluation count omits the rejected token comparison.
Decoding: Target evaluation count omits the rejected token comparison · case 04
Target evaluation count omits the rejected token comparison.
Decoding: Target evaluation count omits the rejected token comparison · case 05
Target evaluation count omits the rejected token comparison.
Decoding: Layered cache commit preserves only positions before rejection · case 01
Layered cache commit preserves only positions before rejection.
Decoding: Layered cache commit preserves only positions before rejection · case 02
Layered cache commit preserves only positions before rejection.
Decoding: Layered cache commit preserves only positions before rejection · case 03
Layered cache commit preserves only positions before rejection.
Decoding: Layered cache commit preserves only positions before rejection · case 04
Layered cache commit preserves only positions before rejection.
Decoding: Layered cache commit preserves only positions before rejection · case 05
Layered cache commit preserves only positions before rejection.
Decoding: Absolute position omits the replacement token after rejection · case 01
Absolute position omits the replacement token after rejection.
Decoding: Absolute position omits the replacement token after rejection · case 02
Absolute position omits the replacement token after rejection.
Decoding: Absolute position omits the replacement token after rejection · case 03
Absolute position omits the replacement token after rejection.
Decoding: Absolute position omits the replacement token after rejection · case 04
Absolute position omits the replacement token after rejection.
Decoding: Absolute position omits the replacement token after rejection · case 05
Absolute position omits the replacement token after rejection.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗