Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Opponent tiebreak drops repeat encounters · case 01
Opponent tiebreak drops repeat encounters.
Opponent tiebreak drops repeat encounters · case 02
Opponent tiebreak drops repeat encounters.
Opponent tiebreak drops repeat encounters · case 03
Opponent tiebreak drops repeat encounters.
Opponent tiebreak drops repeat encounters · case 04
Opponent tiebreak drops repeat encounters.
Opponent tiebreak drops repeat encounters · case 05
Opponent tiebreak drops repeat encounters.
Cut tiebreak removes every tied minimum · case 01
Cut tiebreak removes every tied minimum.
Cut tiebreak removes every tied minimum · case 02
Cut tiebreak removes every tied minimum.
Cut tiebreak removes every tied minimum · case 03
Cut tiebreak removes every tied minimum.
Cut tiebreak removes every tied minimum · case 04
Cut tiebreak removes every tied minimum.
Cut tiebreak removes every tied minimum · case 05
Cut tiebreak removes every tied minimum.
Incomplete mini-league produces a head-to-head ranking · case 01
Incomplete mini-league produces a head-to-head ranking.
Incomplete mini-league produces a head-to-head ranking · case 02
Incomplete mini-league produces a head-to-head ranking.
Incomplete mini-league produces a head-to-head ranking · case 03
Incomplete mini-league produces a head-to-head ranking.
Incomplete mini-league produces a head-to-head ranking · case 04
Incomplete mini-league produces a head-to-head ranking.
Incomplete mini-league produces a head-to-head ranking · case 05
Incomplete mini-league produces a head-to-head ranking.
Unplayed results are credited as draws · case 01
Unplayed results are credited as draws.
Unplayed results are credited as draws · case 02
Unplayed results are credited as draws.
Unplayed results are credited as draws · case 03
Unplayed results are credited as draws.
Unplayed results are credited as draws · case 04
Unplayed results are credited as draws.
Unplayed results are credited as draws · case 05
Unplayed results are credited as draws.
Adjacent seed placement makes top seeds meet early · case 01
Adjacent seed placement makes top seeds meet early.
Adjacent seed placement makes top seeds meet early · case 02
Adjacent seed placement makes top seeds meet early.
Adjacent seed placement makes top seeds meet early · case 03
Adjacent seed placement makes top seeds meet early.
Adjacent seed placement makes top seeds meet early · case 04
Adjacent seed placement makes top seeds meet early.
Adjacent seed placement makes top seeds meet early · case 05
Adjacent seed placement makes top seeds meet early.
Withdrawn opponents disappear from historical tiebreaks · case 01
Withdrawn opponents disappear from historical tiebreaks.
Withdrawn opponents disappear from historical tiebreaks · case 02
Withdrawn opponents disappear from historical tiebreaks.
Withdrawn opponents disappear from historical tiebreaks · case 03
Withdrawn opponents disappear from historical tiebreaks.
Withdrawn opponents disappear from historical tiebreaks · case 04
Withdrawn opponents disappear from historical tiebreaks.
Withdrawn opponents disappear from historical tiebreaks · case 05
Withdrawn opponents disappear from historical tiebreaks.
Pair orientation checks total balance but misses streak bans · case 01
Pair orientation checks total balance but misses streak bans.
Pair orientation checks total balance but misses streak bans · case 02
Pair orientation checks total balance but misses streak bans.
Pair orientation checks total balance but misses streak bans · case 03
Pair orientation checks total balance but misses streak bans.
Pair orientation checks total balance but misses streak bans · case 04
Pair orientation checks total balance but misses streak bans.
Pair orientation checks total balance but misses streak bans · case 05
Pair orientation checks total balance but misses streak bans.
Disqualification shifts a bracket replacement into the wrong slot · case 01
Disqualification shifts a bracket replacement into the wrong slot.
Disqualification shifts a bracket replacement into the wrong slot · case 02
Disqualification shifts a bracket replacement into the wrong slot.
Disqualification shifts a bracket replacement into the wrong slot · case 03
Disqualification shifts a bracket replacement into the wrong slot.
Disqualification shifts a bracket replacement into the wrong slot · case 04
Disqualification shifts a bracket replacement into the wrong slot.
Disqualification shifts a bracket replacement into the wrong slot · case 05
Disqualification shifts a bracket replacement into the wrong slot.
Signed channel totals: Positive and negative channels are collapsed before reporting. · case 01
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Positive and negative channels are collapsed before reporting. · case 02
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Positive and negative channels are collapsed before reporting. · case 03
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Positive and negative channels are collapsed before reporting. · case 04
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Positive and negative channels are collapsed before reporting. · case 05
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The negative subtotal loses its sign. · case 01
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The negative subtotal loses its sign. · case 02
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The negative subtotal loses its sign. · case 03
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The negative subtotal loses its sign. · case 04
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The negative subtotal loses its sign. · case 05
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Equal contributions are deduplicated before channel reduction. · case 01
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Equal contributions are deduplicated before channel reduction. · case 02
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Equal contributions are deduplicated before channel reduction. · case 03
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Equal contributions are deduplicated before channel reduction. · case 04
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Equal contributions are deduplicated before channel reduction. · case 05
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Zero presence is reported instead of zero multiplicity. · case 01
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Zero presence is reported instead of zero multiplicity. · case 02
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Zero presence is reported instead of zero multiplicity. · case 03
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Zero presence is reported instead of zero multiplicity. · case 04
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: Zero presence is reported instead of zero multiplicity. · case 05
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The channel accumulator is overwritten by its final contribution. · case 01
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The channel accumulator is overwritten by its final contribution. · case 02
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The channel accumulator is overwritten by its final contribution. · case 03
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The channel accumulator is overwritten by its final contribution. · case 04
The reduction disagrees with its explicit aggregation oracle.
Signed channel totals: The channel accumulator is overwritten by its final contribution. · case 05
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Absolute value is applied after cancellation of signed deviations. · case 01
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Absolute value is applied after cancellation of signed deviations. · case 02
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Absolute value is applied after cancellation of signed deviations. · case 03
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Absolute value is applied after cancellation of signed deviations. · case 04
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Absolute value is applied after cancellation of signed deviations. · case 05
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The anchor is subtracted once from the total instead of once per observation. · case 01
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The anchor is subtracted once from the total instead of once per observation. · case 02
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The anchor is subtracted once from the total instead of once per observation. · case 03
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The anchor is subtracted once from the total instead of once per observation. · case 04
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The anchor is subtracted once from the total instead of once per observation. · case 05
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Contributions below the anchor are discarded. · case 01
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Contributions below the anchor are discarded. · case 02
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Contributions below the anchor are discarded. · case 03
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Contributions below the anchor are discarded. · case 04
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Contributions below the anchor are discarded. · case 05
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The supplied anchor is replaced by the sample mean. · case 01
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The supplied anchor is replaced by the sample mean. · case 02
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The supplied anchor is replaced by the sample mean. · case 03
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The supplied anchor is replaced by the sample mean. · case 04
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: The supplied anchor is replaced by the sample mean. · case 05
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Only extreme observations contribute to total deviation. · case 01
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Only extreme observations contribute to total deviation. · case 02
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Only extreme observations contribute to total deviation. · case 03
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Only extreme observations contribute to total deviation. · case 04
The reduction disagrees with its explicit aggregation oracle.
Absolute deviation about anchor: Only extreme observations contribute to total deviation. · case 05
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Clipping happens after aggregation rather than before each contribution. · case 01
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Clipping happens after aggregation rather than before each contribution. · case 02
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Clipping happens after aggregation rather than before each contribution. · case 03
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Clipping happens after aggregation rather than before each contribution. · case 04
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Clipping happens after aggregation rather than before each contribution. · case 05
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Outliers are dropped instead of contributing their clipped endpoint. · case 01
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Outliers are dropped instead of contributing their clipped endpoint. · case 02
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Outliers are dropped instead of contributing their clipped endpoint. · case 03
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Outliers are dropped instead of contributing their clipped endpoint. · case 04
The reduction disagrees with its explicit aggregation oracle.
Clipped contribution sum: Outliers are dropped instead of contributing their clipped endpoint. · case 05
The reduction disagrees with its explicit aggregation oracle.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗