Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
The ten is ranked below the king in the matador sequence · case 01
Runs through A-10-K are broken at the wrong card.
The ten is ranked below the king in the matador sequence · case 02
Runs through A-10-K are broken at the wrong card.
The ten is ranked below the king in the matador sequence · case 03
Runs through A-10-K are broken at the wrong card.
The ten is ranked below the king in the matador sequence · case 04
Runs through A-10-K are broken at the wrong card.
The ten is ranked below the king in the matador sequence · case 05
Runs through A-10-K are broken at the wrong card.
The game level is not added to the multiplier · case 01
Every game value is one base short.
The game level is not added to the multiplier · case 02
Every game value is one base short.
The game level is not added to the multiplier · case 03
Every game value is one base short.
The game level is not added to the multiplier · case 04
Every game value is one base short.
The game level is not added to the multiplier · case 05
Every game value is one base short.
Schwarz without an explicit schneider flag skips the schneider level · case 01
A schwarz game is valued one level short.
Schwarz without an explicit schneider flag skips the schneider level · case 02
A schwarz game is valued one level short.
Schwarz without an explicit schneider flag skips the schneider level · case 03
A schwarz game is valued one level short.
Schwarz without an explicit schneider flag skips the schneider level · case 04
A schwarz game is valued one level short.
Schwarz without an explicit schneider flag skips the schneider level · case 05
A schwarz game is valued one level short.
Melds with four or more wild cards are accepted · case 01
A pair of fives with four wilds is a legal meld.
Melds with four or more wild cards are accepted · case 02
A pair of fives with four wilds is a legal meld.
Melds with four or more wild cards are accepted · case 03
A pair of fives with four wilds is a legal meld.
Melds with four or more wild cards are accepted · case 04
A pair of fives with four wilds is a legal meld.
Melds with four or more wild cards are accepted · case 05
A pair of fives with four wilds is a legal meld.
Mixed canastas earn the natural bonus · case 01
A canasta containing wild cards scores 500.
Mixed canastas earn the natural bonus · case 02
A canasta containing wild cards scores 500.
Mixed canastas earn the natural bonus · case 03
A canasta containing wild cards scores 500.
Mixed canastas earn the natural bonus · case 04
A canasta containing wild cards scores 500.
Mixed canastas earn the natural bonus · case 05
A canasta containing wild cards scores 500.
Threes can be melded · case 01
A meld of threes is accepted.
Threes can be melded · case 02
A meld of threes is accepted.
Threes can be melded · case 03
A meld of threes is accepted.
Threes can be melded · case 04
A meld of threes is accepted.
Threes can be melded · case 05
A meld of threes is accepted.
Deuces count five points · case 01
Wild twos are undervalued in meld points.
Deuces count five points · case 02
Wild twos are undervalued in meld points.
Deuces count five points · case 03
Wild twos are undervalued in meld points.
Deuces count five points · case 04
Wild twos are undervalued in meld points.
Deuces count five points · case 05
Wild twos are undervalued in meld points.
A meld with one natural card is accepted · case 01
One king plus two deuces is laid down as a meld.
A meld with one natural card is accepted · case 02
One king plus two deuces is laid down as a meld.
A meld with one natural card is accepted · case 03
One king plus two deuces is laid down as a meld.
A meld with one natural card is accepted · case 04
One king plus two deuces is laid down as a meld.
A meld with one natural card is accepted · case 05
One king plus two deuces is laid down as a meld.
Tennis game ends on a one-point lead after deuce · case 01
After deuce the display jumps straight to a game instead of showing advantage.
Tennis game ends on a one-point lead after deuce · case 02
After deuce the display jumps straight to a game instead of showing advantage.
Tennis game ends on a one-point lead after deuce · case 03
After deuce the display jumps straight to a game instead of showing advantage.
Tennis game ends on a one-point lead after deuce · case 04
After deuce the display jumps straight to a game instead of showing advantage.
Tennis game ends on a one-point lead after deuce · case 05
After deuce the display jumps straight to a game instead of showing advantage.
Tennis game awarded at forty with a two-point lead · case 01
A player leading 40-15 is shown as having won the game one point early.
Tennis game awarded at forty with a two-point lead · case 02
A player leading 40-15 is shown as having won the game one point early.
Tennis game awarded at forty with a two-point lead · case 03
A player leading 40-15 is shown as having won the game one point early.
Tennis game awarded at forty with a two-point lead · case 04
A player leading 40-15 is shown as having won the game one point early.
Tennis game awarded at forty with a two-point lead · case 05
A player leading 40-15 is shown as having won the game one point early.
Advantage shown before both players reach forty · case 01
A 40-15 score is rendered as an advantage instead of the numeric score.
Advantage shown before both players reach forty · case 02
A 40-15 score is rendered as an advantage instead of the numeric score.
Advantage shown before both players reach forty · case 03
A 40-15 score is rendered as an advantage instead of the numeric score.
Advantage shown before both players reach forty · case 04
A 40-15 score is rendered as an advantage instead of the numeric score.
Advantage shown before both players reach forty · case 05
A 40-15 score is rendered as an advantage instead of the numeric score.
Rally log validation ignores the end-of-game cutoff · case 01
Corrupt characters are silently skipped or trailing garbage after a finished game is reported as an error.
Rally log validation ignores the end-of-game cutoff · case 02
Corrupt characters are silently skipped or trailing garbage after a finished game is reported as an error.
Rally log validation ignores the end-of-game cutoff · case 03
Corrupt characters are silently skipped or trailing garbage after a finished game is reported as an error.
Rally log validation ignores the end-of-game cutoff · case 04
Corrupt characters are silently skipped or trailing garbage after a finished game is reported as an error.
Rally log validation ignores the end-of-game cutoff · case 05
Corrupt characters are silently skipped or trailing garbage after a finished game is reported as an error.
Tiebreak closes on a one-point lead · case 01
A tiebreak at 7-6 is reported as won instead of continuing.
Tiebreak closes on a one-point lead · case 02
A tiebreak at 7-6 is reported as won instead of continuing.
Tiebreak closes on a one-point lead · case 03
A tiebreak at 7-6 is reported as won instead of continuing.
Tiebreak closes on a one-point lead · case 04
A tiebreak at 7-6 is reported as won instead of continuing.
Tiebreak closes on a one-point lead · case 05
A tiebreak at 7-6 is reported as won instead of continuing.
Tiebreak target compared with a strict inequality · case 01
A 7-5 tiebreak is not closed and play continues to eight points.
Tiebreak target compared with a strict inequality · case 02
A 7-5 tiebreak is not closed and play continues to eight points.
Tiebreak target compared with a strict inequality · case 03
A 7-5 tiebreak is not closed and play continues to eight points.
Tiebreak target compared with a strict inequality · case 04
A 7-5 tiebreak is not closed and play continues to eight points.
Tiebreak target compared with a strict inequality · case 05
A 7-5 tiebreak is not closed and play continues to eight points.
Tiebreak server rotation is off by one point · case 01
The server indicator shows the wrong player after the first point of a tiebreak.
Tiebreak server rotation is off by one point · case 02
The server indicator shows the wrong player after the first point of a tiebreak.
Tiebreak server rotation is off by one point · case 03
The server indicator shows the wrong player after the first point of a tiebreak.
Tiebreak server rotation is off by one point · case 04
The server indicator shows the wrong player after the first point of a tiebreak.
Tiebreak server rotation is off by one point · case 05
The server indicator shows the wrong player after the first point of a tiebreak.
Points recorded after the tiebreak ended are still counted · case 01
A log that contains trailing warm-up points changes the final tiebreak score.
Points recorded after the tiebreak ended are still counted · case 02
A log that contains trailing warm-up points changes the final tiebreak score.
Points recorded after the tiebreak ended are still counted · case 03
A log that contains trailing warm-up points changes the final tiebreak score.
Points recorded after the tiebreak ended are still counted · case 04
A log that contains trailing warm-up points changes the final tiebreak score.
Points recorded after the tiebreak ended are still counted · case 05
A log that contains trailing warm-up points changes the final tiebreak score.
Change of ends prompted at the start or after the tiebreak ended · case 01
The umpire is told to change ends before any point is played or after a 7-5 finish.
Change of ends prompted at the start or after the tiebreak ended · case 02
The umpire is told to change ends before any point is played or after a 7-5 finish.
Change of ends prompted at the start or after the tiebreak ended · case 03
The umpire is told to change ends before any point is played or after a 7-5 finish.
Change of ends prompted at the start or after the tiebreak ended · case 04
The umpire is told to change ends before any point is played or after a 7-5 finish.
Change of ends prompted at the start or after the tiebreak ended · case 05
The umpire is told to change ends before any point is played or after a 7-5 finish.
Set at six-all continues as an advantage set · case 01
A set that went to a tiebreak is reported as 8-6 or longer instead of 7-6.
Set at six-all continues as an advantage set · case 02
A set that went to a tiebreak is reported as 8-6 or longer instead of 7-6.
Set at six-all continues as an advantage set · case 03
A set that went to a tiebreak is reported as 8-6 or longer instead of 7-6.
Set at six-all continues as an advantage set · case 04
A set that went to a tiebreak is reported as 8-6 or longer instead of 7-6.
Set at six-all continues as an advantage set · case 05
A set that went to a tiebreak is reported as 8-6 or longer instead of 7-6.
Best-of-five match decided after the wrong number of sets · case 01
A best-of-five match is declared over while a player has only two sets.
Best-of-five match decided after the wrong number of sets · case 02
A best-of-five match is declared over while a player has only two sets.
Best-of-five match decided after the wrong number of sets · case 03
A best-of-five match is declared over while a player has only two sets.
Best-of-five match decided after the wrong number of sets · case 04
A best-of-five match is declared over while a player has only two sets.
Best-of-five match decided after the wrong number of sets · case 05
A best-of-five match is declared over while a player has only two sets.
Current set score dropped from or padded into the set list · case 01
An in-progress set is missing from the scoreboard, or a phantom "0-0" set appears after a set ends.
Current set score dropped from or padded into the set list · case 02
An in-progress set is missing from the scoreboard, or a phantom "0-0" set appears after a set ends.
Current set score dropped from or padded into the set list · case 03
An in-progress set is missing from the scoreboard, or a phantom "0-0" set appears after a set ends.
Current set score dropped from or padded into the set list · case 04
An in-progress set is missing from the scoreboard, or a phantom "0-0" set appears after a set ends.
Current set score dropped from or padded into the set list · case 05
An in-progress set is missing from the scoreboard, or a phantom "0-0" set appears after a set ends.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗