Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Visual run list for shaping: lowest odd pass · case 01
Runs at the lowest odd level stay in logical order.
Visual run list for shaping: lowest odd pass · case 02
Runs at the lowest odd level stay in logical order.
Visual run list for shaping: lowest odd pass · case 03
Runs at the lowest odd level stay in logical order.
Visual run list for shaping: lowest odd pass · case 04
Runs at the lowest odd level stay in logical order.
Visual run list for shaping: lowest odd pass · case 05
Runs at the lowest odd level stay in logical order.
Visual run list for shaping: run sequence reversal · case 01
Visual runs come out in logical order.
Visual run list for shaping: run sequence reversal · case 02
Visual runs come out in logical order.
Visual run list for shaping: run sequence reversal · case 03
Visual runs come out in logical order.
Visual run list for shaping: run sequence reversal · case 04
Visual runs come out in logical order.
Visual run list for shaping: run sequence reversal · case 05
Visual runs come out in logical order.
Paragraph separation and direction: CRLF pairing · case 01
Windows line endings create an extra empty paragraph.
Paragraph separation and direction: CRLF pairing · case 02
Windows line endings create an extra empty paragraph.
Paragraph separation and direction: CRLF pairing · case 03
Windows line endings create an extra empty paragraph.
Paragraph separation and direction: CRLF pairing · case 04
Windows line endings create an extra empty paragraph.
Paragraph separation and direction: CRLF pairing · case 05
Windows line endings create an extra empty paragraph.
Paragraph separation and direction: separator ownership · case 01
Paragraph separators appear at the start of the following paragraph.
Paragraph separation and direction: separator ownership · case 02
Paragraph separators appear at the start of the following paragraph.
Paragraph separation and direction: separator ownership · case 03
Paragraph separators appear at the start of the following paragraph.
Paragraph separation and direction: separator ownership · case 04
Paragraph separators appear at the start of the following paragraph.
Paragraph separation and direction: separator ownership · case 05
Paragraph separators appear at the start of the following paragraph.
Paragraph separation and direction: per-paragraph first strong · case 01
All paragraphs get the direction of the first paragraph.
Paragraph separation and direction: per-paragraph first strong · case 02
All paragraphs get the direction of the first paragraph.
Paragraph separation and direction: per-paragraph first strong · case 03
All paragraphs get the direction of the first paragraph.
Paragraph separation and direction: per-paragraph first strong · case 04
All paragraphs get the direction of the first paragraph.
Paragraph separation and direction: per-paragraph first strong · case 05
All paragraphs get the direction of the first paragraph.
Paragraph separation and direction: trailing empty paragraph · case 01
Text ending with a separator gains a phantom empty paragraph.
Paragraph separation and direction: trailing empty paragraph · case 02
Text ending with a separator gains a phantom empty paragraph.
Paragraph separation and direction: trailing empty paragraph · case 03
Text ending with a separator gains a phantom empty paragraph.
Paragraph separation and direction: trailing empty paragraph · case 04
Text ending with a separator gains a phantom empty paragraph.
Paragraph separation and direction: trailing empty paragraph · case 05
Text ending with a separator gains a phantom empty paragraph.
Paragraph separation and direction: default level per paragraph · case 01
A paragraph with no letters inherits the previous paragraph direction.
Paragraph separation and direction: default level per paragraph · case 02
A paragraph with no letters inherits the previous paragraph direction.
Paragraph separation and direction: default level per paragraph · case 03
A paragraph with no letters inherits the previous paragraph direction.
Paragraph separation and direction: default level per paragraph · case 04
A paragraph with no letters inherits the previous paragraph direction.
Paragraph separation and direction: default level per paragraph · case 05
A paragraph with no letters inherits the previous paragraph direction.
Paragraph separation and direction: strong character test · case 01
Paragraphs starting with digits or punctuation are forced LTR.
Paragraph separation and direction: strong character test · case 02
Paragraphs starting with digits or punctuation are forced LTR.
Paragraph separation and direction: strong character test · case 03
Paragraphs starting with digits or punctuation are forced LTR.
Paragraph separation and direction: strong character test · case 04
Paragraphs starting with digits or punctuation are forced LTR.
Paragraph separation and direction: strong character test · case 05
Paragraphs starting with digits or punctuation are forced LTR.
Spelled pitch to MIDI key number: octave numbered from zero at middle C minus four · case 01
Every spelled pitch lands exactly one octave low, so C4 reports 48 and A4 reports 57.
Spelled pitch to MIDI key number: octave numbered from zero at middle C minus four · case 02
Every spelled pitch lands exactly one octave low, so C4 reports 48 and A4 reports 57.
Spelled pitch to MIDI key number: octave numbered from zero at middle C minus four · case 03
Every spelled pitch lands exactly one octave low, so C4 reports 48 and A4 reports 57.
Spelled pitch to MIDI key number: octave numbered from zero at middle C minus four · case 04
Every spelled pitch lands exactly one octave low, so C4 reports 48 and A4 reports 57.
Spelled pitch to MIDI key number: octave numbered from zero at middle C minus four · case 05
Every spelled pitch lands exactly one octave low, so C4 reports 48 and A4 reports 57.
Spelled pitch to MIDI key number: letters treated as evenly spaced whole tones · case 01
F, G, A and B read one semitone sharp (F4 reports 66) while C, D and E stay correct.
Spelled pitch to MIDI key number: letters treated as evenly spaced whole tones · case 02
F, G, A and B read one semitone sharp (F4 reports 66) while C, D and E stay correct.
Spelled pitch to MIDI key number: letters treated as evenly spaced whole tones · case 03
F, G, A and B read one semitone sharp (F4 reports 66) while C, D and E stay correct.
Spelled pitch to MIDI key number: letters treated as evenly spaced whole tones · case 04
F, G, A and B read one semitone sharp (F4 reports 66) while C, D and E stay correct.
Spelled pitch to MIDI key number: letters treated as evenly spaced whole tones · case 05
F, G, A and B read one semitone sharp (F4 reports 66) while C, D and E stay correct.
Spelled pitch to MIDI key number: octave minus one rejected as malformed · case 01
Every pitch in octave -1 (C-1 through B-1) is rejected even though keys 0..11 are valid.
Spelled pitch to MIDI key number: octave minus one rejected as malformed · case 02
Every pitch in octave -1 (C-1 through B-1) is rejected even though keys 0..11 are valid.
Spelled pitch to MIDI key number: octave minus one rejected as malformed · case 03
Every pitch in octave -1 (C-1 through B-1) is rejected even though keys 0..11 are valid.
Spelled pitch to MIDI key number: octave minus one rejected as malformed · case 04
Every pitch in octave -1 (C-1 through B-1) is rejected even though keys 0..11 are valid.
Spelled pitch to MIDI key number: octave minus one rejected as malformed · case 05
Every pitch in octave -1 (C-1 through B-1) is rejected even though keys 0..11 are valid.
Spelled pitch to MIDI key number: range check admits key 128 · case 01
G#9 and Ab9 are returned as key 128 instead of being rejected as out of range.
Spelled pitch to MIDI key number: range check admits key 128 · case 02
G#9 and Ab9 are returned as key 128 instead of being rejected as out of range.
Spelled pitch to MIDI key number: range check admits key 128 · case 03
G#9 and Ab9 are returned as key 128 instead of being rejected as out of range.
Spelled pitch to MIDI key number: range check admits key 128 · case 04
G#9 and Ab9 are returned as key 128 instead of being rejected as out of range.
Spelled pitch to MIDI key number: range check admits key 128 · case 05
G#9 and Ab9 are returned as key 128 instead of being rejected as out of range.
Spelled pitch to MIDI key number: double sharp glyph not consumed · case 01
Any spelling with an x double sharp (Fx4, Cx-1) is rejected as malformed.
Spelled pitch to MIDI key number: double sharp glyph not consumed · case 02
Any spelling with an x double sharp (Fx4, Cx-1) is rejected as malformed.
Spelled pitch to MIDI key number: double sharp glyph not consumed · case 03
Any spelling with an x double sharp (Fx4, Cx-1) is rejected as malformed.
Spelled pitch to MIDI key number: double sharp glyph not consumed · case 04
Any spelling with an x double sharp (Fx4, Cx-1) is rejected as malformed.
Spelled pitch to MIDI key number: double sharp glyph not consumed · case 05
Any spelling with an x double sharp (Fx4, Cx-1) is rejected as malformed.
Key-aware MIDI note spelling: octave label one too high · case 01
Middle C is labelled C5 and every other key also reports an octave one too high.
Key-aware MIDI note spelling: octave label one too high · case 02
Middle C is labelled C5 and every other key also reports an octave one too high.
Key-aware MIDI note spelling: octave label one too high · case 03
Middle C is labelled C5 and every other key also reports an octave one too high.
Key-aware MIDI note spelling: octave label one too high · case 04
Middle C is labelled C5 and every other key also reports an octave one too high.
Key-aware MIDI note spelling: octave label one too high · case 05
Middle C is labelled C5 and every other key also reports an octave one too high.
Key-aware MIDI note spelling: C major chromatic notes spelled with flats · case 01
In a key with no sharps or flats, key 61 is spelled Db4 instead of the stipulated C#4.
Key-aware MIDI note spelling: C major chromatic notes spelled with flats · case 02
In a key with no sharps or flats, key 61 is spelled Db4 instead of the stipulated C#4.
Key-aware MIDI note spelling: C major chromatic notes spelled with flats · case 03
In a key with no sharps or flats, key 61 is spelled Db4 instead of the stipulated C#4.
Key-aware MIDI note spelling: C major chromatic notes spelled with flats · case 04
In a key with no sharps or flats, key 61 is spelled Db4 instead of the stipulated C#4.
Key-aware MIDI note spelling: C major chromatic notes spelled with flats · case 05
In a key with no sharps or flats, key 61 is spelled Db4 instead of the stipulated C#4.
Key-aware MIDI note spelling: seven-accidental keys rejected · case 01
C# major and Cb major (seven accidentals) are rejected although both are legal key signatures.
Key-aware MIDI note spelling: seven-accidental keys rejected · case 02
C# major and Cb major (seven accidentals) are rejected although both are legal key signatures.
Key-aware MIDI note spelling: seven-accidental keys rejected · case 03
C# major and Cb major (seven accidentals) are rejected although both are legal key signatures.
Key-aware MIDI note spelling: seven-accidental keys rejected · case 04
C# major and Cb major (seven accidentals) are rejected although both are legal key signatures.
Key-aware MIDI note spelling: seven-accidental keys rejected · case 05
C# major and Cb major (seven accidentals) are rejected although both are legal key signatures.
Key-aware MIDI note spelling: B sharp keeps the sounding octave number · case 01
In C# major, middle C is spelled B#4, an octave above the sounding pitch, instead of B#3.
Key-aware MIDI note spelling: B sharp keeps the sounding octave number · case 02
In C# major, middle C is spelled B#4, an octave above the sounding pitch, instead of B#3.
Key-aware MIDI note spelling: B sharp keeps the sounding octave number · case 03
In C# major, middle C is spelled B#4, an octave above the sounding pitch, instead of B#3.
Key-aware MIDI note spelling: B sharp keeps the sounding octave number · case 04
In C# major, middle C is spelled B#4, an octave above the sounding pitch, instead of B#3.
Key-aware MIDI note spelling: B sharp keeps the sounding octave number · case 05
In C# major, middle C is spelled B#4, an octave above the sounding pitch, instead of B#3.
Key-aware MIDI note spelling: E sharp spelling limited to seven sharps · case 01
In F# major, key 65 is spelled F4 instead of the stipulated E#4.
Key-aware MIDI note spelling: E sharp spelling limited to seven sharps · case 02
In F# major, key 65 is spelled F4 instead of the stipulated E#4.
Key-aware MIDI note spelling: E sharp spelling limited to seven sharps · case 03
In F# major, key 65 is spelled F4 instead of the stipulated E#4.
Key-aware MIDI note spelling: E sharp spelling limited to seven sharps · case 04
In F# major, key 65 is spelled F4 instead of the stipulated E#4.
Key-aware MIDI note spelling: E sharp spelling limited to seven sharps · case 05
In F# major, key 65 is spelled F4 instead of the stipulated E#4.
Interval quality and number naming: generic number counts letter gaps instead of letters · case 01
C4 to E4 is named AA2 instead of M3, because every generic number comes out one too small.
Interval quality and number naming: generic number counts letter gaps instead of letters · case 02
C4 to E4 is named AA2 instead of M3, because every generic number comes out one too small.
Interval quality and number naming: generic number counts letter gaps instead of letters · case 03
C4 to E4 is named AA2 instead of M3, because every generic number comes out one too small.
Interval quality and number naming: generic number counts letter gaps instead of letters · case 04
C4 to E4 is named AA2 instead of M3, because every generic number comes out one too small.
Interval quality and number naming: generic number counts letter gaps instead of letters · case 05
C4 to E4 is named AA2 instead of M3, because every generic number comes out one too small.
Interval quality and number naming: letter position taken from the sounding octave · case 01
Intervals from B#3 are measured as if the B were in octave 4, so B#3 to C4 is rejected as descending.
Interval quality and number naming: letter position taken from the sounding octave · case 02
Intervals from B#3 are measured as if the B were in octave 4, so B#3 to C4 is rejected as descending.
Interval quality and number naming: letter position taken from the sounding octave · case 03
Intervals from B#3 are measured as if the B were in octave 4, so B#3 to C4 is rejected as descending.
Interval quality and number naming: letter position taken from the sounding octave · case 04
Intervals from B#3 are measured as if the B were in octave 4, so B#3 to C4 is rejected as descending.
Interval quality and number naming: letter position taken from the sounding octave · case 05
Intervals from B#3 are measured as if the B were in octave 4, so B#3 to C4 is rejected as descending.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗