Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Merged chunk offsets ignore inserted separator lengths · case 01
Merged chunk offsets ignore inserted separator lengths.
Merged chunk offsets ignore inserted separator lengths · case 02
Merged chunk offsets ignore inserted separator lengths.
Merged chunk offsets ignore inserted separator lengths · case 03
Merged chunk offsets ignore inserted separator lengths.
Merged chunk offsets ignore inserted separator lengths · case 04
Merged chunk offsets ignore inserted separator lengths.
Merged chunk offsets ignore inserted separator lengths · case 05
Merged chunk offsets ignore inserted separator lengths.
A citation in joined text is attributed to the wrong chunk · case 01
A citation in joined text is attributed to the wrong chunk.
A citation in joined text is attributed to the wrong chunk · case 02
A citation in joined text is attributed to the wrong chunk.
A citation in joined text is attributed to the wrong chunk · case 03
A citation in joined text is attributed to the wrong chunk.
A citation in joined text is attributed to the wrong chunk · case 04
A citation in joined text is attributed to the wrong chunk.
A citation in joined text is attributed to the wrong chunk · case 05
A citation in joined text is attributed to the wrong chunk.
Repeated source IDs create duplicate bibliography entries · case 01
Repeated source IDs create duplicate bibliography entries.
Repeated source IDs create duplicate bibliography entries · case 02
Repeated source IDs create duplicate bibliography entries.
Repeated source IDs create duplicate bibliography entries · case 03
Repeated source IDs create duplicate bibliography entries.
Repeated source IDs create duplicate bibliography entries · case 04
Repeated source IDs create duplicate bibliography entries.
Repeated source IDs create duplicate bibliography entries · case 05
Repeated source IDs create duplicate bibliography entries.
Chunk citation references remain local after bibliography compaction · case 01
Chunk citation references remain local after bibliography compaction.
Chunk citation references remain local after bibliography compaction · case 02
Chunk citation references remain local after bibliography compaction.
Chunk citation references remain local after bibliography compaction · case 03
Chunk citation references remain local after bibliography compaction.
Chunk citation references remain local after bibliography compaction · case 04
Chunk citation references remain local after bibliography compaction.
Chunk citation references remain local after bibliography compaction · case 05
Chunk citation references remain local after bibliography compaction.
An obsolete source revision wins citation resolution · case 01
An obsolete source revision wins citation resolution.
An obsolete source revision wins citation resolution · case 02
An obsolete source revision wins citation resolution.
An obsolete source revision wins citation resolution · case 03
An obsolete source revision wins citation resolution.
An obsolete source revision wins citation resolution · case 04
An obsolete source revision wins citation resolution.
An obsolete source revision wins citation resolution · case 05
An obsolete source revision wins citation resolution.
Citations after an insertion retain stale offsets · case 01
Citations after an insertion retain stale offsets.
Citations after an insertion retain stale offsets · case 02
Citations after an insertion retain stale offsets.
Citations after an insertion retain stale offsets · case 03
Citations after an insertion retain stale offsets.
Citations after an insertion retain stale offsets · case 04
Citations after an insertion retain stale offsets.
Citations after an insertion retain stale offsets · case 05
Citations after an insertion retain stale offsets.
Citations crossing deleted text keep an invalid range · case 01
Citations crossing deleted text keep an invalid range.
Citations crossing deleted text keep an invalid range · case 02
Citations crossing deleted text keep an invalid range.
Citations crossing deleted text keep an invalid range · case 03
Citations crossing deleted text keep an invalid range.
Citations crossing deleted text keep an invalid range · case 04
Citations crossing deleted text keep an invalid range.
Citations crossing deleted text keep an invalid range · case 05
Citations crossing deleted text keep an invalid range.
Line citations count the chunk header as source text · case 01
Line citations count the chunk header as source text.
Line citations count the chunk header as source text · case 02
Line citations count the chunk header as source text.
Line citations count the chunk header as source text · case 03
Line citations count the chunk header as source text.
Line citations count the chunk header as source text · case 04
Line citations count the chunk header as source text.
Line citations count the chunk header as source text · case 05
Line citations count the chunk header as source text.
Required property presence is confused with truthiness · case 01
Required property presence is confused with truthiness.
Required property presence is confused with truthiness · case 02
Required property presence is confused with truthiness.
Required property presence is confused with truthiness · case 03
Required property presence is confused with truthiness.
Required property presence is confused with truthiness · case 04
Required property presence is confused with truthiness.
Required property presence is confused with truthiness · case 05
Required property presence is confused with truthiness.
Pattern-matched properties are rejected as unknown extras · case 01
Pattern-matched properties are rejected as unknown extras.
Pattern-matched properties are rejected as unknown extras · case 02
Pattern-matched properties are rejected as unknown extras.
Pattern-matched properties are rejected as unknown extras · case 03
Pattern-matched properties are rejected as unknown extras.
Pattern-matched properties are rejected as unknown extras · case 04
Pattern-matched properties are rejected as unknown extras.
Pattern-matched properties are rejected as unknown extras · case 05
Pattern-matched properties are rejected as unknown extras.
Integer arguments are accepted as booleans · case 01
Integer arguments are accepted as booleans.
Integer arguments are accepted as booleans · case 02
Integer arguments are accepted as booleans.
Integer arguments are accepted as booleans · case 03
Integer arguments are accepted as booleans.
Integer arguments are accepted as booleans · case 04
Integer arguments are accepted as booleans.
Integer arguments are accepted as booleans · case 05
Integer arguments are accepted as booleans.
Boolean values pass an integer field · case 01
Boolean values pass an integer field.
Boolean values pass an integer field · case 02
Boolean values pass an integer field.
Boolean values pass an integer field · case 03
Boolean values pass an integer field.
Boolean values pass an integer field · case 04
Boolean values pass an integer field.
Boolean values pass an integer field · case 05
Boolean values pass an integer field.
Enum membership ignores JSON value type · case 01
Enum membership ignores JSON value type.
Enum membership ignores JSON value type · case 02
Enum membership ignores JSON value type.
Enum membership ignores JSON value type · case 03
Enum membership ignores JSON value type.
Enum membership ignores JSON value type · case 04
Enum membership ignores JSON value type.
Enum membership ignores JSON value type · case 05
Enum membership ignores JSON value type.
Inclusive minimum rejects equality · case 01
Inclusive minimum rejects equality.
Inclusive minimum rejects equality · case 02
Inclusive minimum rejects equality.
Inclusive minimum rejects equality · case 03
Inclusive minimum rejects equality.
Inclusive minimum rejects equality · case 04
Inclusive minimum rejects equality.
Inclusive minimum rejects equality · case 05
Inclusive minimum rejects equality.
Inclusive maximum accepts out-of-range value · case 01
Inclusive maximum accepts out-of-range value.
Inclusive maximum accepts out-of-range value · case 02
Inclusive maximum accepts out-of-range value.
Inclusive maximum accepts out-of-range value · case 03
Inclusive maximum accepts out-of-range value.
Inclusive maximum accepts out-of-range value · case 04
Inclusive maximum accepts out-of-range value.
Inclusive maximum accepts out-of-range value · case 05
Inclusive maximum accepts out-of-range value.
Exclusive bound is applied without checking its schema mode · case 01
Exclusive bound is applied without checking its schema mode.
Exclusive bound is applied without checking its schema mode · case 02
Exclusive bound is applied without checking its schema mode.
Exclusive bound is applied without checking its schema mode · case 03
Exclusive bound is applied without checking its schema mode.
Exclusive bound is applied without checking its schema mode · case 04
Exclusive bound is applied without checking its schema mode.
Exclusive bound is applied without checking its schema mode · case 05
Exclusive bound is applied without checking its schema mode.
String length uses UTF-8 bytes instead of characters · case 01
String length uses UTF-8 bytes instead of characters.
String length uses UTF-8 bytes instead of characters · case 02
String length uses UTF-8 bytes instead of characters.
String length uses UTF-8 bytes instead of characters · case 03
String length uses UTF-8 bytes instead of characters.
String length uses UTF-8 bytes instead of characters · case 04
String length uses UTF-8 bytes instead of characters.
String length uses UTF-8 bytes instead of characters · case 05
String length uses UTF-8 bytes instead of characters.
Invalid array elements escape item validation · case 01
Invalid array elements escape item validation.
Invalid array elements escape item validation · case 02
Invalid array elements escape item validation.
Invalid array elements escape item validation · case 03
Invalid array elements escape item validation.
Invalid array elements escape item validation · case 04
Invalid array elements escape item validation.
Invalid array elements escape item validation · case 05
Invalid array elements escape item validation.
Duplicate array values pass a uniqueness constraint · case 01
Duplicate array values pass a uniqueness constraint.
Duplicate array values pass a uniqueness constraint · case 02
Duplicate array values pass a uniqueness constraint.
Duplicate array values pass a uniqueness constraint · case 03
Duplicate array values pass a uniqueness constraint.
Duplicate array values pass a uniqueness constraint · case 04
Duplicate array values pass a uniqueness constraint.
Duplicate array values pass a uniqueness constraint · case 05
Duplicate array values pass a uniqueness constraint.
Tuple schema accepts an extra positional item · case 01
Tuple schema accepts an extra positional item.
Tuple schema accepts an extra positional item · case 02
Tuple schema accepts an extra positional item.
Tuple schema accepts an extra positional item · case 03
Tuple schema accepts an extra positional item.
Tuple schema accepts an extra positional item · case 04
Tuple schema accepts an extra positional item.
Tuple schema accepts an extra positional item · case 05
Tuple schema accepts an extra positional item.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗