Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
An explicitly empty math alternative erases structure · case 01
An explicitly empty math alternative erases structure.
An explicitly empty math alternative erases structure · case 02
An explicitly empty math alternative erases structure.
An explicitly empty math alternative erases structure · case 03
An explicitly empty math alternative erases structure.
An explicitly empty math alternative erases structure · case 04
An explicitly empty math alternative erases structure.
An explicitly empty math alternative erases structure · case 05
An explicitly empty math alternative erases structure.
Math fraction operands are spoken in denominator-first order · case 01
Math fraction operands are spoken in denominator-first order.
Math fraction operands are spoken in denominator-first order · case 02
Math fraction operands are spoken in denominator-first order.
Math fraction operands are spoken in denominator-first order · case 03
Math fraction operands are spoken in denominator-first order.
Math fraction operands are spoken in denominator-first order · case 04
Math fraction operands are spoken in denominator-first order.
Math fraction operands are spoken in denominator-first order · case 05
Math fraction operands are spoken in denominator-first order.
Nested math fractions lose their closing boundary · case 01
Nested math fractions lose their closing boundary.
Nested math fractions lose their closing boundary · case 02
Nested math fractions lose their closing boundary.
Nested math fractions lose their closing boundary · case 03
Nested math fractions lose their closing boundary.
Nested math fractions lose their closing boundary · case 04
Nested math fractions lose their closing boundary.
Nested math fractions lose their closing boundary · case 05
Nested math fractions lose their closing boundary.
Math exponent and base are swapped in semantic output · case 01
Math exponent and base are swapped in semantic output.
Math exponent and base are swapped in semantic output · case 02
Math exponent and base are swapped in semantic output.
Math exponent and base are swapped in semantic output · case 03
Math exponent and base are swapped in semantic output.
Math exponent and base are swapped in semantic output · case 04
Math exponent and base are swapped in semantic output.
Math exponent and base are swapped in semantic output · case 05
Math exponent and base are swapped in semantic output.
Root expressions omit their semantic scope boundary · case 01
Root expressions omit their semantic scope boundary.
Root expressions omit their semantic scope boundary · case 02
Root expressions omit their semantic scope boundary.
Root expressions omit their semantic scope boundary · case 03
Root expressions omit their semantic scope boundary.
Root expressions omit their semantic scope boundary · case 04
Root expressions omit their semantic scope boundary.
Root expressions omit their semantic scope boundary · case 05
Root expressions omit their semantic scope boundary.
Fenced math expressions use an opening marker at both ends · case 01
Fenced math expressions use an opening marker at both ends.
Fenced math expressions use an opening marker at both ends · case 02
Fenced math expressions use an opening marker at both ends.
Fenced math expressions use an opening marker at both ends · case 03
Fenced math expressions use an opening marker at both ends.
Fenced math expressions use an opening marker at both ends · case 04
Fenced math expressions use an opening marker at both ends.
Fenced math expressions use an opening marker at both ends · case 05
Fenced math expressions use an opening marker at both ends.
Empty math nodes introduce extra semantic separators · case 01
Empty math nodes introduce extra semantic separators.
Empty math nodes introduce extra semantic separators · case 02
Empty math nodes introduce extra semantic separators.
Empty math nodes introduce extra semantic separators · case 03
Empty math nodes introduce extra semantic separators.
Empty math nodes introduce extra semantic separators · case 04
Empty math nodes introduce extra semantic separators.
Empty math nodes introduce extra semantic separators · case 05
Empty math nodes introduce extra semantic separators.
Unsupported math nodes are silently exposed as empty rows · case 01
Unsupported math nodes are silently exposed as empty rows.
Unsupported math nodes are silently exposed as empty rows · case 02
Unsupported math nodes are silently exposed as empty rows.
Unsupported math nodes are silently exposed as empty rows · case 03
Unsupported math nodes are silently exposed as empty rows.
Unsupported math nodes are silently exposed as empty rows · case 04
Unsupported math nodes are silently exposed as empty rows.
Unsupported math nodes are silently exposed as empty rows · case 05
Unsupported math nodes are silently exposed as empty rows.
Batch semantic reads use an outdated expected revision · case 01
Batch semantic reads use an outdated expected revision.
Batch semantic reads use an outdated expected revision · case 02
Batch semantic reads use an outdated expected revision.
Batch semantic reads use an outdated expected revision · case 03
Batch semantic reads use an outdated expected revision.
Batch semantic reads use an outdated expected revision · case 04
Batch semantic reads use an outdated expected revision.
Batch semantic reads use an outdated expected revision · case 05
Batch semantic reads use an outdated expected revision.
Batch accessibility property results are reordered by identifier · case 01
Batch accessibility property results are reordered by identifier.
Batch accessibility property results are reordered by identifier · case 02
Batch accessibility property results are reordered by identifier.
Batch accessibility property results are reordered by identifier · case 03
Batch accessibility property results are reordered by identifier.
Batch accessibility property results are reordered by identifier · case 04
Batch accessibility property results are reordered by identifier.
Batch accessibility property results are reordered by identifier · case 05
Batch accessibility property results are reordered by identifier.
Repeated property requests lose their positional results · case 01
Repeated property requests lose their positional results.
Repeated property requests lose their positional results · case 02
Repeated property requests lose their positional results.
Repeated property requests lose their positional results · case 03
Repeated property requests lose their positional results.
Repeated property requests lose their positional results · case 04
Repeated property requests lose their positional results.
Repeated property requests lose their positional results · case 05
Repeated property requests lose their positional results.
Defunct accessibility objects retain readable properties · case 01
Defunct accessibility objects retain readable properties.
Defunct accessibility objects retain readable properties · case 02
Defunct accessibility objects retain readable properties.
Defunct accessibility objects retain readable properties · case 03
Defunct accessibility objects retain readable properties.
Defunct accessibility objects retain readable properties · case 04
Defunct accessibility objects retain readable properties.
Defunct accessibility objects retain readable properties · case 05
Defunct accessibility objects retain readable properties.
Private backend properties are exposed through the batch API · case 01
Private backend properties are exposed through the batch API.
Private backend properties are exposed through the batch API · case 02
Private backend properties are exposed through the batch API.
Private backend properties are exposed through the batch API · case 03
Private backend properties are exposed through the batch API.
Private backend properties are exposed through the batch API · case 04
Private backend properties are exposed through the batch API.
Private backend properties are exposed through the batch API · case 05
Private backend properties are exposed through the batch API.
Absent semantic properties are treated as successful nulls · case 01
Absent semantic properties are treated as successful nulls.
Absent semantic properties are treated as successful nulls · case 02
Absent semantic properties are treated as successful nulls.
Absent semantic properties are treated as successful nulls · case 03
Absent semantic properties are treated as successful nulls.
Absent semantic properties are treated as successful nulls · case 04
Absent semantic properties are treated as successful nulls.
Absent semantic properties are treated as successful nulls · case 05
Absent semantic properties are treated as successful nulls.
Successful null properties are misreported as unsupported · case 01
Successful null properties are misreported as unsupported.
Successful null properties are misreported as unsupported · case 02
Successful null properties are misreported as unsupported.
Successful null properties are misreported as unsupported · case 03
Successful null properties are misreported as unsupported.
Successful null properties are misreported as unsupported · case 04
Successful null properties are misreported as unsupported.
Successful null properties are misreported as unsupported · case 05
Successful null properties are misreported as unsupported.
Empty semantic strings are coerced to null · case 01
Empty semantic strings are coerced to null.
Empty semantic strings are coerced to null · case 02
Empty semantic strings are coerced to null.
Empty semantic strings are coerced to null · case 03
Empty semantic strings are coerced to null.
Empty semantic strings are coerced to null · case 04
Empty semantic strings are coerced to null.
Empty semantic strings are coerced to null · case 05
Empty semantic strings are coerced to null.
Property batches report requested rather than current revision · case 01
Property batches report requested rather than current revision.
Property batches report requested rather than current revision · case 02
Property batches report requested rather than current revision.
Property batches report requested rather than current revision · case 03
Property batches report requested rather than current revision.
Property batches report requested rather than current revision · case 04
Property batches report requested rather than current revision.
Property batches report requested rather than current revision · case 05
Property batches report requested rather than current revision.
Missing semantic objects are mislabeled as unsupported properties · case 01
Missing semantic objects are mislabeled as unsupported properties.
Missing semantic objects are mislabeled as unsupported properties · case 02
Missing semantic objects are mislabeled as unsupported properties.
Missing semantic objects are mislabeled as unsupported properties · case 03
Missing semantic objects are mislabeled as unsupported properties.
Missing semantic objects are mislabeled as unsupported properties · case 04
Missing semantic objects are mislabeled as unsupported properties.
Missing semantic objects are mislabeled as unsupported properties · case 05
Missing semantic objects are mislabeled as unsupported properties.
Negative text offsets are clamped to the first visual line · case 01
Negative text offsets are clamped to the first visual line.
Negative text offsets are clamped to the first visual line · case 02
Negative text offsets are clamped to the first visual line.
Negative text offsets are clamped to the first visual line · case 03
Negative text offsets are clamped to the first visual line.
Negative text offsets are clamped to the first visual line · case 04
Negative text offsets are clamped to the first visual line.
Negative text offsets are clamped to the first visual line · case 05
Negative text offsets are clamped to the first visual line.
Out-of-range text offsets resolve to the final visual line · case 01
Out-of-range text offsets resolve to the final visual line.
Out-of-range text offsets resolve to the final visual line · case 02
Out-of-range text offsets resolve to the final visual line.
Out-of-range text offsets resolve to the final visual line · case 03
Out-of-range text offsets resolve to the final visual line.
Out-of-range text offsets resolve to the final visual line · case 04
Out-of-range text offsets resolve to the final visual line.
Out-of-range text offsets resolve to the final visual line · case 05
Out-of-range text offsets resolve to the final visual line.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗