Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Rename similarity pairing: a score equal to the threshold is not a rename · case 01
Files exactly at the configured similarity are shown as delete plus add.
Rename similarity pairing: a score equal to the threshold is not a rename · case 02
Files exactly at the configured similarity are shown as delete plus add.
Rename similarity pairing: a score equal to the threshold is not a rename · case 03
Files exactly at the configured similarity are shown as delete plus add.
Rename similarity pairing: a score equal to the threshold is not a rename · case 04
Files exactly at the configured similarity are shown as delete plus add.
Rename similarity pairing: a score equal to the threshold is not a rename · case 05
Files exactly at the configured similarity are shown as delete plus add.
Rename similarity pairing: one deleted file is renamed to several added files · case 01
A single deleted file appears as the source of multiple renames.
Rename similarity pairing: one deleted file is renamed to several added files · case 02
A single deleted file appears as the source of multiple renames.
Rename similarity pairing: one deleted file is renamed to several added files · case 03
A single deleted file appears as the source of multiple renames.
Rename similarity pairing: one deleted file is renamed to several added files · case 04
A single deleted file appears as the source of multiple renames.
Rename similarity pairing: one deleted file is renamed to several added files · case 05
A single deleted file appears as the source of multiple renames.
Rename similarity pairing: candidates are paired in path order instead of by score · case 01
A weaker pairing claims a file before its best match is considered.
Rename similarity pairing: candidates are paired in path order instead of by score · case 02
A weaker pairing claims a file before its best match is considered.
Rename similarity pairing: candidates are paired in path order instead of by score · case 03
A weaker pairing claims a file before its best match is considered.
Rename similarity pairing: candidates are paired in path order instead of by score · case 04
A weaker pairing claims a file before its best match is considered.
Rename similarity pairing: candidates are paired in path order instead of by score · case 05
A weaker pairing claims a file before its best match is considered.
Merge base selection: a commit is not treated as its own ancestor · case 01
When one branch contains the other, the merge base is reported as an older commit instead of the branch tip.
Merge base selection: a commit is not treated as its own ancestor · case 02
When one branch contains the other, the merge base is reported as an older commit instead of the branch tip.
Merge base selection: a commit is not treated as its own ancestor · case 03
When one branch contains the other, the merge base is reported as an older commit instead of the branch tip.
Merge base selection: a commit is not treated as its own ancestor · case 04
When one branch contains the other, the merge base is reported as an older commit instead of the branch tip.
Merge base selection: a commit is not treated as its own ancestor · case 05
When one branch contains the other, the merge base is reported as an older commit instead of the branch tip.
Merge base selection: traversal follows only first parents · case 01
History reachable only through a merge's second parent is ignored and the base is too old or missing.
Merge base selection: traversal follows only first parents · case 02
History reachable only through a merge's second parent is ignored and the base is too old or missing.
Merge base selection: traversal follows only first parents · case 03
History reachable only through a merge's second parent is ignored and the base is too old or missing.
Merge base selection: traversal follows only first parents · case 04
History reachable only through a merge's second parent is ignored and the base is too old or missing.
Merge base selection: traversal follows only first parents · case 05
History reachable only through a merge's second parent is ignored and the base is too old or missing.
Merge base selection: the oldest common ancestors are kept instead of the newest · case 01
The merge base is the root commit, so merges show every change since the beginning of history.
Merge base selection: the oldest common ancestors are kept instead of the newest · case 02
The merge base is the root commit, so merges show every change since the beginning of history.
Merge base selection: the oldest common ancestors are kept instead of the newest · case 03
The merge base is the root commit, so merges show every change since the beginning of history.
Merge base selection: the oldest common ancestors are kept instead of the newest · case 04
The merge base is the root commit, so merges show every change since the beginning of history.
Merge base selection: the oldest common ancestors are kept instead of the newest · case 05
The merge base is the root commit, so merges show every change since the beginning of history.
Merge base selection: only one of several merge bases is returned · case 01
Criss-cross merges pick an arbitrary base and can report spurious conflicts or silently revert changes.
Merge base selection: only one of several merge bases is returned · case 02
Criss-cross merges pick an arbitrary base and can report spurious conflicts or silently revert changes.
Merge base selection: only one of several merge bases is returned · case 03
Criss-cross merges pick an arbitrary base and can report spurious conflicts or silently revert changes.
Merge base selection: only one of several merge bases is returned · case 04
Criss-cross merges pick an arbitrary base and can report spurious conflicts or silently revert changes.
Merge base selection: only one of several merge bases is returned · case 05
Criss-cross merges pick an arbitrary base and can report spurious conflicts or silently revert changes.
Line mapping through hunks: an insertion after line s also shifts line s · case 01
A comment on the line just above an insertion jumps down by the inserted count.
Line mapping through hunks: an insertion after line s also shifts line s · case 02
A comment on the line just above an insertion jumps down by the inserted count.
Line mapping through hunks: an insertion after line s also shifts line s · case 03
A comment on the line just above an insertion jumps down by the inserted count.
Line mapping through hunks: an insertion after line s also shifts line s · case 04
A comment on the line just above an insertion jumps down by the inserted count.
Line mapping through hunks: an insertion after line s also shifts line s · case 05
A comment on the line just above an insertion jumps down by the inserted count.
Line mapping through hunks: the line after a changed range is reported as removed · case 01
A comment on the first unchanged line after a hunk is dropped.
Line mapping through hunks: the line after a changed range is reported as removed · case 02
A comment on the first unchanged line after a hunk is dropped.
Line mapping through hunks: the line after a changed range is reported as removed · case 03
A comment on the first unchanged line after a hunk is dropped.
Line mapping through hunks: the line after a changed range is reported as removed · case 04
A comment on the first unchanged line after a hunk is dropped.
Line mapping through hunks: the line after a changed range is reported as removed · case 05
A comment on the first unchanged line after a hunk is dropped.
Line mapping through hunks: hunks after a line are still examined · case 01
Lines before a later hunk are reported as removed.
Line mapping through hunks: hunks after a line are still examined · case 02
Lines before a later hunk are reported as removed.
Line mapping through hunks: hunks after a line are still examined · case 03
Lines before a later hunk are reported as removed.
Line mapping through hunks: hunks after a line are still examined · case 04
Lines before a later hunk are reported as removed.
Line mapping through hunks: hunks after a line are still examined · case 05
Lines before a later hunk are reported as removed.
Line mapping through hunks: the size change is applied with the wrong sign · case 01
Comments move up after insertions and down after deletions.
Line mapping through hunks: the size change is applied with the wrong sign · case 02
Comments move up after insertions and down after deletions.
Line mapping through hunks: the size change is applied with the wrong sign · case 03
Comments move up after insertions and down after deletions.
Line mapping through hunks: the size change is applied with the wrong sign · case 04
Comments move up after insertions and down after deletions.
Line mapping through hunks: the size change is applied with the wrong sign · case 05
Comments move up after insertions and down after deletions.
Three-way key-value merge: deleting a key and setting it to null are the same state · case 01
Setting a value to null is merged as a deletion, and a deleted key can come back as null.
Three-way key-value merge: deleting a key and setting it to null are the same state · case 02
Setting a value to null is merged as a deletion, and a deleted key can come back as null.
Three-way key-value merge: deleting a key and setting it to null are the same state · case 03
Setting a value to null is merged as a deletion, and a deleted key can come back as null.
Three-way key-value merge: deleting a key and setting it to null are the same state · case 04
Setting a value to null is merged as a deletion, and a deleted key can come back as null.
Three-way key-value merge: deleting a key and setting it to null are the same state · case 05
Setting a value to null is merged as a deletion, and a deleted key can come back as null.
Three-way key-value merge: matching edits on both branches are a conflict · case 01
Both branches set the same new value and the merge still reports a conflict.
Three-way key-value merge: matching edits on both branches are a conflict · case 02
Both branches set the same new value and the merge still reports a conflict.
Three-way key-value merge: matching edits on both branches are a conflict · case 03
Both branches set the same new value and the merge still reports a conflict.
Three-way key-value merge: matching edits on both branches are a conflict · case 04
Both branches set the same new value and the merge still reports a conflict.
Three-way key-value merge: matching edits on both branches are a conflict · case 05
Both branches set the same new value and the merge still reports a conflict.
Three-way key-value merge: a deletion silently beats a concurrent modification · case 01
A key modified on our branch and deleted on theirs disappears without a conflict.
Three-way key-value merge: a deletion silently beats a concurrent modification · case 02
A key modified on our branch and deleted on theirs disappears without a conflict.
Three-way key-value merge: a deletion silently beats a concurrent modification · case 03
A key modified on our branch and deleted on theirs disappears without a conflict.
Three-way key-value merge: a deletion silently beats a concurrent modification · case 04
A key modified on our branch and deleted on theirs disappears without a conflict.
Three-way key-value merge: a deletion silently beats a concurrent modification · case 05
A key modified on our branch and deleted on theirs disappears without a conflict.
Three-way key-value merge: keys added only by their branch are dropped · case 01
New settings introduced on the other branch are missing after the merge.
Three-way key-value merge: keys added only by their branch are dropped · case 02
New settings introduced on the other branch are missing after the merge.
Three-way key-value merge: keys added only by their branch are dropped · case 03
New settings introduced on the other branch are missing after the merge.
Three-way key-value merge: keys added only by their branch are dropped · case 04
New settings introduced on the other branch are missing after the merge.
Three-way key-value merge: keys added only by their branch are dropped · case 05
New settings introduced on the other branch are missing after the merge.
Intraline change pairing: the common suffix overlaps the common prefix · case 01
Highlighting marks negative-length changes when one line extends the other.
Intraline change pairing: the common suffix overlaps the common prefix · case 02
Highlighting marks negative-length changes when one line extends the other.
Intraline change pairing: the common suffix overlaps the common prefix · case 03
Highlighting marks negative-length changes when one line extends the other.
Intraline change pairing: the common suffix overlaps the common prefix · case 04
Highlighting marks negative-length changes when one line extends the other.
Intraline change pairing: the common suffix overlaps the common prefix · case 05
Highlighting marks negative-length changes when one line extends the other.
Intraline change pairing: removed lines pair with additions after intervening context · case 01
A deletion is highlighted against an unrelated insertion further down.
Intraline change pairing: removed lines pair with additions after intervening context · case 02
A deletion is highlighted against an unrelated insertion further down.
Intraline change pairing: removed lines pair with additions after intervening context · case 03
A deletion is highlighted against an unrelated insertion further down.
Intraline change pairing: removed lines pair with additions after intervening context · case 04
A deletion is highlighted against an unrelated insertion further down.
Intraline change pairing: removed lines pair with additions after intervening context · case 05
A deletion is highlighted against an unrelated insertion further down.
Intraline change pairing: unmatched lines are paired with empty strings · case 01
Extra removed or added lines produce spurious pairs against an empty line.
Intraline change pairing: unmatched lines are paired with empty strings · case 02
Extra removed or added lines produce spurious pairs against an empty line.
Intraline change pairing: unmatched lines are paired with empty strings · case 03
Extra removed or added lines produce spurious pairs against an empty line.
Intraline change pairing: unmatched lines are paired with empty strings · case 04
Extra removed or added lines produce spurious pairs against an empty line.
Intraline change pairing: unmatched lines are paired with empty strings · case 05
Extra removed or added lines produce spurious pairs against an empty line.
Hunk overlap check: adjacent hunks are treated as independent · case 01
Changes to neighbouring lines are auto-merged although the policy treats them as conflicting.
Hunk overlap check: adjacent hunks are treated as independent · case 02
Changes to neighbouring lines are auto-merged although the policy treats them as conflicting.
Hunk overlap check: adjacent hunks are treated as independent · case 03
Changes to neighbouring lines are auto-merged although the policy treats them as conflicting.
Hunk overlap check: adjacent hunks are treated as independent · case 04
Changes to neighbouring lines are auto-merged although the policy treats them as conflicting.
Hunk overlap check: adjacent hunks are treated as independent · case 05
Changes to neighbouring lines are auto-merged although the policy treats them as conflicting.
Hunk overlap check: an insertion is modelled as an empty range at its start line · case 01
Insertions directly before a changed range are not flagged.
Hunk overlap check: an insertion is modelled as an empty range at its start line · case 02
Insertions directly before a changed range are not flagged.
Hunk overlap check: an insertion is modelled as an empty range at its start line · case 03
Insertions directly before a changed range are not flagged.
Hunk overlap check: an insertion is modelled as an empty range at its start line · case 04
Insertions directly before a changed range are not flagged.
Hunk overlap check: an insertion is modelled as an empty range at its start line · case 05
Insertions directly before a changed range are not flagged.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗