Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Batch normalization scales by the entire batch norm · case 01
Batch normalization scales by the entire batch norm.
Batch normalization scales by the entire batch norm · case 02
Batch normalization scales by the entire batch norm.
Batch normalization scales by the entire batch norm · case 03
Batch normalization scales by the entire batch norm.
Batch normalization scales by the entire batch norm · case 04
Batch normalization scales by the entire batch norm.
Batch normalization scales by the entire batch norm · case 05
Batch normalization scales by the entire batch norm.
Batch centering subtracts a row average · case 01
Batch centering subtracts a row average.
Batch centering subtracts a row average · case 02
Batch centering subtracts a row average.
Batch centering subtracts a row average · case 03
Batch centering subtracts a row average.
Batch centering subtracts a row average · case 04
Batch centering subtracts a row average.
Batch centering subtracts a row average · case 05
Batch centering subtracts a row average.
Feature standardization divides by variance · case 01
Feature standardization divides by variance.
Feature standardization divides by variance · case 02
Feature standardization divides by variance.
Feature standardization divides by variance · case 03
Feature standardization divides by variance.
Feature standardization divides by variance · case 04
Feature standardization divides by variance.
Feature standardization divides by variance · case 05
Feature standardization divides by variance.
Weighted centering subtracts an unweighted mean · case 01
Weighted centering subtracts an unweighted mean.
Weighted centering subtracts an unweighted mean · case 02
Weighted centering subtracts an unweighted mean.
Weighted centering subtracts an unweighted mean · case 03
Weighted centering subtracts an unweighted mean.
Weighted centering subtracts an unweighted mean · case 04
Weighted centering subtracts an unweighted mean.
Weighted centering subtracts an unweighted mean · case 05
Weighted centering subtracts an unweighted mean.
Coordinate power pooling averages before applying sign · case 01
Coordinate power pooling averages before applying sign.
Coordinate power pooling averages before applying sign · case 02
Coordinate power pooling averages before applying sign.
Coordinate power pooling averages before applying sign · case 03
Coordinate power pooling averages before applying sign.
Coordinate power pooling averages before applying sign · case 04
Coordinate power pooling averages before applying sign.
Coordinate power pooling averages before applying sign · case 05
Coordinate power pooling averages before applying sign.
Max pooling selects coordinate minima · case 01
Max pooling selects coordinate minima.
Max pooling selects coordinate minima · case 02
Max pooling selects coordinate minima.
Max pooling selects coordinate minima · case 03
Max pooling selects coordinate minima.
Max pooling selects coordinate minima · case 04
Max pooling selects coordinate minima.
Max pooling selects coordinate minima · case 05
Max pooling selects coordinate minima.
Min pooling selects coordinate maxima · case 01
Min pooling selects coordinate maxima.
Min pooling selects coordinate maxima · case 02
Min pooling selects coordinate maxima.
Min pooling selects coordinate maxima · case 03
Min pooling selects coordinate maxima.
Min pooling selects coordinate maxima · case 04
Min pooling selects coordinate maxima.
Min pooling selects coordinate maxima · case 05
Min pooling selects coordinate maxima.
Projection removal subtracts the reference direction unchanged · case 01
Projection removal subtracts the reference direction unchanged.
Projection removal subtracts the reference direction unchanged · case 02
Projection removal subtracts the reference direction unchanged.
Projection removal subtracts the reference direction unchanged · case 03
Projection removal subtracts the reference direction unchanged.
Projection removal subtracts the reference direction unchanged · case 04
Projection removal subtracts the reference direction unchanged.
Projection removal subtracts the reference direction unchanged · case 05
Projection removal subtracts the reference direction unchanged.
Vector retrieval: Nearest L2 candidate is ranked last · case 01
Nearest L2 candidate is ranked last.
Vector retrieval: Nearest L2 candidate is ranked last · case 02
Nearest L2 candidate is ranked last.
Vector retrieval: Nearest L2 candidate is ranked last · case 03
Nearest L2 candidate is ranked last.
Vector retrieval: Nearest L2 candidate is ranked last · case 04
Nearest L2 candidate is ranked last.
Vector retrieval: Nearest L2 candidate is ranked last · case 05
Nearest L2 candidate is ranked last.
Vector retrieval: Cosine similarity is treated as a distance · case 01
Cosine similarity is treated as a distance.
Vector retrieval: Cosine similarity is treated as a distance · case 02
Cosine similarity is treated as a distance.
Vector retrieval: Cosine similarity is treated as a distance · case 03
Cosine similarity is treated as a distance.
Vector retrieval: Cosine similarity is treated as a distance · case 04
Cosine similarity is treated as a distance.
Vector retrieval: Cosine similarity is treated as a distance · case 05
Cosine similarity is treated as a distance.
Vector retrieval: Retrieval channel weights disappear during fusion · case 01
Retrieval channel weights disappear during fusion.
Vector retrieval: Retrieval channel weights disappear during fusion · case 02
Retrieval channel weights disappear during fusion.
Vector retrieval: Retrieval channel weights disappear during fusion · case 03
Retrieval channel weights disappear during fusion.
Vector retrieval: Retrieval channel weights disappear during fusion · case 04
Retrieval channel weights disappear during fusion.
Vector retrieval: Retrieval channel weights disappear during fusion · case 05
Retrieval channel weights disappear during fusion.
Vector retrieval: Equal scores inherit input order instead of ID order · case 01
Equal scores inherit input order instead of ID order.
Vector retrieval: Equal scores inherit input order instead of ID order · case 02
Equal scores inherit input order instead of ID order.
Vector retrieval: Equal scores inherit input order instead of ID order · case 03
Equal scores inherit input order instead of ID order.
Vector retrieval: Equal scores inherit input order instead of ID order · case 04
Equal scores inherit input order instead of ID order.
Vector retrieval: Equal scores inherit input order instead of ID order · case 05
Equal scores inherit input order instead of ID order.
Vector retrieval: Candidates exactly on the score floor disappear · case 01
Candidates exactly on the score floor disappear.
Vector retrieval: Candidates exactly on the score floor disappear · case 02
Candidates exactly on the score floor disappear.
Vector retrieval: Candidates exactly on the score floor disappear · case 03
Candidates exactly on the score floor disappear.
Vector retrieval: Candidates exactly on the score floor disappear · case 04
Candidates exactly on the score floor disappear.
Vector retrieval: Candidates exactly on the score floor disappear · case 05
Candidates exactly on the score floor disappear.
Vector retrieval: A later retrieval channel overwrites earlier candidates · case 01
A later retrieval channel overwrites earlier candidates.
Vector retrieval: A later retrieval channel overwrites earlier candidates · case 02
A later retrieval channel overwrites earlier candidates.
Vector retrieval: A later retrieval channel overwrites earlier candidates · case 03
A later retrieval channel overwrites earlier candidates.
Vector retrieval: A later retrieval channel overwrites earlier candidates · case 04
A later retrieval channel overwrites earlier candidates.
Vector retrieval: A later retrieval channel overwrites earlier candidates · case 05
A later retrieval channel overwrites earlier candidates.
Vector retrieval: Multiple query vectors duplicate the same document · case 01
Multiple query vectors duplicate the same document.
Vector retrieval: Multiple query vectors duplicate the same document · case 02
Multiple query vectors duplicate the same document.
Vector retrieval: Multiple query vectors duplicate the same document · case 03
Multiple query vectors duplicate the same document.
Vector retrieval: Multiple query vectors duplicate the same document · case 04
Multiple query vectors duplicate the same document.
Vector retrieval: Multiple query vectors duplicate the same document · case 05
Multiple query vectors duplicate the same document.
Vector retrieval: Multi-vector aggregation uses the maximum chunk distance · case 01
Multi-vector aggregation uses the maximum chunk distance.
Vector retrieval: Multi-vector aggregation uses the maximum chunk distance · case 02
Multi-vector aggregation uses the maximum chunk distance.
Vector retrieval: Multi-vector aggregation uses the maximum chunk distance · case 03
Multi-vector aggregation uses the maximum chunk distance.
Vector retrieval: Multi-vector aggregation uses the maximum chunk distance · case 04
Multi-vector aggregation uses the maximum chunk distance.
Vector retrieval: Multi-vector aggregation uses the maximum chunk distance · case 05
Multi-vector aggregation uses the maximum chunk distance.
Vector retrieval: Vector coordinates are compared lexicographically · case 01
Vector coordinates are compared lexicographically.
Vector retrieval: Vector coordinates are compared lexicographically · case 02
Vector coordinates are compared lexicographically.
Vector retrieval: Vector coordinates are compared lexicographically · case 03
Vector coordinates are compared lexicographically.
Vector retrieval: Vector coordinates are compared lexicographically · case 04
Vector coordinates are compared lexicographically.
Vector retrieval: Vector coordinates are compared lexicographically · case 05
Vector coordinates are compared lexicographically.
Vector retrieval: Weighted distance ignores per-coordinate importance · case 01
Weighted distance ignores per-coordinate importance.
Vector retrieval: Weighted distance ignores per-coordinate importance · case 02
Weighted distance ignores per-coordinate importance.
Vector retrieval: Weighted distance ignores per-coordinate importance · case 03
Weighted distance ignores per-coordinate importance.
Vector retrieval: Weighted distance ignores per-coordinate importance · case 04
Weighted distance ignores per-coordinate importance.
Vector retrieval: Weighted distance ignores per-coordinate importance · case 05
Weighted distance ignores per-coordinate importance.
Vector retrieval: Unequal vector norms distort cosine ranking · case 01
Unequal vector norms distort cosine ranking.
Vector retrieval: Unequal vector norms distort cosine ranking · case 02
Unequal vector norms distort cosine ranking.
Vector retrieval: Unequal vector norms distort cosine ranking · case 03
Unequal vector norms distort cosine ranking.
Vector retrieval: Unequal vector norms distort cosine ranking · case 04
Unequal vector norms distort cosine ranking.
Vector retrieval: Unequal vector norms distort cosine ranking · case 05
Unequal vector norms distort cosine ranking.
Vector retrieval: The item at rank k is dropped · case 01
The item at rank k is dropped.
Vector retrieval: The item at rank k is dropped · case 02
The item at rank k is dropped.
Vector retrieval: The item at rank k is dropped · case 03
The item at rank k is dropped.
Vector retrieval: The item at rank k is dropped · case 04
The item at rank k is dropped.
Vector retrieval: The item at rank k is dropped · case 05
The item at rank k is dropped.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗