Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Group balanced squared loss: Within-group loss totals are not normalized by group sizes. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Within-group loss totals are not normalized by group sizes. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Within-group loss totals are not normalized by group sizes. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Within-group loss totals are not normalized by group sizes. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Within-group loss totals are not normalized by group sizes. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Missing residuals are imputed as zero and counted. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Missing residuals are imputed as zero and counted. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Missing residuals are imputed as zero and counted. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Missing residuals are imputed as zero and counted. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Missing residuals are imputed as zero and counted. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Zero residuals are dropped as if missing. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Zero residuals are dropped as if missing. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Zero residuals are dropped as if missing. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Zero residuals are dropped as if missing. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Zero residuals are dropped as if missing. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group retains only its latest residual. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group retains only its latest residual. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group retains only its latest residual. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group retains only its latest residual. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group retains only its latest residual. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Only the largest group is evaluated. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Only the largest group is evaluated. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Only the largest group is evaluated. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Only the largest group is evaluated. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Only the largest group is evaluated. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Worst-group loss replaces the group average. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Worst-group loss replaces the group average. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Worst-group loss replaces the group average. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Worst-group loss replaces the group average. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Worst-group loss replaces the group average. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Distinct case-sensitive group labels are merged. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Distinct case-sensitive group labels are merged. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Distinct case-sensitive group labels are merged. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Distinct case-sensitive group labels are merged. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Distinct case-sensitive group labels are merged. · case 05
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group mean is integer-truncated before balancing. · case 01
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group mean is integer-truncated before balancing. · case 02
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group mean is integer-truncated before balancing. · case 03
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group mean is integer-truncated before balancing. · case 04
The reduction disagrees with its explicit aggregation oracle.
Group balanced squared loss: Each group mean is integer-truncated before balancing. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: All rows share the same mutable storage. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: All rows share the same mutable storage. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: All rows share the same mutable storage. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: All rows share the same mutable storage. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: All rows share the same mutable storage. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: A weighted update overwrites its bucket instead of adding. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: A weighted update overwrites its bucket instead of adding. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: A weighted update overwrites its bucket instead of adding. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: A weighted update overwrites its bucket instead of adding. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: A weighted update overwrites its bucket instead of adding. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Every weighted contribution increments by one. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Every weighted contribution increments by one. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Every weighted contribution increments by one. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Every weighted contribution increments by one. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Every weighted contribution increments by one. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Each block clears previously accumulated rows. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Each block clears previously accumulated rows. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Each block clears previously accumulated rows. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Each block clears previously accumulated rows. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Each block clears previously accumulated rows. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation sums the three collision-contaminated cells. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation sums the three collision-contaminated cells. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation sums the three collision-contaminated cells. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation sums the three collision-contaminated cells. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation sums the three collision-contaminated cells. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation chooses the most contaminated row. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation chooses the most contaminated row. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation chooses the most contaminated row. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation chooses the most contaminated row. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query estimation chooses the most contaminated row. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Updates omit the third independent hash row. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Updates omit the third independent hash row. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Updates omit the third independent hash row. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Updates omit the third independent hash row. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Updates omit the third independent hash row. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query buckets are read from the next hash row. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query buckets are read from the next hash row. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query buckets are read from the next hash row. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query buckets are read from the next hash row. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Query buckets are read from the next hash row. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Identical weighted events are deduplicated within a block. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Identical weighted events are deduplicated within a block. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Identical weighted events are deduplicated within a block. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Identical weighted events are deduplicated within a block. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Identical weighted events are deduplicated within a block. · case 05
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Queries use each row minimum instead of the key-specific bucket. · case 01
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Queries use each row minimum instead of the key-specific bucket. · case 02
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Queries use each row minimum instead of the key-specific bucket. · case 03
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Queries use each row minimum instead of the key-specific bucket. · case 04
The reduction disagrees with its explicit aggregation oracle.
Three row count sketch: Queries use each row minimum instead of the key-specific bucket. · case 05
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: The denominator squares total weight instead of summing squares. · case 01
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: The denominator squares total weight instead of summing squares. · case 02
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: The denominator squares total weight instead of summing squares. · case 03
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: The denominator squares total weight instead of summing squares. · case 04
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: The denominator squares total weight instead of summing squares. · case 05
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: Total weight is not squared in the numerator. · case 01
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: Total weight is not squared in the numerator. · case 02
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: Total weight is not squared in the numerator. · case 03
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: Total weight is not squared in the numerator. · case 04
The reduction disagrees with its explicit aggregation oracle.
Capped weight effective size: Total weight is not squared in the numerator. · case 05
The reduction disagrees with its explicit aggregation oracle.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗