Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Intersect set: Drops common NULL set values · case 01
The query violates its result contract: drops common null set values.
Intersect set: Drops common NULL set values · case 02
The query violates its result contract: drops common null set values.
Intersect set: Drops common NULL set values · case 03
The query violates its result contract: drops common null set values.
Intersect set: Drops common NULL set values · case 04
The query violates its result contract: drops common null set values.
Intersect set: Drops common NULL set values · case 05
The query violates its result contract: drops common null set values.
Intersect set: Returns keys in either relation · case 01
The query violates its result contract: returns keys in either relation.
Intersect set: Returns keys in either relation · case 02
The query violates its result contract: returns keys in either relation.
Intersect set: Returns keys in either relation · case 03
The query violates its result contract: returns keys in either relation.
Intersect set: Returns keys in either relation · case 04
The query violates its result contract: returns keys in either relation.
Intersect set: Returns keys in either relation · case 05
The query violates its result contract: returns keys in either relation.
Intersect set: Returns left-only keys instead of common keys · case 01
The query violates its result contract: returns left-only keys instead of common keys.
Intersect set: Returns left-only keys instead of common keys · case 02
The query violates its result contract: returns left-only keys instead of common keys.
Intersect set: Returns left-only keys instead of common keys · case 03
The query violates its result contract: returns left-only keys instead of common keys.
Intersect set: Returns left-only keys instead of common keys · case 04
The query violates its result contract: returns left-only keys instead of common keys.
Intersect set: Returns left-only keys instead of common keys · case 05
The query violates its result contract: returns left-only keys instead of common keys.
Intersect set: Preserves duplicate left occurrences in a set intersection · case 01
The query violates its result contract: preserves duplicate left occurrences in a set intersection.
Intersect set: Preserves duplicate left occurrences in a set intersection · case 02
The query violates its result contract: preserves duplicate left occurrences in a set intersection.
Intersect set: Preserves duplicate left occurrences in a set intersection · case 03
The query violates its result contract: preserves duplicate left occurrences in a set intersection.
Intersect set: Preserves duplicate left occurrences in a set intersection · case 04
The query violates its result contract: preserves duplicate left occurrences in a set intersection.
Intersect set: Preserves duplicate left occurrences in a set intersection · case 05
The query violates its result contract: preserves duplicate left occurrences in a set intersection.
Except set: Retains NULL despite a right NULL set member · case 01
The query violates its result contract: retains null despite a right null set member.
Except set: Retains NULL despite a right NULL set member · case 02
The query violates its result contract: retains null despite a right null set member.
Except set: Retains NULL despite a right NULL set member · case 03
The query violates its result contract: retains null despite a right null set member.
Except set: Retains NULL despite a right NULL set member · case 04
The query violates its result contract: retains null despite a right null set member.
Except set: Retains NULL despite a right NULL set member · case 05
The query violates its result contract: retains null despite a right null set member.
Except set: Preserves left multiplicity for a set difference · case 01
The query violates its result contract: preserves left multiplicity for a set difference.
Except set: Preserves left multiplicity for a set difference · case 02
The query violates its result contract: preserves left multiplicity for a set difference.
Except set: Preserves left multiplicity for a set difference · case 03
The query violates its result contract: preserves left multiplicity for a set difference.
Except set: Preserves left multiplicity for a set difference · case 04
The query violates its result contract: preserves left multiplicity for a set difference.
Except set: Preserves left multiplicity for a set difference · case 05
The query violates its result contract: preserves left multiplicity for a set difference.
Except set: Computes b minus a instead of a minus b · case 01
The query violates its result contract: computes b minus a instead of a minus b.
Except set: Computes b minus a instead of a minus b · case 02
The query violates its result contract: computes b minus a instead of a minus b.
Except set: Computes b minus a instead of a minus b · case 03
The query violates its result contract: computes b minus a instead of a minus b.
Except set: Computes b minus a instead of a minus b · case 04
The query violates its result contract: computes b minus a instead of a minus b.
Except set: Computes b minus a instead of a minus b · case 05
The query violates its result contract: computes b minus a instead of a minus b.
Except set: Allows unknown right values to poison NOT IN · case 01
The query violates its result contract: allows unknown right values to poison not in.
Except set: Allows unknown right values to poison NOT IN · case 02
The query violates its result contract: allows unknown right values to poison not in.
Except set: Allows unknown right values to poison NOT IN · case 03
The query violates its result contract: allows unknown right values to poison not in.
Except set: Allows unknown right values to poison NOT IN · case 04
The query violates its result contract: allows unknown right values to poison not in.
Except set: Allows unknown right values to poison NOT IN · case 05
The query violates its result contract: allows unknown right values to poison not in.
Except set: Returns common values instead of absent values · case 01
The query violates its result contract: returns common values instead of absent values.
Except set: Returns common values instead of absent values · case 02
The query violates its result contract: returns common values instead of absent values.
Except set: Returns common values instead of absent values · case 03
The query violates its result contract: returns common values instead of absent values.
Except set: Returns common values instead of absent values · case 04
The query violates its result contract: returns common values instead of absent values.
Except set: Returns common values instead of absent values · case 05
The query violates its result contract: returns common values instead of absent values.
Projection set: Preserves duplicate projected pairs · case 01
The query violates its result contract: preserves duplicate projected pairs.
Projection set: Preserves duplicate projected pairs · case 02
The query violates its result contract: preserves duplicate projected pairs.
Projection set: Preserves duplicate projected pairs · case 03
The query violates its result contract: preserves duplicate projected pairs.
Projection set: Preserves duplicate projected pairs · case 04
The query violates its result contract: preserves duplicate projected pairs.
Projection set: Preserves duplicate projected pairs · case 05
The query violates its result contract: preserves duplicate projected pairs.
Projection set: Groups only by k and picks an arbitrary value · case 01
The query violates its result contract: groups only by k and picks an arbitrary value.
Projection set: Groups only by k and picks an arbitrary value · case 02
The query violates its result contract: groups only by k and picks an arbitrary value.
Projection set: Groups only by k and picks an arbitrary value · case 03
The query violates its result contract: groups only by k and picks an arbitrary value.
Projection set: Groups only by k and picks an arbitrary value · case 04
The query violates its result contract: groups only by k and picks an arbitrary value.
Projection set: Groups only by k and picks an arbitrary value · case 05
The query violates its result contract: groups only by k and picks an arbitrary value.
Projection set: Groups only by v and picks a key · case 01
The query violates its result contract: groups only by v and picks a key.
Projection set: Groups only by v and picks a key · case 02
The query violates its result contract: groups only by v and picks a key.
Projection set: Groups only by v and picks a key · case 03
The query violates its result contract: groups only by v and picks a key.
Projection set: Groups only by v and picks a key · case 04
The query violates its result contract: groups only by v and picks a key.
Projection set: Groups only by v and picks a key · case 05
The query violates its result contract: groups only by v and picks a key.
Projection set: Drops pairs containing unknown values · case 01
The query violates its result contract: drops pairs containing unknown values.
Projection set: Drops pairs containing unknown values · case 02
The query violates its result contract: drops pairs containing unknown values.
Projection set: Drops pairs containing unknown values · case 03
The query violates its result contract: drops pairs containing unknown values.
Projection set: Drops pairs containing unknown values · case 04
The query violates its result contract: drops pairs containing unknown values.
Projection set: Drops pairs containing unknown values · case 05
The query violates its result contract: drops pairs containing unknown values.
Projection set: Coalesces unknown values into zero before DISTINCT · case 01
The query violates its result contract: coalesces unknown values into zero before distinct.
Projection set: Coalesces unknown values into zero before DISTINCT · case 02
The query violates its result contract: coalesces unknown values into zero before distinct.
Projection set: Coalesces unknown values into zero before DISTINCT · case 03
The query violates its result contract: coalesces unknown values into zero before distinct.
Projection set: Coalesces unknown values into zero before DISTINCT · case 04
The query violates its result contract: coalesces unknown values into zero before distinct.
Projection set: Coalesces unknown values into zero before DISTINCT · case 05
The query violates its result contract: coalesces unknown values into zero before distinct.
Group row count: Counts known measurements instead of rows · case 01
The query violates its result contract: counts known measurements instead of rows.
Group row count: Counts known measurements instead of rows · case 02
The query violates its result contract: counts known measurements instead of rows.
Group row count: Counts known measurements instead of rows · case 03
The query violates its result contract: counts known measurements instead of rows.
Group row count: Counts known measurements instead of rows · case 04
The query violates its result contract: counts known measurements instead of rows.
Group row count: Counts known measurements instead of rows · case 05
The query violates its result contract: counts known measurements instead of rows.
Group row count: Counts distinct measurements instead of facts · case 01
The query violates its result contract: counts distinct measurements instead of facts.
Group row count: Counts distinct measurements instead of facts · case 02
The query violates its result contract: counts distinct measurements instead of facts.
Group row count: Counts distinct measurements instead of facts · case 03
The query violates its result contract: counts distinct measurements instead of facts.
Group row count: Counts distinct measurements instead of facts · case 04
The query violates its result contract: counts distinct measurements instead of facts.
Group row count: Counts distinct measurements instead of facts · case 05
The query violates its result contract: counts distinct measurements instead of facts.
Group row count: Excludes the unknown-key group · case 01
The query violates its result contract: excludes the unknown-key group.
Group row count: Excludes the unknown-key group · case 02
The query violates its result contract: excludes the unknown-key group.
Group row count: Excludes the unknown-key group · case 03
The query violates its result contract: excludes the unknown-key group.
Group row count: Excludes the unknown-key group · case 04
The query violates its result contract: excludes the unknown-key group.
Group row count: Excludes the unknown-key group · case 05
The query violates its result contract: excludes the unknown-key group.
Group row count: Splits key groups by their flag · case 01
The query violates its result contract: splits key groups by their flag.
Group row count: Splits key groups by their flag · case 02
The query violates its result contract: splits key groups by their flag.
Group row count: Splits key groups by their flag · case 03
The query violates its result contract: splits key groups by their flag.
Group row count: Splits key groups by their flag · case 04
The query violates its result contract: splits key groups by their flag.
Group row count: Splits key groups by their flag · case 05
The query violates its result contract: splits key groups by their flag.
Group row count: Uses HAVING to drop singleton groups · case 01
The query violates its result contract: uses having to drop singleton groups.
Group row count: Uses HAVING to drop singleton groups · case 02
The query violates its result contract: uses having to drop singleton groups.
Group row count: Uses HAVING to drop singleton groups · case 03
The query violates its result contract: uses having to drop singleton groups.
Group row count: Uses HAVING to drop singleton groups · case 04
The query violates its result contract: uses having to drop singleton groups.
Group row count: Uses HAVING to drop singleton groups · case 05
The query violates its result contract: uses having to drop singleton groups.
Group known count: Counts unknown measurements as observations · case 01
The query violates its result contract: counts unknown measurements as observations.
Group known count: Counts unknown measurements as observations · case 02
The query violates its result contract: counts unknown measurements as observations.
Group known count: Counts unknown measurements as observations · case 03
The query violates its result contract: counts unknown measurements as observations.
Group known count: Counts unknown measurements as observations · case 04
The query violates its result contract: counts unknown measurements as observations.
Group known count: Counts unknown measurements as observations · case 05
The query violates its result contract: counts unknown measurements as observations.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗