Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Rate shopping: service rejected at its weight limit · case 01
A parcel weighing exactly a service maximum is not offered that service.
Rate shopping: service rejected at its weight limit · case 02
A parcel weighing exactly a service maximum is not offered that service.
Rate shopping: service rejected at its weight limit · case 03
A parcel weighing exactly a service maximum is not offered that service.
Rate shopping: service rejected at its weight limit · case 04
A parcel weighing exactly a service maximum is not offered that service.
Rate shopping: service rejected at its weight limit · case 05
A parcel weighing exactly a service maximum is not offered that service.
Rate shopping: exclusions case-sensitive · case 01
A carrier blocked by the shipper is still selected when the block list uses different case.
Rate shopping: exclusions case-sensitive · case 02
A carrier blocked by the shipper is still selected when the block list uses different case.
Rate shopping: exclusions case-sensitive · case 03
A carrier blocked by the shipper is still selected when the block list uses different case.
Rate shopping: exclusions case-sensitive · case 04
A carrier blocked by the shipper is still selected when the block list uses different case.
Rate shopping: exclusions case-sensitive · case 05
A carrier blocked by the shipper is still selected when the block list uses different case.
Rate shopping: slower service wins price tie · case 01
Between equally priced services, the slower one is chosen.
Rate shopping: slower service wins price tie · case 02
Between equally priced services, the slower one is chosen.
Rate shopping: slower service wins price tie · case 03
Between equally priced services, the slower one is chosen.
Rate shopping: slower service wins price tie · case 04
Between equally priced services, the slower one is chosen.
Rate shopping: slower service wins price tie · case 05
Between equally priced services, the slower one is chosen.
Area surcharge: first ZIP of a range missed · case 01
The first ZIP of each surcharge range is not surcharged.
Area surcharge: first ZIP of a range missed · case 02
The first ZIP of each surcharge range is not surcharged.
Area surcharge: first ZIP of a range missed · case 03
The first ZIP of each surcharge range is not surcharged.
Area surcharge: first ZIP of a range missed · case 04
The first ZIP of each surcharge range is not surcharged.
Area surcharge: first ZIP of a range missed · case 05
The first ZIP of each surcharge range is not surcharged.
Area surcharge: last ZIP of a range missed · case 01
The final ZIP in each surcharge range escapes the fee; single-ZIP ranges never match.
Area surcharge: last ZIP of a range missed · case 02
The final ZIP in each surcharge range escapes the fee; single-ZIP ranges never match.
Area surcharge: last ZIP of a range missed · case 03
The final ZIP in each surcharge range escapes the fee; single-ZIP ranges never match.
Area surcharge: last ZIP of a range missed · case 04
The final ZIP in each surcharge range escapes the fee; single-ZIP ranges never match.
Area surcharge: last ZIP of a range missed · case 05
The final ZIP in each surcharge range escapes the fee; single-ZIP ranges never match.
Area surcharge: ZIPs below all ranges hit last range · case 01
ZIPs below the first range are charged the fee of the last range.
Area surcharge: ZIPs below all ranges hit last range · case 02
ZIPs below the first range are charged the fee of the last range.
Area surcharge: ZIPs below all ranges hit last range · case 03
ZIPs below the first range are charged the fee of the last range.
Area surcharge: ZIPs below all ranges hit last range · case 04
ZIPs below the first range are charged the fee of the last range.
Area surcharge: ZIPs below all ranges hit last range · case 05
ZIPs below the first range are charged the fee of the last range.
Area surcharge: binary search over unsorted ranges · case 01
ZIPs in ranges listed out of order are not surcharged.
Area surcharge: binary search over unsorted ranges · case 02
ZIPs in ranges listed out of order are not surcharged.
Area surcharge: binary search over unsorted ranges · case 03
ZIPs in ranges listed out of order are not surcharged.
Area surcharge: binary search over unsorted ranges · case 04
ZIPs in ranges listed out of order are not surcharged.
Area surcharge: binary search over unsorted ranges · case 05
ZIPs in ranges listed out of order are not surcharged.
Area surcharge: ZIP+4 read as nine digits · case 01
ZIP+4 addresses are never surcharged because the nine-digit number exceeds every range.
Area surcharge: ZIP+4 read as nine digits · case 02
ZIP+4 addresses are never surcharged because the nine-digit number exceeds every range.
Area surcharge: ZIP+4 read as nine digits · case 03
ZIP+4 addresses are never surcharged because the nine-digit number exceeds every range.
Area surcharge: ZIP+4 read as nine digits · case 04
ZIP+4 addresses are never surcharged because the nine-digit number exceeds every range.
Area surcharge: ZIP+4 read as nine digits · case 05
ZIP+4 addresses are never surcharged because the nine-digit number exceeds every range.
Region matrix: same ZIP3 not local · case 01
Deliveries within the same ZIP3 are rated as regional zone 2.
Region matrix: same ZIP3 not local · case 02
Deliveries within the same ZIP3 are rated as regional zone 2.
Region matrix: same ZIP3 not local · case 03
Deliveries within the same ZIP3 are rated as regional zone 2.
Region matrix: same ZIP3 not local · case 04
Deliveries within the same ZIP3 are rated as regional zone 2.
Region matrix: same ZIP3 not local · case 05
Deliveries within the same ZIP3 are rated as regional zone 2.
Region matrix: pair only found in listed direction · case 01
Return shipments between two regions find no zone.
Region matrix: pair only found in listed direction · case 02
Return shipments between two regions find no zone.
Region matrix: pair only found in listed direction · case 03
Return shipments between two regions find no zone.
Region matrix: pair only found in listed direction · case 04
Return shipments between two regions find no zone.
Region matrix: pair only found in listed direction · case 05
Return shipments between two regions find no zone.
Region matrix: region end ZIP3 unassigned · case 01
ZIP3s at the upper edge of a region are treated as unknown.
Region matrix: region end ZIP3 unassigned · case 02
ZIP3s at the upper edge of a region are treated as unknown.
Region matrix: region end ZIP3 unassigned · case 03
ZIP3s at the upper edge of a region are treated as unknown.
Region matrix: region end ZIP3 unassigned · case 04
ZIP3s at the upper edge of a region are treated as unknown.
Region matrix: region end ZIP3 unassigned · case 05
ZIP3s at the upper edge of a region are treated as unknown.
Region matrix: intra-region shipments unrated · case 01
Shipments within one region fall through to the matrix and return NO_ZONE.
Region matrix: intra-region shipments unrated · case 02
Shipments within one region fall through to the matrix and return NO_ZONE.
Region matrix: intra-region shipments unrated · case 03
Shipments within one region fall through to the matrix and return NO_ZONE.
Region matrix: intra-region shipments unrated · case 04
Shipments within one region fall through to the matrix and return NO_ZONE.
Region matrix: intra-region shipments unrated · case 05
Shipments within one region fall through to the matrix and return NO_ZONE.
Fuel table: current week price used · case 01
Fuel surcharge changes two weeks early, before the price is published for billing.
Fuel table: current week price used · case 02
Fuel surcharge changes two weeks early, before the price is published for billing.
Fuel table: current week price used · case 03
Fuel surcharge changes two weeks early, before the price is published for billing.
Fuel table: current week price used · case 04
Fuel surcharge changes two weeks early, before the price is published for billing.
Fuel table: current week price used · case 05
Fuel surcharge changes two weeks early, before the price is published for billing.
Fuel table: missing publication week drops surcharge · case 01
Weeks without a published price return no surcharge.
Fuel table: missing publication week drops surcharge · case 02
Weeks without a published price return no surcharge.
Fuel table: missing publication week drops surcharge · case 03
Weeks without a published price return no surcharge.
Fuel table: missing publication week drops surcharge · case 04
Weeks without a published price return no surcharge.
Fuel table: missing publication week drops surcharge · case 05
Weeks without a published price return no surcharge.
Fuel table: price on band edge in lower band · case 01
A diesel price exactly on a band boundary gets the lower band percentage.
Fuel table: price on band edge in lower band · case 02
A diesel price exactly on a band boundary gets the lower band percentage.
Fuel table: price on band edge in lower band · case 03
A diesel price exactly on a band boundary gets the lower band percentage.
Fuel table: price on band edge in lower band · case 04
A diesel price exactly on a band boundary gets the lower band percentage.
Fuel table: price on band edge in lower band · case 05
A diesel price exactly on a band boundary gets the lower band percentage.
Fuel table: first step above table not charged · case 01
Prices at or just above the top band add nothing to the top percentage.
Fuel table: first step above table not charged · case 02
Prices at or just above the top band add nothing to the top percentage.
Fuel table: first step above table not charged · case 03
Prices at or just above the top band add nothing to the top percentage.
Fuel table: first step above table not charged · case 04
Prices at or just above the top band add nothing to the top percentage.
Fuel table: first step above table not charged · case 05
Prices at or just above the top band add nothing to the top percentage.
Fuel table: lowest band charged below table · case 01
Diesel below the table still bills the lowest band surcharge.
Fuel table: lowest band charged below table · case 02
Diesel below the table still bills the lowest band surcharge.
Fuel table: lowest band charged below table · case 03
Diesel below the table still bills the lowest band surcharge.
Fuel table: lowest band charged below table · case 04
Diesel below the table still bills the lowest band surcharge.
Fuel table: lowest band charged below table · case 05
Diesel below the table still bills the lowest band surcharge.
Ounce tiers: 16 oz priced as ounces · case 01
A parcel of exactly one pound is sold the lightweight ounce price.
Ounce tiers: 16 oz priced as ounces · case 02
A parcel of exactly one pound is sold the lightweight ounce price.
Ounce tiers: 16 oz priced as ounces · case 03
A parcel of exactly one pound is sold the lightweight ounce price.
Ounce tiers: 16 oz priced as ounces · case 04
A parcel of exactly one pound is sold the lightweight ounce price.
Ounce tiers: 16 oz priced as ounces · case 05
A parcel of exactly one pound is sold the lightweight ounce price.
Ounce tiers: fractional ounces rounded to nearest · case 01
A 4.2 oz parcel is priced in the first tier.
Ounce tiers: fractional ounces rounded to nearest · case 02
A 4.2 oz parcel is priced in the first tier.
Ounce tiers: fractional ounces rounded to nearest · case 03
A 4.2 oz parcel is priced in the first tier.
Ounce tiers: fractional ounces rounded to nearest · case 04
A 4.2 oz parcel is priced in the first tier.
Ounce tiers: fractional ounces rounded to nearest · case 05
A 4.2 oz parcel is priced in the first tier.
Ounce tiers: exact multiples pushed up a tier · case 01
Parcels of exactly 4, 8 or 12 oz are priced one tier high.
Ounce tiers: exact multiples pushed up a tier · case 02
Parcels of exactly 4, 8 or 12 oz are priced one tier high.
Ounce tiers: exact multiples pushed up a tier · case 03
Parcels of exactly 4, 8 or 12 oz are priced one tier high.
Ounce tiers: exact multiples pushed up a tier · case 04
Parcels of exactly 4, 8 or 12 oz are priced one tier high.
Ounce tiers: exact multiples pushed up a tier · case 05
Parcels of exactly 4, 8 or 12 oz are priced one tier high.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗