Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Rate table: weight on bracket edge bumped up · case 01
A 2 lb parcel is charged the 5 lb bracket rate.
Rate table: weight on bracket edge bumped up · case 02
A 2 lb parcel is charged the 5 lb bracket rate.
Rate table: weight on bracket edge bumped up · case 03
A 2 lb parcel is charged the 5 lb bracket rate.
Rate table: weight on bracket edge bumped up · case 04
A 2 lb parcel is charged the 5 lb bracket rate.
Rate table: weight on bracket edge bumped up · case 05
A 2 lb parcel is charged the 5 lb bracket rate.
Rate table: zone 1 indexes the last column · case 01
Local zone 1 shipments are charged the zone 8 rate.
Rate table: zone 1 indexes the last column · case 02
Local zone 1 shipments are charged the zone 8 rate.
Rate table: zone 1 indexes the last column · case 03
Local zone 1 shipments are charged the zone 8 rate.
Rate table: zone 1 indexes the last column · case 04
Local zone 1 shipments are charged the zone 8 rate.
Rate table: zone 1 indexes the last column · case 05
Local zone 1 shipments are charged the zone 8 rate.
Rate table: per-pound excess counted from zero · case 01
Heavy parcels are charged the per-pound increment on their whole weight on top of the top bracket.
Rate table: per-pound excess counted from zero · case 02
Heavy parcels are charged the per-pound increment on their whole weight on top of the top bracket.
Rate table: per-pound excess counted from zero · case 03
Heavy parcels are charged the per-pound increment on their whole weight on top of the top bracket.
Rate table: per-pound excess counted from zero · case 04
Heavy parcels are charged the per-pound increment on their whole weight on top of the top bracket.
Rate table: per-pound excess counted from zero · case 05
Heavy parcels are charged the per-pound increment on their whole weight on top of the top bracket.
Rate table: fractional pounds not rounded up · case 01
Weights just above a bracket edge are rated in the lower bracket.
Rate table: fractional pounds not rounded up · case 02
Weights just above a bracket edge are rated in the lower bracket.
Rate table: fractional pounds not rounded up · case 03
Weights just above a bracket edge are rated in the lower bracket.
Rate table: fractional pounds not rounded up · case 04
Weights just above a bracket edge are rated in the lower bracket.
Rate table: fractional pounds not rounded up · case 05
Weights just above a bracket edge are rated in the lower bracket.
Rate table: unavailable cell returns null rate · case 01
A zone and weight that the service does not offer returns a null price instead of NO_RATE.
Rate table: unavailable cell returns null rate · case 02
A zone and weight that the service does not offer returns a null price instead of NO_RATE.
Rate table: unavailable cell returns null rate · case 03
A zone and weight that the service does not offer returns a null price instead of NO_RATE.
Rate table: unavailable cell returns null rate · case 04
A zone and weight that the service does not offer returns a null price instead of NO_RATE.
Rate table: unavailable cell returns null rate · case 05
A zone and weight that the service does not offer returns a null price instead of NO_RATE.
Surcharges: fuel on base rate only · case 01
Fuel surcharge is under-collected on residential and remote deliveries.
Surcharges: fuel on base rate only · case 02
Fuel surcharge is under-collected on residential and remote deliveries.
Surcharges: fuel on base rate only · case 03
Fuel surcharge is under-collected on residential and remote deliveries.
Surcharges: fuel on base rate only · case 04
Fuel surcharge is under-collected on residential and remote deliveries.
Surcharges: fuel on base rate only · case 05
Fuel surcharge is under-collected on residential and remote deliveries.
Surcharges: fuel cents truncated · case 01
Fuel surcharges are a cent low on roughly half of shipments.
Surcharges: fuel cents truncated · case 02
Fuel surcharges are a cent low on roughly half of shipments.
Surcharges: fuel cents truncated · case 03
Fuel surcharges are a cent low on roughly half of shipments.
Surcharges: fuel cents truncated · case 04
Fuel surcharges are a cent low on roughly half of shipments.
Surcharges: fuel cents truncated · case 05
Fuel surcharges are a cent low on roughly half of shipments.
Surcharges: residential area uplift missing · case 01
Residential deliveries in surcharge areas are billed the commercial area fee.
Surcharges: residential area uplift missing · case 02
Residential deliveries in surcharge areas are billed the commercial area fee.
Surcharges: residential area uplift missing · case 03
Residential deliveries in surcharge areas are billed the commercial area fee.
Surcharges: residential area uplift missing · case 04
Residential deliveries in surcharge areas are billed the commercial area fee.
Surcharges: residential area uplift missing · case 05
Residential deliveries in surcharge areas are billed the commercial area fee.
Surcharges: free declared value not deducted · case 01
Every declared dollar is charged, including the free first $100.
Surcharges: free declared value not deducted · case 02
Every declared dollar is charged, including the free first $100.
Surcharges: free declared value not deducted · case 03
Every declared dollar is charged, including the free first $100.
Surcharges: free declared value not deducted · case 04
Every declared dollar is charged, including the free first $100.
Surcharges: free declared value not deducted · case 05
Every declared dollar is charged, including the free first $100.
Surcharges: partial hundred not charged · case 01
Declared value of $101 to $199 above the allowance is charged for fewer hundreds than required.
Surcharges: partial hundred not charged · case 02
Declared value of $101 to $199 above the allowance is charged for fewer hundreds than required.
Surcharges: partial hundred not charged · case 03
Declared value of $101 to $199 above the allowance is charged for fewer hundreds than required.
Surcharges: partial hundred not charged · case 04
Declared value of $101 to $199 above the allowance is charged for fewer hundreds than required.
Surcharges: partial hundred not charged · case 05
Declared value of $101 to $199 above the allowance is charged for fewer hundreds than required.
Surcharges: declared value missing from total · case 01
The shipment total omits the declared value fee shown on the same quote.
Surcharges: declared value missing from total · case 02
The shipment total omits the declared value fee shown on the same quote.
Surcharges: declared value missing from total · case 03
The shipment total omits the declared value fee shown on the same quote.
Surcharges: declared value missing from total · case 04
The shipment total omits the declared value fee shown on the same quote.
Surcharges: declared value missing from total · case 05
The shipment total omits the declared value fee shown on the same quote.
Size classes: first listed side taken as length · case 01
Girth is computed around the longest side when dims are not entered longest-first.
Size classes: first listed side taken as length · case 02
Girth is computed around the longest side when dims are not entered longest-first.
Size classes: first listed side taken as length · case 03
Girth is computed around the longest side when dims are not entered longest-first.
Size classes: first listed side taken as length · case 04
Girth is computed around the longest side when dims are not entered longest-first.
Size classes: first listed side taken as length · case 05
Girth is computed around the longest side when dims are not entered longest-first.
Size classes: girth not doubled · case 01
Parcels far above the oversize threshold are treated as standard.
Size classes: girth not doubled · case 02
Parcels far above the oversize threshold are treated as standard.
Size classes: girth not doubled · case 03
Parcels far above the oversize threshold are treated as standard.
Size classes: girth not doubled · case 04
Parcels far above the oversize threshold are treated as standard.
Size classes: girth not doubled · case 05
Parcels far above the oversize threshold are treated as standard.
Size classes: oversize also billed dimension handling · case 01
Oversize parcels are double charged with an additional handling dimension fee.
Size classes: oversize also billed dimension handling · case 02
Oversize parcels are double charged with an additional handling dimension fee.
Size classes: oversize also billed dimension handling · case 03
Oversize parcels are double charged with an additional handling dimension fee.
Size classes: oversize also billed dimension handling · case 04
Oversize parcels are double charged with an additional handling dimension fee.
Size classes: oversize also billed dimension handling · case 05
Oversize parcels are double charged with an additional handling dimension fee.
Size classes: shortest side used for width rule · case 01
Wide flat boxes escape the additional handling fee.
Size classes: shortest side used for width rule · case 02
Wide flat boxes escape the additional handling fee.
Size classes: shortest side used for width rule · case 03
Wide flat boxes escape the additional handling fee.
Size classes: shortest side used for width rule · case 04
Wide flat boxes escape the additional handling fee.
Size classes: shortest side used for width rule · case 05
Wide flat boxes escape the additional handling fee.
Size classes: 50 lb exactly treated as heavy · case 01
Parcels of exactly 50 lb are charged the weight handling fee.
Size classes: 50 lb exactly treated as heavy · case 02
Parcels of exactly 50 lb are charged the weight handling fee.
Size classes: 50 lb exactly treated as heavy · case 03
Parcels of exactly 50 lb are charged the weight handling fee.
Size classes: 50 lb exactly treated as heavy · case 04
Parcels of exactly 50 lb are charged the weight handling fee.
Size classes: 50 lb exactly treated as heavy · case 05
Parcels of exactly 50 lb are charged the weight handling fee.
Size classes: overweight parcel accepted · case 01
A 151 lb parcel is quoted as a heavy parcel instead of refused.
Size classes: overweight parcel accepted · case 02
A 151 lb parcel is quoted as a heavy parcel instead of refused.
Size classes: overweight parcel accepted · case 03
A 151 lb parcel is quoted as a heavy parcel instead of refused.
Size classes: overweight parcel accepted · case 04
A 151 lb parcel is quoted as a heavy parcel instead of refused.
Size classes: overweight parcel accepted · case 05
A 151 lb parcel is quoted as a heavy parcel instead of refused.
Rate card: dates compared as text · case 01
October cards are treated as older than February cards when months are not zero padded.
Rate card: dates compared as text · case 02
October cards are treated as older than February cards when months are not zero padded.
Rate card: dates compared as text · case 03
October cards are treated as older than February cards when months are not zero padded.
Rate card: dates compared as text · case 04
October cards are treated as older than February cards when months are not zero padded.
Rate card: dates compared as text · case 05
October cards are treated as older than February cards when months are not zero padded.
Rate card: card used on its expiry date · case 01
Shipments on the expiry date are still rated with the expired card.
Rate card: card used on its expiry date · case 02
Shipments on the expiry date are still rated with the expired card.
Rate card: card used on its expiry date · case 03
Shipments on the expiry date are still rated with the expired card.
Rate card: card used on its expiry date · case 04
Shipments on the expiry date are still rated with the expired card.
Rate card: card used on its expiry date · case 05
Shipments on the expiry date are still rated with the expired card.
Rate card: new card ignored on its first day · case 01
Shipments on the effective date of a new card are rated with the previous card.
Rate card: new card ignored on its first day · case 02
Shipments on the effective date of a new card are rated with the previous card.
Rate card: new card ignored on its first day · case 03
Shipments on the effective date of a new card are rated with the previous card.
Rate card: new card ignored on its first day · case 04
Shipments on the effective date of a new card are rated with the previous card.
Rate card: new card ignored on its first day · case 05
Shipments on the effective date of a new card are rated with the previous card.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗