Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Premium installment schedule: Installment fee is charged on the down payment · case 01
The down payment is surcharged with an installment fee.
Premium installment schedule: Installment fee is charged on the down payment · case 02
The down payment is surcharged with an installment fee.
Premium installment schedule: Installment fee is charged on the down payment · case 03
The down payment is surcharged with an installment fee.
Premium installment schedule: Installment fee is charged on the down payment · case 04
The down payment is surcharged with an installment fee.
Premium installment schedule: Installment fee is charged on the down payment · case 05
The down payment is surcharged with an installment fee.
Premium installment schedule: Paid-in-full policies are charged an installment fee · case 01
A single-payment plan bills premium plus an installment fee.
Premium installment schedule: Paid-in-full policies are charged an installment fee · case 02
A single-payment plan bills premium plus an installment fee.
Premium installment schedule: Paid-in-full policies are charged an installment fee · case 03
A single-payment plan bills premium plus an installment fee.
Premium installment schedule: Paid-in-full policies are charged an installment fee · case 04
A single-payment plan bills premium plus an installment fee.
Premium installment schedule: Paid-in-full policies are charged an installment fee · case 05
A single-payment plan bills premium plus an installment fee.
Premium installment schedule: Level installments are rounded up · case 01
Level installments exceed the balance share and the last installment is reduced.
Premium installment schedule: Level installments are rounded up · case 02
Level installments exceed the balance share and the last installment is reduced.
Premium installment schedule: Level installments are rounded up · case 03
Level installments exceed the balance share and the last installment is reduced.
Premium installment schedule: Level installments are rounded up · case 04
Level installments exceed the balance share and the last installment is reduced.
Premium installment schedule: Level installments are rounded up · case 05
Level installments exceed the balance share and the last installment is reduced.
Multi-policy discount stacking: Duplicate discount codes are double counted · case 01
A discount reported twice is granted twice.
Multi-policy discount stacking: Duplicate discount codes are double counted · case 02
A discount reported twice is granted twice.
Multi-policy discount stacking: Duplicate discount codes are double counted · case 03
A discount reported twice is granted twice.
Multi-policy discount stacking: Duplicate discount codes are double counted · case 04
A discount reported twice is granted twice.
Multi-policy discount stacking: Duplicate discount codes are double counted · case 05
A discount reported twice is granted twice.
Multi-policy discount stacking: Discounts within a group are compounded · case 01
Same-group discounts are worth less than the filed additive total.
Multi-policy discount stacking: Discounts within a group are compounded · case 02
Same-group discounts are worth less than the filed additive total.
Multi-policy discount stacking: Discounts within a group are compounded · case 03
Same-group discounts are worth less than the filed additive total.
Multi-policy discount stacking: Discounts within a group are compounded · case 04
Same-group discounts are worth less than the filed additive total.
Multi-policy discount stacking: Discounts within a group are compounded · case 05
Same-group discounts are worth less than the filed additive total.
Multi-policy discount stacking: Discount groups are added instead of multiplied · case 01
Policies with several discount groups receive too large a discount.
Multi-policy discount stacking: Discount groups are added instead of multiplied · case 02
Policies with several discount groups receive too large a discount.
Multi-policy discount stacking: Discount groups are added instead of multiplied · case 03
Policies with several discount groups receive too large a discount.
Multi-policy discount stacking: Discount groups are added instead of multiplied · case 04
Policies with several discount groups receive too large a discount.
Multi-policy discount stacking: Discount groups are added instead of multiplied · case 05
Policies with several discount groups receive too large a discount.
Multi-policy discount stacking: Total discount cap is ignored · case 01
Stacked discounts exceed the maximum allowed discount.
Multi-policy discount stacking: Total discount cap is ignored · case 02
Stacked discounts exceed the maximum allowed discount.
Multi-policy discount stacking: Total discount cap is ignored · case 03
Stacked discounts exceed the maximum allowed discount.
Multi-policy discount stacking: Total discount cap is ignored · case 04
Stacked discounts exceed the maximum allowed discount.
Multi-policy discount stacking: Total discount cap is ignored · case 05
Stacked discounts exceed the maximum allowed discount.
Multi-policy discount stacking: Discounted premium falls below the minimum · case 01
Heavily discounted small policies are billed below the minimum premium.
Multi-policy discount stacking: Discounted premium falls below the minimum · case 02
Heavily discounted small policies are billed below the minimum premium.
Multi-policy discount stacking: Discounted premium falls below the minimum · case 03
Heavily discounted small policies are billed below the minimum premium.
Multi-policy discount stacking: Discounted premium falls below the minimum · case 04
Heavily discounted small policies are billed below the minimum premium.
Multi-policy discount stacking: Discounted premium falls below the minimum · case 05
Heavily discounted small policies are billed below the minimum premium.
Workers compensation payroll audit premium: All overtime pay is excluded · case 01
Premium is understated for classes that work overtime.
Workers compensation payroll audit premium: All overtime pay is excluded · case 02
Premium is understated for classes that work overtime.
Workers compensation payroll audit premium: All overtime pay is excluded · case 03
Premium is understated for classes that work overtime.
Workers compensation payroll audit premium: All overtime pay is excluded · case 04
Premium is understated for classes that work overtime.
Workers compensation payroll audit premium: All overtime pay is excluded · case 05
Premium is understated for classes that work overtime.
Workers compensation payroll audit premium: Officer cap is applied to gross pay · case 01
Officers near the cap lose their overtime exclusion.
Workers compensation payroll audit premium: Officer cap is applied to gross pay · case 02
Officers near the cap lose their overtime exclusion.
Workers compensation payroll audit premium: Officer cap is applied to gross pay · case 03
Officers near the cap lose their overtime exclusion.
Workers compensation payroll audit premium: Officer cap is applied to gross pay · case 04
Officers near the cap lose their overtime exclusion.
Workers compensation payroll audit premium: Officer cap is applied to gross pay · case 05
Officers near the cap lose their overtime exclusion.
Workers compensation payroll audit premium: Officer minimum payroll is not enforced · case 01
Part-time officers are rated below the statutory minimum payroll.
Workers compensation payroll audit premium: Officer minimum payroll is not enforced · case 02
Part-time officers are rated below the statutory minimum payroll.
Workers compensation payroll audit premium: Officer minimum payroll is not enforced · case 03
Part-time officers are rated below the statutory minimum payroll.
Workers compensation payroll audit premium: Officer minimum payroll is not enforced · case 04
Part-time officers are rated below the statutory minimum payroll.
Workers compensation payroll audit premium: Officer minimum payroll is not enforced · case 05
Part-time officers are rated below the statutory minimum payroll.
Workers compensation payroll audit premium: Premium is rounded per employee instead of per class · case 01
Audits with many employees drift by cents from the class-level premium.
Workers compensation payroll audit premium: Premium is rounded per employee instead of per class · case 02
Audits with many employees drift by cents from the class-level premium.
Workers compensation payroll audit premium: Premium is rounded per employee instead of per class · case 03
Audits with many employees drift by cents from the class-level premium.
Workers compensation payroll audit premium: Premium is rounded per employee instead of per class · case 04
Audits with many employees drift by cents from the class-level premium.
Workers compensation payroll audit premium: Premium is rounded per employee instead of per class · case 05
Audits with many employees drift by cents from the class-level premium.
Workers compensation payroll audit premium: Unknown class codes are skipped · case 01
Payroll reported under an unfiled class code disappears from the audit.
Workers compensation payroll audit premium: Unknown class codes are skipped · case 02
Payroll reported under an unfiled class code disappears from the audit.
Workers compensation payroll audit premium: Unknown class codes are skipped · case 03
Payroll reported under an unfiled class code disappears from the audit.
Workers compensation payroll audit premium: Unknown class codes are skipped · case 04
Payroll reported under an unfiled class code disappears from the audit.
Workers compensation payroll audit premium: Unknown class codes are skipped · case 05
Payroll reported under an unfiled class code disappears from the audit.
Calendar period earned and unearned premium: Overlap ignores the period start · case 01
Policies written before the period are credited with earnings from before it began.
Calendar period earned and unearned premium: Overlap ignores the period start · case 02
Policies written before the period are credited with earnings from before it began.
Calendar period earned and unearned premium: Overlap ignores the period start · case 03
Policies written before the period are credited with earnings from before it began.
Calendar period earned and unearned premium: Overlap ignores the period start · case 04
Policies written before the period are credited with earnings from before it began.
Calendar period earned and unearned premium: Overlap ignores the period start · case 05
Policies written before the period are credited with earnings from before it began.
Calendar period earned and unearned premium: Expired policies contribute negative earnings · case 01
Policies that expired before the period reduce earned premium.
Calendar period earned and unearned premium: Expired policies contribute negative earnings · case 02
Policies that expired before the period reduce earned premium.
Calendar period earned and unearned premium: Expired policies contribute negative earnings · case 03
Policies that expired before the period reduce earned premium.
Calendar period earned and unearned premium: Expired policies contribute negative earnings · case 04
Policies that expired before the period reduce earned premium.
Calendar period earned and unearned premium: Expired policies contribute negative earnings · case 05
Policies that expired before the period reduce earned premium.
Calendar period earned and unearned premium: Policies written on the period end count as unearned · case 01
The unearned reserve includes premium for policies written after the valuation date.
Calendar period earned and unearned premium: Policies written on the period end count as unearned · case 02
The unearned reserve includes premium for policies written after the valuation date.
Calendar period earned and unearned premium: Policies written on the period end count as unearned · case 03
The unearned reserve includes premium for policies written after the valuation date.
Calendar period earned and unearned premium: Policies written on the period end count as unearned · case 04
The unearned reserve includes premium for policies written after the valuation date.
Calendar period earned and unearned premium: Policies written on the period end count as unearned · case 05
The unearned reserve includes premium for policies written after the valuation date.
Calendar period earned and unearned premium: Earned premium is rounded per policy · case 01
Portfolio earned premium differs by cents from the exact total.
Calendar period earned and unearned premium: Earned premium is rounded per policy · case 02
Portfolio earned premium differs by cents from the exact total.
Calendar period earned and unearned premium: Earned premium is rounded per policy · case 03
Portfolio earned premium differs by cents from the exact total.
Calendar period earned and unearned premium: Earned premium is rounded per policy · case 04
Portfolio earned premium differs by cents from the exact total.
Calendar period earned and unearned premium: Earned premium is rounded per policy · case 05
Portfolio earned premium differs by cents from the exact total.
Calendar period earned and unearned premium: Unearned premium is measured from the period start · case 01
The unearned reserve is overstated by the earned portion.
Calendar period earned and unearned premium: Unearned premium is measured from the period start · case 02
The unearned reserve is overstated by the earned portion.
Calendar period earned and unearned premium: Unearned premium is measured from the period start · case 03
The unearned reserve is overstated by the earned portion.
Calendar period earned and unearned premium: Unearned premium is measured from the period start · case 04
The unearned reserve is overstated by the earned portion.
Calendar period earned and unearned premium: Unearned premium is measured from the period start · case 05
The unearned reserve is overstated by the earned portion.
Premium tax, surcharge and stamping fee allocation: Premium tax is charged on the inspection fee · case 01
Premium tax is overstated on inspected policies.
Premium tax, surcharge and stamping fee allocation: Premium tax is charged on the inspection fee · case 02
Premium tax is overstated on inspected policies.
Premium tax, surcharge and stamping fee allocation: Premium tax is charged on the inspection fee · case 03
Premium tax is overstated on inspected policies.
Premium tax, surcharge and stamping fee allocation: Premium tax is charged on the inspection fee · case 04
Premium tax is overstated on inspected policies.
Premium tax, surcharge and stamping fee allocation: Premium tax is charged on the inspection fee · case 05
Premium tax is overstated on inspected policies.
Premium tax, surcharge and stamping fee allocation: Fire surcharge includes the policy fee · case 01
The surcharge is levied on fees.
Premium tax, surcharge and stamping fee allocation: Fire surcharge includes the policy fee · case 02
The surcharge is levied on fees.
Premium tax, surcharge and stamping fee allocation: Fire surcharge includes the policy fee · case 03
The surcharge is levied on fees.
Premium tax, surcharge and stamping fee allocation: Fire surcharge includes the policy fee · case 04
The surcharge is levied on fees.
Premium tax, surcharge and stamping fee allocation: Fire surcharge includes the policy fee · case 05
The surcharge is levied on fees.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗