Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
The surcharge takes the greater of the full rate and the gliding cap · case 01
High incomes pay 11.9% of the excess instead of 5.5% of the tax.
The surcharge takes the greater of the full rate and the gliding cap · case 02
High incomes pay 11.9% of the excess instead of 5.5% of the tax.
The surcharge takes the greater of the full rate and the gliding cap · case 03
High incomes pay 11.9% of the excess instead of 5.5% of the tax.
The surcharge takes the greater of the full rate and the gliding cap · case 04
High incomes pay 11.9% of the excess instead of 5.5% of the tax.
The surcharge takes the greater of the full rate and the gliding cap · case 05
High incomes pay 11.9% of the excess instead of 5.5% of the tax.
Joint assessments use the single exemption limit · case 01
Couples pay the surcharge from half the stipulated joint limit.
Joint assessments use the single exemption limit · case 02
Couples pay the surcharge from half the stipulated joint limit.
Joint assessments use the single exemption limit · case 03
Couples pay the surcharge from half the stipulated joint limit.
Joint assessments use the single exemption limit · case 04
Couples pay the surcharge from half the stipulated joint limit.
Joint assessments use the single exemption limit · case 05
Couples pay the surcharge from half the stipulated joint limit.
The surcharge is rounded to the nearest cent instead of down · case 01
Some surcharges are one cent higher than stipulated.
The surcharge is rounded to the nearest cent instead of down · case 02
Some surcharges are one cent higher than stipulated.
The surcharge is rounded to the nearest cent instead of down · case 03
Some surcharges are one cent higher than stipulated.
The surcharge is rounded to the nearest cent instead of down · case 04
Some surcharges are one cent higher than stipulated.
The surcharge is rounded to the nearest cent instead of down · case 05
Some surcharges are one cent higher than stipulated.
Partial thousands above the threshold do not reduce the credit · case 01
MAGI 1 dollar over the threshold keeps the full credit.
Partial thousands above the threshold do not reduce the credit · case 02
MAGI 1 dollar over the threshold keeps the full credit.
Partial thousands above the threshold do not reduce the credit · case 03
MAGI 1 dollar over the threshold keeps the full credit.
Partial thousands above the threshold do not reduce the credit · case 04
MAGI 1 dollar over the threshold keeps the full credit.
Partial thousands above the threshold do not reduce the credit · case 05
MAGI 1 dollar over the threshold keeps the full credit.
Joint filers use the single phase-out threshold · case 01
Married couples start losing the credit at 200000.
Joint filers use the single phase-out threshold · case 02
Married couples start losing the credit at 200000.
Joint filers use the single phase-out threshold · case 03
Married couples start losing the credit at 200000.
Joint filers use the single phase-out threshold · case 04
Married couples start losing the credit at 200000.
Joint filers use the single phase-out threshold · case 05
Married couples start losing the credit at 200000.
The phase-out drives the credit negative · case 01
Very high earners with children report a negative credit.
The phase-out drives the credit negative · case 02
Very high earners with children report a negative credit.
The phase-out drives the credit negative · case 03
Very high earners with children report a negative credit.
The phase-out drives the credit negative · case 04
Very high earners with children report a negative credit.
The phase-out drives the credit negative · case 05
Very high earners with children report a negative credit.
The refundable portion ignores the credit already used against tax · case 01
A filer receives a refund of credit that already reduced their tax.
The refundable portion ignores the credit already used against tax · case 02
A filer receives a refund of credit that already reduced their tax.
The refundable portion ignores the credit already used against tax · case 03
A filer receives a refund of credit that already reduced their tax.
The refundable portion ignores the credit already used against tax · case 04
A filer receives a refund of credit that already reduced their tax.
The refundable portion ignores the credit already used against tax · case 05
A filer receives a refund of credit that already reduced their tax.
Other dependents raise the refundable cap · case 01
Families with non-child dependents receive larger refunds than stipulated.
Other dependents raise the refundable cap · case 02
Families with non-child dependents receive larger refunds than stipulated.
Other dependents raise the refundable cap · case 03
Families with non-child dependents receive larger refunds than stipulated.
Other dependents raise the refundable cap · case 04
Families with non-child dependents receive larger refunds than stipulated.
Other dependents raise the refundable cap · case 05
Families with non-child dependents receive larger refunds than stipulated.
Personal auto multiplicative rating: Age 25 is still rated as youthful · case 01
A 25-year-old driver is charged the 1.50 youthful factor.
Personal auto multiplicative rating: Age 25 is still rated as youthful · case 02
A 25-year-old driver is charged the 1.50 youthful factor.
Personal auto multiplicative rating: Age 25 is still rated as youthful · case 03
A 25-year-old driver is charged the 1.50 youthful factor.
Personal auto multiplicative rating: Age 25 is still rated as youthful · case 04
A 25-year-old driver is charged the 1.50 youthful factor.
Personal auto multiplicative rating: Age 25 is still rated as youthful · case 05
A 25-year-old driver is charged the 1.50 youthful factor.
Personal auto multiplicative rating: Two-car households lose the multi-car discount · case 01
A household with exactly two cars pays the undiscounted premium.
Personal auto multiplicative rating: Two-car households lose the multi-car discount · case 02
A household with exactly two cars pays the undiscounted premium.
Personal auto multiplicative rating: Two-car households lose the multi-car discount · case 03
A household with exactly two cars pays the undiscounted premium.
Personal auto multiplicative rating: Two-car households lose the multi-car discount · case 04
A household with exactly two cars pays the undiscounted premium.
Personal auto multiplicative rating: Two-car households lose the multi-car discount · case 05
A household with exactly two cars pays the undiscounted premium.
Personal auto multiplicative rating: Symbol 1 carries a surcharge · case 01
Every vehicle symbol is rated one step too high; symbol 1 is not the unit relativity.
Personal auto multiplicative rating: Symbol 1 carries a surcharge · case 02
Every vehicle symbol is rated one step too high; symbol 1 is not the unit relativity.
Personal auto multiplicative rating: Symbol 1 carries a surcharge · case 03
Every vehicle symbol is rated one step too high; symbol 1 is not the unit relativity.
Personal auto multiplicative rating: Symbol 1 carries a surcharge · case 04
Every vehicle symbol is rated one step too high; symbol 1 is not the unit relativity.
Personal auto multiplicative rating: Symbol 1 carries a surcharge · case 05
Every vehicle symbol is rated one step too high; symbol 1 is not the unit relativity.
Personal auto multiplicative rating: Rated premium is truncated instead of rounded half-up · case 01
Premiums with a fractional cent of one half or more come out one cent low.
Personal auto multiplicative rating: Rated premium is truncated instead of rounded half-up · case 02
Premiums with a fractional cent of one half or more come out one cent low.
Personal auto multiplicative rating: Rated premium is truncated instead of rounded half-up · case 03
Premiums with a fractional cent of one half or more come out one cent low.
Personal auto multiplicative rating: Rated premium is truncated instead of rounded half-up · case 04
Premiums with a fractional cent of one half or more come out one cent low.
Personal auto multiplicative rating: Rated premium is truncated instead of rounded half-up · case 05
Premiums with a fractional cent of one half or more come out one cent low.
Personal auto multiplicative rating: Minimum premium is enforced before the multi-car discount · case 01
A discounted small policy is charged less than the 30000-cent minimum premium.
Personal auto multiplicative rating: Minimum premium is enforced before the multi-car discount · case 02
A discounted small policy is charged less than the 30000-cent minimum premium.
Personal auto multiplicative rating: Minimum premium is enforced before the multi-car discount · case 03
A discounted small policy is charged less than the 30000-cent minimum premium.
Personal auto multiplicative rating: Minimum premium is enforced before the multi-car discount · case 04
A discounted small policy is charged less than the 30000-cent minimum premium.
Personal auto multiplicative rating: Minimum premium is enforced before the multi-car discount · case 05
A discounted small policy is charged less than the 30000-cent minimum premium.
Workers compensation experience modification: Medical-only claims are reduced by 30% instead of to 30% · case 01
Employers with medical-only claims receive a higher mod than the plan allows.
Workers compensation experience modification: Medical-only claims are reduced by 30% instead of to 30% · case 02
Employers with medical-only claims receive a higher mod than the plan allows.
Workers compensation experience modification: Medical-only claims are reduced by 30% instead of to 30% · case 03
Employers with medical-only claims receive a higher mod than the plan allows.
Workers compensation experience modification: Medical-only claims are reduced by 30% instead of to 30% · case 04
Employers with medical-only claims receive a higher mod than the plan allows.
Workers compensation experience modification: Medical-only claims are reduced by 30% instead of to 30% · case 05
Employers with medical-only claims receive a higher mod than the plan allows.
Workers compensation experience modification: Primary losses are split on the aggregate instead of per claim · case 01
Several moderate claims are treated as mostly excess, understating the mod.
Workers compensation experience modification: Primary losses are split on the aggregate instead of per claim · case 02
Several moderate claims are treated as mostly excess, understating the mod.
Workers compensation experience modification: Primary losses are split on the aggregate instead of per claim · case 03
Several moderate claims are treated as mostly excess, understating the mod.
Workers compensation experience modification: Primary losses are split on the aggregate instead of per claim · case 04
Several moderate claims are treated as mostly excess, understating the mod.
Workers compensation experience modification: Primary losses are split on the aggregate instead of per claim · case 05
Several moderate claims are treated as mostly excess, understating the mod.
Workers compensation experience modification: Excess credibility weight is applied to the wrong term · case 01
Mods overreact to excess losses and the expected-excess ballast is mostly discarded.
Workers compensation experience modification: Excess credibility weight is applied to the wrong term · case 02
Mods overreact to excess losses and the expected-excess ballast is mostly discarded.
Workers compensation experience modification: Excess credibility weight is applied to the wrong term · case 03
Mods overreact to excess losses and the expected-excess ballast is mostly discarded.
Workers compensation experience modification: Excess credibility weight is applied to the wrong term · case 04
Mods overreact to excess losses and the expected-excess ballast is mostly discarded.
Workers compensation experience modification: Excess credibility weight is applied to the wrong term · case 05
Mods overreact to excess losses and the expected-excess ballast is mostly discarded.
Workers compensation experience modification: Ballast is omitted from the denominator · case 01
Mods are systematically inflated because ballast is added only to the numerator.
Workers compensation experience modification: Ballast is omitted from the denominator · case 02
Mods are systematically inflated because ballast is added only to the numerator.
Workers compensation experience modification: Ballast is omitted from the denominator · case 03
Mods are systematically inflated because ballast is added only to the numerator.
Workers compensation experience modification: Ballast is omitted from the denominator · case 04
Mods are systematically inflated because ballast is added only to the numerator.
Workers compensation experience modification: Ballast is omitted from the denominator · case 05
Mods are systematically inflated because ballast is added only to the numerator.
Workers compensation experience modification: Mod is rounded to three places before two · case 01
The published mod is one point low whenever the third decimal would round it up.
Workers compensation experience modification: Mod is rounded to three places before two · case 02
The published mod is one point low whenever the third decimal would round it up.
Workers compensation experience modification: Mod is rounded to three places before two · case 03
The published mod is one point low whenever the third decimal would round it up.
Workers compensation experience modification: Mod is rounded to three places before two · case 04
The published mod is one point low whenever the third decimal would round it up.
Workers compensation experience modification: Mod is rounded to three places before two · case 05
The published mod is one point low whenever the third decimal would round it up.
Deductible credit interpolation: Credits are extrapolated past the largest deductible · case 01
Very large deductibles receive credits beyond the largest filed credit.
Deductible credit interpolation: Credits are extrapolated past the largest deductible · case 02
Very large deductibles receive credits beyond the largest filed credit.
Deductible credit interpolation: Credits are extrapolated past the largest deductible · case 03
Very large deductibles receive credits beyond the largest filed credit.
Deductible credit interpolation: Credits are extrapolated past the largest deductible · case 04
Very large deductibles receive credits beyond the largest filed credit.
Deductible credit interpolation: Credits are extrapolated past the largest deductible · case 05
Very large deductibles receive credits beyond the largest filed credit.
Deductible credit interpolation: Interpolation weight uses the upper deductible as denominator · case 01
Deductibles between tabulated rows get credits that do not lie on the segment.
Deductible credit interpolation: Interpolation weight uses the upper deductible as denominator · case 02
Deductibles between tabulated rows get credits that do not lie on the segment.
Deductible credit interpolation: Interpolation weight uses the upper deductible as denominator · case 03
Deductibles between tabulated rows get credits that do not lie on the segment.
Deductible credit interpolation: Interpolation weight uses the upper deductible as denominator · case 04
Deductibles between tabulated rows get credits that do not lie on the segment.
Deductible credit interpolation: Interpolation weight uses the upper deductible as denominator · case 05
Deductibles between tabulated rows get credits that do not lie on the segment.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗