Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Loss development and trend to rating period: Policy period midpoint treated like an accident year · case 01
Trend is six months short for annual policies.
Loss development and trend to rating period: Policy period midpoint treated like an accident year · case 02
Trend is six months short for annual policies.
Loss development and trend to rating period: Policy period midpoint treated like an accident year · case 03
Trend is six months short for annual policies.
Loss development and trend to rating period: Policy period midpoint treated like an accident year · case 04
Trend is six months short for annual policies.
Loss development and trend to rating period: Policy period midpoint treated like an accident year · case 05
Trend is six months short for annual policies.
Loss development and trend to rating period: Remainder months are trended as whole years · case 01
Trend factors explode when the span is not a whole number of years.
Loss development and trend to rating period: Remainder months are trended as whole years · case 02
Trend factors explode when the span is not a whole number of years.
Loss development and trend to rating period: Remainder months are trended as whole years · case 03
Trend factors explode when the span is not a whole number of years.
Loss development and trend to rating period: Remainder months are trended as whole years · case 04
Trend factors explode when the span is not a whole number of years.
Loss development and trend to rating period: Remainder months are trended as whole years · case 05
Trend factors explode when the span is not a whole number of years.
Loss development and trend to rating period: Evaluation ages between anniversaries are accepted · case 01
An 18-month evaluation is developed with annual factors.
Loss development and trend to rating period: Evaluation ages between anniversaries are accepted · case 02
An 18-month evaluation is developed with annual factors.
Loss development and trend to rating period: Evaluation ages between anniversaries are accepted · case 03
An 18-month evaluation is developed with annual factors.
Loss development and trend to rating period: Evaluation ages between anniversaries are accepted · case 04
An 18-month evaluation is developed with annual factors.
Loss development and trend to rating period: Evaluation ages between anniversaries are accepted · case 05
An 18-month evaluation is developed with annual factors.
Parallelogram on-level factor: Current-year change area is not triangular · case 01
Changes within the calendar year are credited with too much earned exposure.
Parallelogram on-level factor: Current-year change area is not triangular · case 02
Changes within the calendar year are credited with too much earned exposure.
Parallelogram on-level factor: Current-year change area is not triangular · case 03
Changes within the calendar year are credited with too much earned exposure.
Parallelogram on-level factor: Current-year change area is not triangular · case 04
Changes within the calendar year are credited with too much earned exposure.
Parallelogram on-level factor: Current-year change area is not triangular · case 05
Changes within the calendar year are credited with too much earned exposure.
Parallelogram on-level factor: Prior-year change area is complemented incorrectly · case 01
Changes effective late in the prior year are treated as barely earned.
Parallelogram on-level factor: Prior-year change area is complemented incorrectly · case 02
Changes effective late in the prior year are treated as barely earned.
Parallelogram on-level factor: Prior-year change area is complemented incorrectly · case 03
Changes effective late in the prior year are treated as barely earned.
Parallelogram on-level factor: Prior-year change area is complemented incorrectly · case 04
Changes effective late in the prior year are treated as barely earned.
Parallelogram on-level factor: Prior-year change area is complemented incorrectly · case 05
Changes effective late in the prior year are treated as barely earned.
Parallelogram on-level factor: Rate levels are added instead of compounded · case 01
Sequences of rate changes produce the wrong current level.
Parallelogram on-level factor: Rate levels are added instead of compounded · case 02
Sequences of rate changes produce the wrong current level.
Parallelogram on-level factor: Rate levels are added instead of compounded · case 03
Sequences of rate changes produce the wrong current level.
Parallelogram on-level factor: Rate levels are added instead of compounded · case 04
Sequences of rate changes produce the wrong current level.
Parallelogram on-level factor: Rate levels are added instead of compounded · case 05
Sequences of rate changes produce the wrong current level.
Parallelogram on-level factor: Each segment is weighted with the following rate level · case 01
Average earned level is overstated so the on-level factor is too small.
Parallelogram on-level factor: Each segment is weighted with the following rate level · case 02
Average earned level is overstated so the on-level factor is too small.
Parallelogram on-level factor: Each segment is weighted with the following rate level · case 03
Average earned level is overstated so the on-level factor is too small.
Parallelogram on-level factor: Each segment is weighted with the following rate level · case 04
Average earned level is overstated so the on-level factor is too small.
Parallelogram on-level factor: Each segment is weighted with the following rate level · case 05
Average earned level is overstated so the on-level factor is too small.
Parallelogram on-level factor: On-level factor is inverted · case 01
Historical premium is deflated when rates have risen.
Parallelogram on-level factor: On-level factor is inverted · case 02
Historical premium is deflated when rates have risen.
Parallelogram on-level factor: On-level factor is inverted · case 03
Historical premium is deflated when rates have risen.
Parallelogram on-level factor: On-level factor is inverted · case 04
Historical premium is deflated when rates have risen.
Parallelogram on-level factor: On-level factor is inverted · case 05
Historical premium is deflated when rates have risen.
Schedule rating credits and debits: Category caps are not applied · case 01
Stacked items in one category exceed that category's maximum.
Schedule rating credits and debits: Category caps are not applied · case 02
Stacked items in one category exceed that category's maximum.
Schedule rating credits and debits: Category caps are not applied · case 03
Stacked items in one category exceed that category's maximum.
Schedule rating credits and debits: Category caps are not applied · case 04
Stacked items in one category exceed that category's maximum.
Schedule rating credits and debits: Category caps are not applied · case 05
Stacked items in one category exceed that category's maximum.
Schedule rating credits and debits: Later items overwrite earlier items in a category · case 01
Only the last credit or debit in a category counts.
Schedule rating credits and debits: Later items overwrite earlier items in a category · case 02
Only the last credit or debit in a category counts.
Schedule rating credits and debits: Later items overwrite earlier items in a category · case 03
Only the last credit or debit in a category counts.
Schedule rating credits and debits: Later items overwrite earlier items in a category · case 04
Only the last credit or debit in a category counts.
Schedule rating credits and debits: Later items overwrite earlier items in a category · case 05
Only the last credit or debit in a category counts.
Schedule rating credits and debits: Unknown credit categories are silently ignored · case 01
Unfiled schedule categories pass without an error.
Schedule rating credits and debits: Unknown credit categories are silently ignored · case 02
Unfiled schedule categories pass without an error.
Schedule rating credits and debits: Unknown credit categories are silently ignored · case 03
Unfiled schedule categories pass without an error.
Schedule rating credits and debits: Unknown credit categories are silently ignored · case 04
Unfiled schedule categories pass without an error.
Schedule rating credits and debits: Unknown credit categories are silently ignored · case 05
Unfiled schedule categories pass without an error.
Schedule rating credits and debits: Credits are applied as surcharges · case 01
A net credit raises the premium.
Schedule rating credits and debits: Credits are applied as surcharges · case 02
A net credit raises the premium.
Schedule rating credits and debits: Credits are applied as surcharges · case 03
A net credit raises the premium.
Schedule rating credits and debits: Credits are applied as surcharges · case 04
A net credit raises the premium.
Schedule rating credits and debits: Credits are applied as surcharges · case 05
A net credit raises the premium.
Schedule rating credits and debits: Overall credit limit is missing · case 01
Large combined credits exceed the plan's maximum schedule credit.
Schedule rating credits and debits: Overall credit limit is missing · case 02
Large combined credits exceed the plan's maximum schedule credit.
Schedule rating credits and debits: Overall credit limit is missing · case 03
Large combined credits exceed the plan's maximum schedule credit.
Schedule rating credits and debits: Overall credit limit is missing · case 04
Large combined credits exceed the plan's maximum schedule credit.
Schedule rating credits and debits: Overall credit limit is missing · case 05
Large combined credits exceed the plan's maximum schedule credit.
Bonus-malus class transitions: Claim-free years fall below class 1 · case 01
A long claim-free driver drops to class 0 and is rated with the worst coefficient via negative indexing.
Bonus-malus class transitions: Claim-free years fall below class 1 · case 02
A long claim-free driver drops to class 0 and is rated with the worst coefficient via negative indexing.
Bonus-malus class transitions: Claim-free years fall below class 1 · case 03
A long claim-free driver drops to class 0 and is rated with the worst coefficient via negative indexing.
Bonus-malus class transitions: Claim-free years fall below class 1 · case 04
A long claim-free driver drops to class 0 and is rated with the worst coefficient via negative indexing.
Bonus-malus class transitions: Claim-free years fall below class 1 · case 05
A long claim-free driver drops to class 0 and is rated with the worst coefficient via negative indexing.
Bonus-malus class transitions: Multiple claims earn a single penalty · case 01
Two claims in a year only move the policy up three classes.
Bonus-malus class transitions: Multiple claims earn a single penalty · case 02
Two claims in a year only move the policy up three classes.
Bonus-malus class transitions: Multiple claims earn a single penalty · case 03
Two claims in a year only move the policy up three classes.
Bonus-malus class transitions: Multiple claims earn a single penalty · case 04
Two claims in a year only move the policy up three classes.
Bonus-malus class transitions: Multiple claims earn a single penalty · case 05
Two claims in a year only move the policy up three classes.
Bonus-malus class transitions: Three claims do not trigger the worst class · case 01
A year with exactly three claims is rated as an ordinary penalty.
Bonus-malus class transitions: Three claims do not trigger the worst class · case 02
A year with exactly three claims is rated as an ordinary penalty.
Bonus-malus class transitions: Three claims do not trigger the worst class · case 03
A year with exactly three claims is rated as an ordinary penalty.
Bonus-malus class transitions: Three claims do not trigger the worst class · case 04
A year with exactly three claims is rated as an ordinary penalty.
Bonus-malus class transitions: Three claims do not trigger the worst class · case 05
A year with exactly three claims is rated as an ordinary penalty.
Bonus-malus class transitions: The worst class is rejected as a start class · case 01
Renewals starting in class 18 are refused.
Bonus-malus class transitions: The worst class is rejected as a start class · case 02
Renewals starting in class 18 are refused.
Bonus-malus class transitions: The worst class is rejected as a start class · case 03
Renewals starting in class 18 are refused.
Bonus-malus class transitions: The worst class is rejected as a start class · case 04
Renewals starting in class 18 are refused.
Bonus-malus class transitions: The worst class is rejected as a start class · case 05
Renewals starting in class 18 are refused.
Bonus-malus class transitions: Class history records the class before each transition · case 01
The reported class history lags one year behind the transitions.
Bonus-malus class transitions: Class history records the class before each transition · case 02
The reported class history lags one year behind the transitions.
Bonus-malus class transitions: Class history records the class before each transition · case 03
The reported class history lags one year behind the transitions.
Bonus-malus class transitions: Class history records the class before each transition · case 04
The reported class history lags one year behind the transitions.
Bonus-malus class transitions: Class history records the class before each transition · case 05
The reported class history lags one year behind the transitions.
Premium installment schedule: Rounding remainder lands on the first installment · case 01
The first installment is larger than the rest instead of the final one.
Premium installment schedule: Rounding remainder lands on the first installment · case 02
The first installment is larger than the rest instead of the final one.
Premium installment schedule: Rounding remainder lands on the first installment · case 03
The first installment is larger than the rest instead of the final one.
Premium installment schedule: Rounding remainder lands on the first installment · case 04
The first installment is larger than the rest instead of the final one.
Premium installment schedule: Rounding remainder lands on the first installment · case 05
The first installment is larger than the rest instead of the final one.
Premium installment schedule: Down payment is not counted in the payment total · case 01
A 12-pay plan produces 13 payments.
Premium installment schedule: Down payment is not counted in the payment total · case 02
A 12-pay plan produces 13 payments.
Premium installment schedule: Down payment is not counted in the payment total · case 03
A 12-pay plan produces 13 payments.
Premium installment schedule: Down payment is not counted in the payment total · case 04
A 12-pay plan produces 13 payments.
Premium installment schedule: Down payment is not counted in the payment total · case 05
A 12-pay plan produces 13 payments.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗