Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Tip shortfall allocation by hours: allocation eligibility · case 01
Employees who reported exactly their share are listed with zero-cent allocations.
Tip shortfall allocation by hours: allocation eligibility · case 02
Employees who reported exactly their share are listed with zero-cent allocations.
Tip shortfall allocation by hours: allocation eligibility · case 03
Employees who reported exactly their share are listed with zero-cent allocations.
Tip shortfall allocation by hours: allocation eligibility · case 04
Employees who reported exactly their share are listed with zero-cent allocations.
Tip shortfall allocation by hours: allocation eligibility · case 05
Employees who reported exactly their share are listed with zero-cent allocations.
Tip shortfall allocation by hours: hours share base · case 01
The wrong employees are selected for allocation because shares are too small.
Tip shortfall allocation by hours: hours share base · case 02
The wrong employees are selected for allocation because shares are too small.
Tip shortfall allocation by hours: hours share base · case 03
The wrong employees are selected for allocation because shares are too small.
Tip shortfall allocation by hours: hours share base · case 04
The wrong employees are selected for allocation because shares are too small.
Tip shortfall allocation by hours: hours share base · case 05
The wrong employees are selected for allocation because shares are too small.
Nonresident alien wage add-on: FICA base excludes add-on · case 01
Nonresident employees have social security and Medicare withheld on money they never earned.
Nonresident alien wage add-on: FICA base excludes add-on · case 02
Nonresident employees have social security and Medicare withheld on money they never earned.
Nonresident alien wage add-on: FICA base excludes add-on · case 03
Nonresident employees have social security and Medicare withheld on money they never earned.
Nonresident alien wage add-on: FICA base excludes add-on · case 04
Nonresident employees have social security and Medicare withheld on money they never earned.
Nonresident alien wage add-on: FICA base excludes add-on · case 05
Nonresident employees have social security and Medicare withheld on money they never earned.
Nonresident alien wage add-on: student exemption window · case 01
Students in their fifth year lose their FICA exemption early.
Nonresident alien wage add-on: student exemption window · case 02
Students in their fifth year lose their FICA exemption early.
Nonresident alien wage add-on: student exemption window · case 03
Students in their fifth year lose their FICA exemption early.
Nonresident alien wage add-on: student exemption window · case 04
Students in their fifth year lose their FICA exemption early.
Nonresident alien wage add-on: student exemption window · case 05
Students in their fifth year lose their FICA exemption early.
Nonresident alien wage add-on: form vintage table · case 01
Employees with a 2020 form get the pre-2020 add-on amount.
Nonresident alien wage add-on: form vintage table · case 02
Employees with a 2020 form get the pre-2020 add-on amount.
Nonresident alien wage add-on: form vintage table · case 03
Employees with a 2020 form get the pre-2020 add-on amount.
Nonresident alien wage add-on: form vintage table · case 04
Employees with a 2020 form get the pre-2020 add-on amount.
Nonresident alien wage add-on: form vintage table · case 05
Employees with a 2020 form get the pre-2020 add-on amount.
Nonresident alien wage add-on: nonresident-only add-on · case 01
Citizens and resident aliens are over-withheld by the nonresident add-on.
Nonresident alien wage add-on: nonresident-only add-on · case 02
Citizens and resident aliens are over-withheld by the nonresident add-on.
Nonresident alien wage add-on: nonresident-only add-on · case 03
Citizens and resident aliens are over-withheld by the nonresident add-on.
Nonresident alien wage add-on: nonresident-only add-on · case 04
Citizens and resident aliens are over-withheld by the nonresident add-on.
Nonresident alien wage add-on: nonresident-only add-on · case 05
Citizens and resident aliens are over-withheld by the nonresident add-on.
Whole-dollar income tax withholding option: extra before rounding · case 01
Employees with additional withholding end up with odd-cent income tax despite electing whole dollars.
Whole-dollar income tax withholding option: extra before rounding · case 02
Employees with additional withholding end up with odd-cent income tax despite electing whole dollars.
Whole-dollar income tax withholding option: extra before rounding · case 03
Employees with additional withholding end up with odd-cent income tax despite electing whole dollars.
Whole-dollar income tax withholding option: extra before rounding · case 04
Employees with additional withholding end up with odd-cent income tax despite electing whole dollars.
Whole-dollar income tax withholding option: extra before rounding · case 05
Employees with additional withholding end up with odd-cent income tax despite electing whole dollars.
Whole-dollar income tax withholding option: rounding scope · case 01
Total tax withheld does not equal income tax plus FICA.
Whole-dollar income tax withholding option: rounding scope · case 02
Total tax withheld does not equal income tax plus FICA.
Whole-dollar income tax withholding option: rounding scope · case 03
Total tax withheld does not equal income tax plus FICA.
Whole-dollar income tax withholding option: rounding scope · case 04
Total tax withheld does not equal income tax plus FICA.
Whole-dollar income tax withholding option: rounding scope · case 05
Total tax withheld does not equal income tax plus FICA.
Daily plan-change proration: remaining day count · case 01
Customers who upgrade are charged and credited for one day fewer than remain in the period.
Daily plan-change proration: remaining day count · case 02
Customers who upgrade are charged and credited for one day fewer than remain in the period.
Daily plan-change proration: remaining day count · case 03
Customers who upgrade are charged and credited for one day fewer than remain in the period.
Daily plan-change proration: remaining day count · case 04
Customers who upgrade are charged and credited for one day fewer than remain in the period.
Daily plan-change proration: remaining day count · case 05
Customers who upgrade are charged and credited for one day fewer than remain in the period.
Daily plan-change proration: period length basis · case 01
Proration amounts are skewed in 31-day months and February, and exceed the plan price on annual plans.
Daily plan-change proration: period length basis · case 02
Proration amounts are skewed in 31-day months and February, and exceed the plan price on annual plans.
Daily plan-change proration: period length basis · case 03
Proration amounts are skewed in 31-day months and February, and exceed the plan price on annual plans.
Daily plan-change proration: period length basis · case 04
Proration amounts are skewed in 31-day months and February, and exceed the plan price on annual plans.
Daily plan-change proration: period length basis · case 05
Proration amounts are skewed in 31-day months and February, and exceed the plan price on annual plans.
Daily plan-change proration: net rounding composition · case 01
The amount due differs by a cent from the printed charge minus the printed credit.
Daily plan-change proration: net rounding composition · case 02
The amount due differs by a cent from the printed charge minus the printed credit.
Daily plan-change proration: net rounding composition · case 03
The amount due differs by a cent from the printed charge minus the printed credit.
Daily plan-change proration: net rounding composition · case 04
The amount due differs by a cent from the printed charge minus the printed credit.
Daily plan-change proration: net rounding composition · case 05
The amount due differs by a cent from the printed charge minus the printed credit.
Daily plan-change proration: downgrade credit balance · case 01
Downgrades produce a negative amount due on the invoice.
Daily plan-change proration: downgrade credit balance · case 02
Downgrades produce a negative amount due on the invoice.
Daily plan-change proration: downgrade credit balance · case 03
Downgrades produce a negative amount due on the invoice.
Daily plan-change proration: downgrade credit balance · case 04
Downgrades produce a negative amount due on the invoice.
Daily plan-change proration: downgrade credit balance · case 05
Downgrades produce a negative amount due on the invoice.
Second-granular proration line items: credit line rounding direction · case 01
Credit lines ending in exactly half a cent are a cent smaller than the matching debit lines.
Second-granular proration line items: credit line rounding direction · case 02
Credit lines ending in exactly half a cent are a cent smaller than the matching debit lines.
Second-granular proration line items: credit line rounding direction · case 03
Credit lines ending in exactly half a cent are a cent smaller than the matching debit lines.
Second-granular proration line items: credit line rounding direction · case 04
Credit lines ending in exactly half a cent are a cent smaller than the matching debit lines.
Second-granular proration line items: credit line rounding direction · case 05
Credit lines ending in exactly half a cent are a cent smaller than the matching debit lines.
Second-granular proration line items: new quantity scaling · case 01
Seat upgrades are charged for a single seat on the new plan.
Second-granular proration line items: new quantity scaling · case 02
Seat upgrades are charged for a single seat on the new plan.
Second-granular proration line items: new quantity scaling · case 03
Seat upgrades are charged for a single seat on the new plan.
Second-granular proration line items: new quantity scaling · case 04
Seat upgrades are charged for a single seat on the new plan.
Second-granular proration line items: new quantity scaling · case 05
Seat upgrades are charged for a single seat on the new plan.
Second-granular proration line items: change at period end · case 01
A change exactly at renewal emits zero-amount proration lines.
Second-granular proration line items: change at period end · case 02
A change exactly at renewal emits zero-amount proration lines.
Second-granular proration line items: change at period end · case 03
A change exactly at renewal emits zero-amount proration lines.
Second-granular proration line items: change at period end · case 04
A change exactly at renewal emits zero-amount proration lines.
Second-granular proration line items: change at period end · case 05
A change exactly at renewal emits zero-amount proration lines.
Second-granular proration line items: zero-value line suppression · case 01
Moving from zero seats produces an empty zero-cent credit line.
Second-granular proration line items: zero-value line suppression · case 02
Moving from zero seats produces an empty zero-cent credit line.
Second-granular proration line items: zero-value line suppression · case 03
Moving from zero seats produces an empty zero-cent credit line.
Second-granular proration line items: zero-value line suppression · case 04
Moving from zero seats produces an empty zero-cent credit line.
Second-granular proration line items: zero-value line suppression · case 05
Moving from zero seats produces an empty zero-cent credit line.
Month-end anchored renewal dates: anchor day drift · case 01
After a short month, renewals stay on the 28th or 30th for the rest of the subscription.
Month-end anchored renewal dates: anchor day drift · case 02
After a short month, renewals stay on the 28th or 30th for the rest of the subscription.
Month-end anchored renewal dates: anchor day drift · case 03
After a short month, renewals stay on the 28th or 30th for the rest of the subscription.
Month-end anchored renewal dates: anchor day drift · case 04
After a short month, renewals stay on the 28th or 30th for the rest of the subscription.
Month-end anchored renewal dates: anchor day drift · case 05
After a short month, renewals stay on the 28th or 30th for the rest of the subscription.
Month-end anchored renewal dates: year carry · case 01
Renewals that roll into January keep the old year.
Month-end anchored renewal dates: year carry · case 02
Renewals that roll into January keep the old year.
Month-end anchored renewal dates: year carry · case 03
Renewals that roll into January keep the old year.
Month-end anchored renewal dates: year carry · case 04
Renewals that roll into January keep the old year.
Month-end anchored renewal dates: year carry · case 05
Renewals that roll into January keep the old year.
Month-end anchored renewal dates: leap-year month length · case 01
Renewals in February of a leap year land on the 28th instead of the 29th.
Month-end anchored renewal dates: leap-year month length · case 02
Renewals in February of a leap year land on the 28th instead of the 29th.
Month-end anchored renewal dates: leap-year month length · case 03
Renewals in February of a leap year land on the 28th instead of the 29th.
Month-end anchored renewal dates: leap-year month length · case 04
Renewals in February of a leap year land on the 28th instead of the 29th.
Month-end anchored renewal dates: leap-year month length · case 05
Renewals in February of a leap year land on the 28th instead of the 29th.
Month-end anchored renewal dates: zero-based month index · case 01
Every renewal is scheduled one month late.
Month-end anchored renewal dates: zero-based month index · case 02
Every renewal is scheduled one month late.
Month-end anchored renewal dates: zero-based month index · case 03
Every renewal is scheduled one month late.
Month-end anchored renewal dates: zero-based month index · case 04
Every renewal is scheduled one month late.
Month-end anchored renewal dates: zero-based month index · case 05
Every renewal is scheduled one month late.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗