Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Dunning retry schedule and status: terminal status stop · case 01
Unpaid subscriptions keep counting failures or are silently reactivated.
Dunning retry schedule and status: terminal status stop · case 02
Unpaid subscriptions keep counting failures or are silently reactivated.
Dunning retry schedule and status: terminal status stop · case 03
Unpaid subscriptions keep counting failures or are silently reactivated.
Dunning retry schedule and status: terminal status stop · case 04
Unpaid subscriptions keep counting failures or are silently reactivated.
Dunning retry schedule and status: terminal status stop · case 05
Unpaid subscriptions keep counting failures or are silently reactivated.
Sequential plan changes within one cycle: same-time collapse · case 01
Two changes submitted in the same second produce two sets of proration lines.
Sequential plan changes within one cycle: same-time collapse · case 02
Two changes submitted in the same second produce two sets of proration lines.
Sequential plan changes within one cycle: same-time collapse · case 03
Two changes submitted in the same second produce two sets of proration lines.
Sequential plan changes within one cycle: same-time collapse · case 04
Two changes submitted in the same second produce two sets of proration lines.
Sequential plan changes within one cycle: same-time collapse · case 05
Two changes submitted in the same second produce two sets of proration lines.
Sequential plan changes within one cycle: current price tracking · case 01
The second plan change credits the original plan instead of the plan in effect.
Sequential plan changes within one cycle: current price tracking · case 02
The second plan change credits the original plan instead of the plan in effect.
Sequential plan changes within one cycle: current price tracking · case 03
The second plan change credits the original plan instead of the plan in effect.
Sequential plan changes within one cycle: current price tracking · case 04
The second plan change credits the original plan instead of the plan in effect.
Sequential plan changes within one cycle: current price tracking · case 05
The second plan change credits the original plan instead of the plan in effect.
Sequential plan changes within one cycle: no-op change skip · case 01
Switching to the same plan emits a pair of offsetting lines.
Sequential plan changes within one cycle: no-op change skip · case 02
Switching to the same plan emits a pair of offsetting lines.
Sequential plan changes within one cycle: no-op change skip · case 03
Switching to the same plan emits a pair of offsetting lines.
Sequential plan changes within one cycle: no-op change skip · case 04
Switching to the same plan emits a pair of offsetting lines.
Sequential plan changes within one cycle: no-op change skip · case 05
Switching to the same plan emits a pair of offsetting lines.
Sequential plan changes within one cycle: change window · case 01
Changes timestamped before the period start produce prorations larger than the period price.
Sequential plan changes within one cycle: change window · case 02
Changes timestamped before the period start produce prorations larger than the period price.
Sequential plan changes within one cycle: change window · case 03
Changes timestamped before the period start produce prorations larger than the period price.
Sequential plan changes within one cycle: change window · case 04
Changes timestamped before the period start produce prorations larger than the period price.
Sequential plan changes within one cycle: change window · case 05
Changes timestamped before the period start produce prorations larger than the period price.
Annual plan proration across leap years: term length · case 01
Terms that include Feb 29 prorate at a daily rate that sums to more than the annual price.
Annual plan proration across leap years: term length · case 02
Terms that include Feb 29 prorate at a daily rate that sums to more than the annual price.
Annual plan proration across leap years: term length · case 03
Terms that include Feb 29 prorate at a daily rate that sums to more than the annual price.
Annual plan proration across leap years: term length · case 04
Terms that include Feb 29 prorate at a daily rate that sums to more than the annual price.
Annual plan proration across leap years: term length · case 05
Terms that include Feb 29 prorate at a daily rate that sums to more than the annual price.
Annual plan proration across leap years: range start clip · case 01
Ranges that begin before the term start are billed for days outside the term.
Annual plan proration across leap years: range start clip · case 02
Ranges that begin before the term start are billed for days outside the term.
Annual plan proration across leap years: range start clip · case 03
Ranges that begin before the term start are billed for days outside the term.
Annual plan proration across leap years: range start clip · case 04
Ranges that begin before the term start are billed for days outside the term.
Annual plan proration across leap years: range start clip · case 05
Ranges that begin before the term start are billed for days outside the term.
Annual plan proration across leap years: exclusive range end · case 01
Every prorated range is billed one day too many.
Annual plan proration across leap years: exclusive range end · case 02
Every prorated range is billed one day too many.
Annual plan proration across leap years: exclusive range end · case 03
Every prorated range is billed one day too many.
Annual plan proration across leap years: exclusive range end · case 04
Every prorated range is billed one day too many.
Annual plan proration across leap years: exclusive range end · case 05
Every prorated range is billed one day too many.
Minimum threshold for invoicing prorations: carried amount inclusion · case 01
Small deferred prorations from earlier cycles are never billed.
Minimum threshold for invoicing prorations: carried amount inclusion · case 02
Small deferred prorations from earlier cycles are never billed.
Minimum threshold for invoicing prorations: carried amount inclusion · case 03
Small deferred prorations from earlier cycles are never billed.
Minimum threshold for invoicing prorations: carried amount inclusion · case 04
Small deferred prorations from earlier cycles are never billed.
Minimum threshold for invoicing prorations: carried amount inclusion · case 05
Small deferred prorations from earlier cycles are never billed.
Minimum threshold for invoicing prorations: invoice threshold · case 01
Pending amounts exactly at the threshold are carried instead of invoiced.
Minimum threshold for invoicing prorations: invoice threshold · case 02
Pending amounts exactly at the threshold are carried instead of invoiced.
Minimum threshold for invoicing prorations: invoice threshold · case 03
Pending amounts exactly at the threshold are carried instead of invoiced.
Minimum threshold for invoicing prorations: invoice threshold · case 04
Pending amounts exactly at the threshold are carried instead of invoiced.
Minimum threshold for invoicing prorations: invoice threshold · case 05
Pending amounts exactly at the threshold are carried instead of invoiced.
Minimum threshold for invoicing prorations: credit threshold · case 01
Tiny credits are applied to balances immediately instead of being carried.
Minimum threshold for invoicing prorations: credit threshold · case 02
Tiny credits are applied to balances immediately instead of being carried.
Minimum threshold for invoicing prorations: credit threshold · case 03
Tiny credits are applied to balances immediately instead of being carried.
Minimum threshold for invoicing prorations: credit threshold · case 04
Tiny credits are applied to balances immediately instead of being carried.
Minimum threshold for invoicing prorations: credit threshold · case 05
Tiny credits are applied to balances immediately instead of being carried.
Flat fee proration with metered usage: usage not prorated · case 01
Late joiners are billed for only a fraction of the usage they consumed.
Flat fee proration with metered usage: usage not prorated · case 02
Late joiners are billed for only a fraction of the usage they consumed.
Flat fee proration with metered usage: usage not prorated · case 03
Late joiners are billed for only a fraction of the usage they consumed.
Flat fee proration with metered usage: usage not prorated · case 04
Late joiners are billed for only a fraction of the usage they consumed.
Flat fee proration with metered usage: usage not prorated · case 05
Late joiners are billed for only a fraction of the usage they consumed.
Flat fee proration with metered usage: usage window end · case 01
Usage recorded on the cancellation day is billed after cancellation.
Flat fee proration with metered usage: usage window end · case 02
Usage recorded on the cancellation day is billed after cancellation.
Flat fee proration with metered usage: usage window end · case 03
Usage recorded on the cancellation day is billed after cancellation.
Flat fee proration with metered usage: usage window end · case 04
Usage recorded on the cancellation day is billed after cancellation.
Flat fee proration with metered usage: usage window end · case 05
Usage recorded on the cancellation day is billed after cancellation.
Flat fee proration with metered usage: cancel beyond period · case 01
Cancellations scheduled after period end over-charge the flat fee beyond one period.
Flat fee proration with metered usage: cancel beyond period · case 02
Cancellations scheduled after period end over-charge the flat fee beyond one period.
Flat fee proration with metered usage: cancel beyond period · case 03
Cancellations scheduled after period end over-charge the flat fee beyond one period.
Flat fee proration with metered usage: cancel beyond period · case 04
Cancellations scheduled after period end over-charge the flat fee beyond one period.
Flat fee proration with metered usage: cancel beyond period · case 05
Cancellations scheduled after period end over-charge the flat fee beyond one period.
Immediate upgrades and scheduled downgrades: upgrade clears schedule · case 01
Customers who upgrade are later silently downgraded by an old scheduled change.
Immediate upgrades and scheduled downgrades: upgrade clears schedule · case 02
Customers who upgrade are later silently downgraded by an old scheduled change.
Immediate upgrades and scheduled downgrades: upgrade clears schedule · case 03
Customers who upgrade are later silently downgraded by an old scheduled change.
Immediate upgrades and scheduled downgrades: upgrade clears schedule · case 04
Customers who upgrade are later silently downgraded by an old scheduled change.
Immediate upgrades and scheduled downgrades: upgrade clears schedule · case 05
Customers who upgrade are later silently downgraded by an old scheduled change.
Immediate upgrades and scheduled downgrades: downgrade deferral · case 01
Downgrades take effect immediately without credit, cutting service mid-period.
Immediate upgrades and scheduled downgrades: downgrade deferral · case 02
Downgrades take effect immediately without credit, cutting service mid-period.
Immediate upgrades and scheduled downgrades: downgrade deferral · case 03
Downgrades take effect immediately without credit, cutting service mid-period.
Immediate upgrades and scheduled downgrades: downgrade deferral · case 04
Downgrades take effect immediately without credit, cutting service mid-period.
Immediate upgrades and scheduled downgrades: downgrade deferral · case 05
Downgrades take effect immediately without credit, cutting service mid-period.
Immediate upgrades and scheduled downgrades: upgrade charge basis · case 01
Upgrades charge the full new price for remaining days with no credit for the old plan.
Immediate upgrades and scheduled downgrades: upgrade charge basis · case 02
Upgrades charge the full new price for remaining days with no credit for the old plan.
Immediate upgrades and scheduled downgrades: upgrade charge basis · case 03
Upgrades charge the full new price for remaining days with no credit for the old plan.
Immediate upgrades and scheduled downgrades: upgrade charge basis · case 04
Upgrades charge the full new price for remaining days with no credit for the old plan.
Immediate upgrades and scheduled downgrades: upgrade charge basis · case 05
Upgrades charge the full new price for remaining days with no credit for the old plan.
Volume discount tiers on quantity changes: tier for each quantity · case 01
Customers adding seats across a tier boundary keep the old tier's discount.
Volume discount tiers on quantity changes: tier for each quantity · case 02
Customers adding seats across a tier boundary keep the old tier's discount.
Volume discount tiers on quantity changes: tier for each quantity · case 03
Customers adding seats across a tier boundary keep the old tier's discount.
Volume discount tiers on quantity changes: tier for each quantity · case 04
Customers adding seats across a tier boundary keep the old tier's discount.
Volume discount tiers on quantity changes: tier for each quantity · case 05
Customers adding seats across a tier boundary keep the old tier's discount.
Volume discount tiers on quantity changes: tier threshold inclusion · case 01
Buying exactly the tier minimum does not unlock the discount.
Volume discount tiers on quantity changes: tier threshold inclusion · case 02
Buying exactly the tier minimum does not unlock the discount.
Volume discount tiers on quantity changes: tier threshold inclusion · case 03
Buying exactly the tier minimum does not unlock the discount.
Volume discount tiers on quantity changes: tier threshold inclusion · case 04
Buying exactly the tier minimum does not unlock the discount.
Volume discount tiers on quantity changes: tier threshold inclusion · case 05
Buying exactly the tier minimum does not unlock the discount.
Volume discount tiers on quantity changes: discounted unit rounding · case 01
Discounted prorations drift by cents proportional to the quantity.
Volume discount tiers on quantity changes: discounted unit rounding · case 02
Discounted prorations drift by cents proportional to the quantity.
Volume discount tiers on quantity changes: discounted unit rounding · case 03
Discounted prorations drift by cents proportional to the quantity.
Volume discount tiers on quantity changes: discounted unit rounding · case 04
Discounted prorations drift by cents proportional to the quantity.
Volume discount tiers on quantity changes: discounted unit rounding · case 05
Discounted prorations drift by cents proportional to the quantity.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗