Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Seat additions billed above the high-water mark: high-water baseline · case 01
Seats removed and re-added in the same period are charged twice.
Seat additions billed above the high-water mark: high-water baseline · case 02
Seats removed and re-added in the same period are charged twice.
Seat additions billed above the high-water mark: high-water baseline · case 03
Seats removed and re-added in the same period are charged twice.
Seat additions billed above the high-water mark: high-water baseline · case 04
Seats removed and re-added in the same period are charged twice.
Seat additions billed above the high-water mark: high-water baseline · case 05
Seats removed and re-added in the same period are charged twice.
Seat additions billed above the high-water mark: no mid-cycle removal credit · case 01
Removing seats mid-cycle reduces the invoice although the contract grants credit only at renewal.
Seat additions billed above the high-water mark: no mid-cycle removal credit · case 02
Removing seats mid-cycle reduces the invoice although the contract grants credit only at renewal.
Seat additions billed above the high-water mark: no mid-cycle removal credit · case 03
Removing seats mid-cycle reduces the invoice although the contract grants credit only at renewal.
Seat additions billed above the high-water mark: no mid-cycle removal credit · case 04
Removing seats mid-cycle reduces the invoice although the contract grants credit only at renewal.
Seat additions billed above the high-water mark: no mid-cycle removal credit · case 05
Removing seats mid-cycle reduces the invoice although the contract grants credit only at renewal.
Seat additions billed above the high-water mark: chronological change order · case 01
Out-of-order change logs charge seats for the wrong number of days.
Seat additions billed above the high-water mark: chronological change order · case 02
Out-of-order change logs charge seats for the wrong number of days.
Seat additions billed above the high-water mark: chronological change order · case 03
Out-of-order change logs charge seats for the wrong number of days.
Seat additions billed above the high-water mark: chronological change order · case 04
Out-of-order change logs charge seats for the wrong number of days.
Seat additions billed above the high-water mark: chronological change order · case 05
Out-of-order change logs charge seats for the wrong number of days.
Seat additions billed above the high-water mark: renewal seat count · case 01
Renewal invoices bill the peak seat count even after seats were removed.
Seat additions billed above the high-water mark: renewal seat count · case 02
Renewal invoices bill the peak seat count even after seats were removed.
Seat additions billed above the high-water mark: renewal seat count · case 03
Renewal invoices bill the peak seat count even after seats were removed.
Seat additions billed above the high-water mark: renewal seat count · case 04
Renewal invoices bill the peak seat count even after seats were removed.
Seat additions billed above the high-water mark: renewal seat count · case 05
Renewal invoices bill the peak seat count even after seats were removed.
Trial conversion stub period: stub cycle length · case 01
Stub invoices after a trial exceed the monthly price when the stub spans a 31-day cycle.
Trial conversion stub period: stub cycle length · case 02
Stub invoices after a trial exceed the monthly price when the stub spans a 31-day cycle.
Trial conversion stub period: stub cycle length · case 03
Stub invoices after a trial exceed the monthly price when the stub spans a 31-day cycle.
Trial conversion stub period: stub cycle length · case 04
Stub invoices after a trial exceed the monthly price when the stub spans a 31-day cycle.
Trial conversion stub period: stub cycle length · case 05
Stub invoices after a trial exceed the monthly price when the stub spans a 31-day cycle.
Trial conversion stub period: previous anchor date · case 01
Stub invoices for trials ending early in the month use a 30-day cycle regardless of the real anchor cycle.
Trial conversion stub period: previous anchor date · case 02
Stub invoices for trials ending early in the month use a 30-day cycle regardless of the real anchor cycle.
Trial conversion stub period: previous anchor date · case 03
Stub invoices for trials ending early in the month use a 30-day cycle regardless of the real anchor cycle.
Trial conversion stub period: previous anchor date · case 04
Stub invoices for trials ending early in the month use a 30-day cycle regardless of the real anchor cycle.
Trial conversion stub period: previous anchor date · case 05
Stub invoices for trials ending early in the month use a 30-day cycle regardless of the real anchor cycle.
Trial conversion stub period: previous anchor January wrap · case 01
January trials that end before the anchor day produce a negative stub charge.
Trial conversion stub period: previous anchor January wrap · case 02
January trials that end before the anchor day produce a negative stub charge.
Trial conversion stub period: previous anchor January wrap · case 03
January trials that end before the anchor day produce a negative stub charge.
Trial conversion stub period: previous anchor January wrap · case 04
January trials that end before the anchor day produce a negative stub charge.
Trial conversion stub period: previous anchor January wrap · case 05
January trials that end before the anchor day produce a negative stub charge.
Coupons on invoices with proration lines: repeating coupon window · case 01
A three-month coupon discounts a fourth invoice.
Coupons on invoices with proration lines: repeating coupon window · case 02
A three-month coupon discounts a fourth invoice.
Coupons on invoices with proration lines: repeating coupon window · case 03
A three-month coupon discounts a fourth invoice.
Coupons on invoices with proration lines: repeating coupon window · case 04
A three-month coupon discounts a fourth invoice.
Coupons on invoices with proration lines: repeating coupon window · case 05
A three-month coupon discounts a fourth invoice.
Coupons on invoices with proration lines: credit lines excluded from percent · case 01
Invoices with an unused-time credit line receive a smaller percentage discount than their charges warrant.
Coupons on invoices with proration lines: credit lines excluded from percent · case 02
Invoices with an unused-time credit line receive a smaller percentage discount than their charges warrant.
Coupons on invoices with proration lines: credit lines excluded from percent · case 03
Invoices with an unused-time credit line receive a smaller percentage discount than their charges warrant.
Coupons on invoices with proration lines: credit lines excluded from percent · case 04
Invoices with an unused-time credit line receive a smaller percentage discount than their charges warrant.
Coupons on invoices with proration lines: credit lines excluded from percent · case 05
Invoices with an unused-time credit line receive a smaller percentage discount than their charges warrant.
Coupons on invoices with proration lines: amount-off cap · case 01
Small invoices with a large amount-off coupon produce a credit balance out of the coupon.
Coupons on invoices with proration lines: amount-off cap · case 02
Small invoices with a large amount-off coupon produce a credit balance out of the coupon.
Coupons on invoices with proration lines: amount-off cap · case 03
Small invoices with a large amount-off coupon produce a credit balance out of the coupon.
Coupons on invoices with proration lines: amount-off cap · case 04
Small invoices with a large amount-off coupon produce a credit balance out of the coupon.
Coupons on invoices with proration lines: amount-off cap · case 05
Small invoices with a large amount-off coupon produce a credit balance out of the coupon.
Coupons on invoices with proration lines: single-use coupon · case 01
A once coupon keeps discounting every renewal.
Coupons on invoices with proration lines: single-use coupon · case 02
A once coupon keeps discounting every renewal.
Coupons on invoices with proration lines: single-use coupon · case 03
A once coupon keeps discounting every renewal.
Coupons on invoices with proration lines: single-use coupon · case 04
A once coupon keeps discounting every renewal.
Coupons on invoices with proration lines: single-use coupon · case 05
A once coupon keeps discounting every renewal.
Paused days extending the billing period: clip pauses to period · case 01
Pauses that straddle the renewal extend the period by days that belong to the next period.
Paused days extending the billing period: clip pauses to period · case 02
Pauses that straddle the renewal extend the period by days that belong to the next period.
Paused days extending the billing period: clip pauses to period · case 03
Pauses that straddle the renewal extend the period by days that belong to the next period.
Paused days extending the billing period: clip pauses to period · case 04
Pauses that straddle the renewal extend the period by days that belong to the next period.
Paused days extending the billing period: clip pauses to period · case 05
Pauses that straddle the renewal extend the period by days that belong to the next period.
Paused days extending the billing period: overlap merge extent · case 01
A short pause nested inside a longer one shortens the extension.
Paused days extending the billing period: overlap merge extent · case 02
A short pause nested inside a longer one shortens the extension.
Paused days extending the billing period: overlap merge extent · case 03
A short pause nested inside a longer one shortens the extension.
Paused days extending the billing period: overlap merge extent · case 04
A short pause nested inside a longer one shortens the extension.
Paused days extending the billing period: overlap merge extent · case 05
A short pause nested inside a longer one shortens the extension.
Paused days extending the billing period: empty interval filter · case 01
Pauses entirely outside the period shorten the subscription.
Paused days extending the billing period: empty interval filter · case 02
Pauses entirely outside the period shorten the subscription.
Paused days extending the billing period: empty interval filter · case 03
Pauses entirely outside the period shorten the subscription.
Paused days extending the billing period: empty interval filter · case 04
Pauses entirely outside the period shorten the subscription.
Paused days extending the billing period: empty interval filter · case 05
Pauses entirely outside the period shorten the subscription.
Cancellation refund for unused days: setup fee exclusion · case 01
Customers who cancel get part of the one-time setup fee back.
Cancellation refund for unused days: setup fee exclusion · case 02
Customers who cancel get part of the one-time setup fee back.
Cancellation refund for unused days: setup fee exclusion · case 03
Customers who cancel get part of the one-time setup fee back.
Cancellation refund for unused days: setup fee exclusion · case 04
Customers who cancel get part of the one-time setup fee back.
Cancellation refund for unused days: setup fee exclusion · case 05
Customers who cancel get part of the one-time setup fee back.
Cancellation refund for unused days: minimum charge term · case 01
Customers cancelling in the first days get refunds exceeding the minimum-charge terms.
Cancellation refund for unused days: minimum charge term · case 02
Customers cancelling in the first days get refunds exceeding the minimum-charge terms.
Cancellation refund for unused days: minimum charge term · case 03
Customers cancelling in the first days get refunds exceeding the minimum-charge terms.
Cancellation refund for unused days: minimum charge term · case 04
Customers cancelling in the first days get refunds exceeding the minimum-charge terms.
Cancellation refund for unused days: minimum charge term · case 05
Customers cancelling in the first days get refunds exceeding the minimum-charge terms.
Cancellation refund for unused days: period-end access · case 01
Customers cancelling at period end lose access immediately.
Cancellation refund for unused days: period-end access · case 02
Customers cancelling at period end lose access immediately.
Cancellation refund for unused days: period-end access · case 03
Customers cancelling at period end lose access immediately.
Cancellation refund for unused days: period-end access · case 04
Customers cancelling at period end lose access immediately.
Cancellation refund for unused days: period-end access · case 05
Customers cancelling at period end lose access immediately.
Cancellation refund for unused days: refund rounding direction · case 01
Refunds sometimes exceed the exact unused value by a cent.
Cancellation refund for unused days: refund rounding direction · case 02
Refunds sometimes exceed the exact unused value by a cent.
Cancellation refund for unused days: refund rounding direction · case 03
Refunds sometimes exceed the exact unused value by a cent.
Cancellation refund for unused days: refund rounding direction · case 04
Refunds sometimes exceed the exact unused value by a cent.
Cancellation refund for unused days: refund rounding direction · case 05
Refunds sometimes exceed the exact unused value by a cent.
Usage tiers after a prorated included allowance: volume tier selection · case 01
Usage landing exactly on a tier limit is priced at the next tier.
Usage tiers after a prorated included allowance: volume tier selection · case 02
Usage landing exactly on a tier limit is priced at the next tier.
Usage tiers after a prorated included allowance: volume tier selection · case 03
Usage landing exactly on a tier limit is priced at the next tier.
Usage tiers after a prorated included allowance: volume tier selection · case 04
Usage landing exactly on a tier limit is priced at the next tier.
Usage tiers after a prorated included allowance: volume tier selection · case 05
Usage landing exactly on a tier limit is priced at the next tier.
Usage tiers after a prorated included allowance: allowance proration · case 01
Customers joining late in the period get the full monthly allowance.
Usage tiers after a prorated included allowance: allowance proration · case 02
Customers joining late in the period get the full monthly allowance.
Usage tiers after a prorated included allowance: allowance proration · case 03
Customers joining late in the period get the full monthly allowance.
Usage tiers after a prorated included allowance: allowance proration · case 04
Customers joining late in the period get the full monthly allowance.
Usage tiers after a prorated included allowance: allowance proration · case 05
Customers joining late in the period get the full monthly allowance.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗