Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Guarded progressive ramp: Healthy intervals resume a rolled-back ramp · case 01
After an automatic rollback the ramp quietly climbs back up without human review.
Guarded progressive ramp: Healthy intervals resume a rolled-back ramp · case 02
After an automatic rollback the ramp quietly climbs back up without human review.
Guarded progressive ramp: Healthy intervals resume a rolled-back ramp · case 03
After an automatic rollback the ramp quietly climbs back up without human review.
Guarded progressive ramp: Healthy intervals resume a rolled-back ramp · case 04
After an automatic rollback the ramp quietly climbs back up without human review.
Guarded progressive ramp: Healthy intervals resume a rolled-back ramp · case 05
After an automatic rollback the ramp quietly climbs back up without human review.
Guarded progressive ramp: An error rate equal to the threshold rolls back · case 01
Launches sitting exactly at the tolerated 2 percent error rate are rolled back.
Guarded progressive ramp: An error rate equal to the threshold rolls back · case 02
Launches sitting exactly at the tolerated 2 percent error rate are rolled back.
Guarded progressive ramp: An error rate equal to the threshold rolls back · case 03
Launches sitting exactly at the tolerated 2 percent error rate are rolled back.
Guarded progressive ramp: An error rate equal to the threshold rolls back · case 04
Launches sitting exactly at the tolerated 2 percent error rate are rolled back.
Guarded progressive ramp: An error rate equal to the threshold rolls back · case 05
Launches sitting exactly at the tolerated 2 percent error rate are rolled back.
Guarded progressive ramp: History lags one stage behind · case 01
Audit history shows the percentage before each advance, hiding when the ramp reached full traffic.
Guarded progressive ramp: History lags one stage behind · case 02
Audit history shows the percentage before each advance, hiding when the ramp reached full traffic.
Guarded progressive ramp: History lags one stage behind · case 03
Audit history shows the percentage before each advance, hiding when the ramp reached full traffic.
Guarded progressive ramp: History lags one stage behind · case 04
Audit history shows the percentage before each advance, hiding when the ramp reached full traffic.
Guarded progressive ramp: History lags one stage behind · case 05
Audit history shows the percentage before each advance, hiding when the ramp reached full traffic.
Guarded progressive ramp: Rolled-back ramps report their last stage · case 01
The summary claims the feature is still at 25 percent after it was rolled back.
Guarded progressive ramp: Rolled-back ramps report their last stage · case 02
The summary claims the feature is still at 25 percent after it was rolled back.
Guarded progressive ramp: Rolled-back ramps report their last stage · case 03
The summary claims the feature is still at 25 percent after it was rolled back.
Guarded progressive ramp: Rolled-back ramps report their last stage · case 04
The summary claims the feature is still at 25 percent after it was rolled back.
Guarded progressive ramp: Rolled-back ramps report their last stage · case 05
The summary claims the feature is still at 25 percent after it was rolled back.
Flag override resolution: Booleans satisfy integer flags · case 01
A remote payload of true for a numeric limit flag is served as 1.
Flag override resolution: Booleans satisfy integer flags · case 02
A remote payload of true for a numeric limit flag is served as 1.
Flag override resolution: Booleans satisfy integer flags · case 03
A remote payload of true for a numeric limit flag is served as 1.
Flag override resolution: Booleans satisfy integer flags · case 04
A remote payload of true for a numeric limit flag is served as 1.
Flag override resolution: Booleans satisfy integer flags · case 05
A remote payload of true for a numeric limit flag is served as 1.
Flag override resolution: Local overrides beat environment variables · case 01
An incident responder's environment pin is ignored on hosts with a local override file.
Flag override resolution: Local overrides beat environment variables · case 02
An incident responder's environment pin is ignored on hosts with a local override file.
Flag override resolution: Local overrides beat environment variables · case 03
An incident responder's environment pin is ignored on hosts with a local override file.
Flag override resolution: Local overrides beat environment variables · case 04
An incident responder's environment pin is ignored on hosts with a local override file.
Flag override resolution: Local overrides beat environment variables · case 05
An incident responder's environment pin is ignored on hosts with a local override file.
Flag override resolution: Dashes survive in environment variable names · case 01
FLAG_DARK-MODE-V2 can never be set from a shell, so the env layer is unreachable for dashed keys.
Flag override resolution: Dashes survive in environment variable names · case 02
FLAG_DARK-MODE-V2 can never be set from a shell, so the env layer is unreachable for dashed keys.
Flag override resolution: Dashes survive in environment variable names · case 03
FLAG_DARK-MODE-V2 can never be set from a shell, so the env layer is unreachable for dashed keys.
Flag override resolution: Dashes survive in environment variable names · case 04
FLAG_DARK-MODE-V2 can never be set from a shell, so the env layer is unreachable for dashed keys.
Flag override resolution: Dashes survive in environment variable names · case 05
FLAG_DARK-MODE-V2 can never be set from a shell, so the env layer is unreachable for dashed keys.
Flag override resolution: An invalid layer short-circuits to the default · case 01
A malformed environment value hides a valid remote value.
Flag override resolution: An invalid layer short-circuits to the default · case 02
A malformed environment value hides a valid remote value.
Flag override resolution: An invalid layer short-circuits to the default · case 03
A malformed environment value hides a valid remote value.
Flag override resolution: An invalid layer short-circuits to the default · case 04
A malformed environment value hides a valid remote value.
Flag override resolution: An invalid layer short-circuits to the default · case 05
A malformed environment value hides a valid remote value.
Flag override resolution: Negative integers from the environment are rejected · case 01
An operator cannot pin an offset flag to -3 via the environment.
Flag override resolution: Negative integers from the environment are rejected · case 02
An operator cannot pin an offset flag to -3 via the environment.
Flag override resolution: Negative integers from the environment are rejected · case 03
An operator cannot pin an offset flag to -3 via the environment.
Flag override resolution: Negative integers from the environment are rejected · case 04
An operator cannot pin an offset flag to -3 via the environment.
Flag override resolution: Negative integers from the environment are rejected · case 05
An operator cannot pin an offset flag to -3 via the environment.
Flag override resolution: Environment booleans accept any text · case 01
FLAG_BETA=TRUE disables the flag and FLAG_BETA=yes silently means false.
Flag override resolution: Environment booleans accept any text · case 02
FLAG_BETA=TRUE disables the flag and FLAG_BETA=yes silently means false.
Flag override resolution: Environment booleans accept any text · case 03
FLAG_BETA=TRUE disables the flag and FLAG_BETA=yes silently means false.
Flag override resolution: Environment booleans accept any text · case 04
FLAG_BETA=TRUE disables the flag and FLAG_BETA=yes silently means false.
Flag override resolution: Environment booleans accept any text · case 05
FLAG_BETA=TRUE disables the flag and FLAG_BETA=yes silently means false.
Bootstrapped flag payload merge: Equal flag versions overwrite local values · case 01
A replayed stream event with the same flag version flips the value back to an older payload.
Bootstrapped flag payload merge: Equal flag versions overwrite local values · case 02
A replayed stream event with the same flag version flips the value back to an older payload.
Bootstrapped flag payload merge: Equal flag versions overwrite local values · case 03
A replayed stream event with the same flag version flips the value back to an older payload.
Bootstrapped flag payload merge: Equal flag versions overwrite local values · case 04
A replayed stream event with the same flag version flips the value back to an older payload.
Bootstrapped flag payload merge: Equal flag versions overwrite local values · case 05
A replayed stream event with the same flag version flips the value back to an older payload.
Bootstrapped flag payload merge: Deleted flags are still returned · case 01
Flags archived on the server keep appearing in the client flag list.
Bootstrapped flag payload merge: Deleted flags are still returned · case 02
Flags archived on the server keep appearing in the client flag list.
Bootstrapped flag payload merge: Deleted flags are still returned · case 03
Flags archived on the server keep appearing in the client flag list.
Bootstrapped flag payload merge: Deleted flags are still returned · case 04
Flags archived on the server keep appearing in the client flag list.
Bootstrapped flag payload merge: Deleted flags are still returned · case 05
Flags archived on the server keep appearing in the client flag list.
Bootstrapped flag payload merge: An equal payload version is treated as stale · case 01
A second payload with the same version but newer per-flag versions is discarded.
Bootstrapped flag payload merge: An equal payload version is treated as stale · case 02
A second payload with the same version but newer per-flag versions is discarded.
Bootstrapped flag payload merge: An equal payload version is treated as stale · case 03
A second payload with the same version but newer per-flag versions is discarded.
Bootstrapped flag payload merge: An equal payload version is treated as stale · case 04
A second payload with the same version but newer per-flag versions is discarded.
Bootstrapped flag payload merge: An equal payload version is treated as stale · case 05
A second payload with the same version but newer per-flag versions is discarded.
Bootstrapped flag payload merge: Flags unknown locally are never added · case 01
Newly created flags never reach clients until a full reload.
Bootstrapped flag payload merge: Flags unknown locally are never added · case 02
Newly created flags never reach clients until a full reload.
Bootstrapped flag payload merge: Flags unknown locally are never added · case 03
Newly created flags never reach clients until a full reload.
Bootstrapped flag payload merge: Flags unknown locally are never added · case 04
Newly created flags never reach clients until a full reload.
Bootstrapped flag payload merge: Flags unknown locally are never added · case 05
Newly created flags never reach clients until a full reload.
Traffic then variant allocation: The variant split reuses the traffic bucket · case 01
At 20 percent traffic every allocated user lands in the first arm.
Traffic then variant allocation: The variant split reuses the traffic bucket · case 02
At 20 percent traffic every allocated user lands in the first arm.
Traffic then variant allocation: The variant split reuses the traffic bucket · case 03
At 20 percent traffic every allocated user lands in the first arm.
Traffic then variant allocation: The variant split reuses the traffic bucket · case 04
At 20 percent traffic every allocated user lands in the first arm.
Traffic then variant allocation: The variant split reuses the traffic bucket · case 05
At 20 percent traffic every allocated user lands in the first arm.
Traffic then variant allocation: Arm weights are scaled by traffic · case 01
At partial traffic most allocated users fall through to the last arm.
Traffic then variant allocation: Arm weights are scaled by traffic · case 02
At partial traffic most allocated users fall through to the last arm.
Traffic then variant allocation: Arm weights are scaled by traffic · case 03
At partial traffic most allocated users fall through to the last arm.
Traffic then variant allocation: Arm weights are scaled by traffic · case 04
At partial traffic most allocated users fall through to the last arm.
Traffic then variant allocation: Arm weights are scaled by traffic · case 05
At partial traffic most allocated users fall through to the last arm.
Traffic then variant allocation: Traffic allocation admits one extra bucket · case 01
A 10 percent experiment enrols 11 percent of users.
Traffic then variant allocation: Traffic allocation admits one extra bucket · case 02
A 10 percent experiment enrols 11 percent of users.
Traffic then variant allocation: Traffic allocation admits one extra bucket · case 03
A 10 percent experiment enrols 11 percent of users.
Traffic then variant allocation: Traffic allocation admits one extra bucket · case 04
A 10 percent experiment enrols 11 percent of users.
Traffic then variant allocation: Traffic allocation admits one extra bucket · case 05
A 10 percent experiment enrols 11 percent of users.
Timestamp targeting clause: ISO dates are converted to seconds · case 01
Every millisecond timestamp is "after" any ISO clause date, so scheduled launches fire immediately.
Timestamp targeting clause: ISO dates are converted to seconds · case 02
Every millisecond timestamp is "after" any ISO clause date, so scheduled launches fire immediately.
Timestamp targeting clause: ISO dates are converted to seconds · case 03
Every millisecond timestamp is "after" any ISO clause date, so scheduled launches fire immediately.
Timestamp targeting clause: ISO dates are converted to seconds · case 04
Every millisecond timestamp is "after" any ISO clause date, so scheduled launches fire immediately.
Timestamp targeting clause: ISO dates are converted to seconds · case 05
Every millisecond timestamp is "after" any ISO clause date, so scheduled launches fire immediately.
Timestamp targeting clause: Booleans are treated as epoch milliseconds · case 01
A user attribute true compares as 1 ms after the epoch and matches "before" clauses.
Timestamp targeting clause: Booleans are treated as epoch milliseconds · case 02
A user attribute true compares as 1 ms after the epoch and matches "before" clauses.
Timestamp targeting clause: Booleans are treated as epoch milliseconds · case 03
A user attribute true compares as 1 ms after the epoch and matches "before" clauses.
Timestamp targeting clause: Booleans are treated as epoch milliseconds · case 04
A user attribute true compares as 1 ms after the epoch and matches "before" clauses.
Timestamp targeting clause: Booleans are treated as epoch milliseconds · case 05
A user attribute true compares as 1 ms after the epoch and matches "before" clauses.
Timestamp targeting clause: Equal timestamps satisfy "before" · case 01
A user created at exactly the clause instant matches both before-rules and the next launch rule.
Timestamp targeting clause: Equal timestamps satisfy "before" · case 02
A user created at exactly the clause instant matches both before-rules and the next launch rule.
Timestamp targeting clause: Equal timestamps satisfy "before" · case 03
A user created at exactly the clause instant matches both before-rules and the next launch rule.
Timestamp targeting clause: Equal timestamps satisfy "before" · case 04
A user created at exactly the clause instant matches both before-rules and the next launch rule.
Timestamp targeting clause: Equal timestamps satisfy "before" · case 05
A user created at exactly the clause instant matches both before-rules and the next launch rule.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗