Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Always-valid sequential p-value: The ratio uses the latest observation instead of the mean · case 01
One noisy observation swings the evidence regardless of history.
Always-valid sequential p-value: The ratio uses the latest observation instead of the mean · case 02
One noisy observation swings the evidence regardless of history.
Always-valid sequential p-value: The ratio uses the latest observation instead of the mean · case 03
One noisy observation swings the evidence regardless of history.
Always-valid sequential p-value: The ratio uses the latest observation instead of the mean · case 04
One noisy observation swings the evidence regardless of history.
Always-valid sequential p-value: The ratio uses the latest observation instead of the mean · case 05
One noisy observation swings the evidence regardless of history.
Switchback carryover washout: Every block start is washed out · case 01
Data after repeated-arm blocks is discarded, wasting sample for no carryover.
Switchback carryover washout: Every block start is washed out · case 02
Data after repeated-arm blocks is discarded, wasting sample for no carryover.
Switchback carryover washout: Every block start is washed out · case 03
Data after repeated-arm blocks is discarded, wasting sample for no carryover.
Switchback carryover washout: Every block start is washed out · case 04
Data after repeated-arm blocks is discarded, wasting sample for no carryover.
Switchback carryover washout: Every block start is washed out · case 05
Data after repeated-arm blocks is discarded, wasting sample for no carryover.
Switchback carryover washout: The washout lasts one minute too long · case 01
Events at exactly the washout offset are discarded.
Switchback carryover washout: The washout lasts one minute too long · case 02
Events at exactly the washout offset are discarded.
Switchback carryover washout: The washout lasts one minute too long · case 03
Events at exactly the washout offset are discarded.
Switchback carryover washout: The washout lasts one minute too long · case 04
Events at exactly the washout offset are discarded.
Switchback carryover washout: The washout lasts one minute too long · case 05
Events at exactly the washout offset are discarded.
Switchback carryover washout: Events are assigned to the nearest block · case 01
Events late in a block are credited to the next block's arm.
Switchback carryover washout: Events are assigned to the nearest block · case 02
Events late in a block are credited to the next block's arm.
Switchback carryover washout: Events are assigned to the nearest block · case 03
Events late in a block are credited to the next block's arm.
Switchback carryover washout: Events are assigned to the nearest block · case 04
Events late in a block are credited to the next block's arm.
Switchback carryover washout: Events are assigned to the nearest block · case 05
Events late in a block are credited to the next block's arm.
Switchback carryover washout: The first block inherits the last scheduled arm · case 01
Opening minutes of the experiment are washed out as if a switch had occurred.
Switchback carryover washout: The first block inherits the last scheduled arm · case 02
Opening minutes of the experiment are washed out as if a switch had occurred.
Switchback carryover washout: The first block inherits the last scheduled arm · case 03
Opening minutes of the experiment are washed out as if a switch had occurred.
Switchback carryover washout: The first block inherits the last scheduled arm · case 04
Opening minutes of the experiment are washed out as if a switch had occurred.
Switchback carryover washout: The first block inherits the last scheduled arm · case 05
Opening minutes of the experiment are washed out as if a switch had occurred.
Percent change log-ratio interval: Interval ends stay on the log scale · case 01
Bounds are reported as log points while the estimate is a percent change.
Percent change log-ratio interval: Interval ends stay on the log scale · case 02
Bounds are reported as log points while the estimate is a percent change.
Percent change log-ratio interval: Interval ends stay on the log scale · case 03
Bounds are reported as log points while the estimate is a percent change.
Percent change log-ratio interval: Interval ends stay on the log scale · case 04
Bounds are reported as log points while the estimate is a percent change.
Percent change log-ratio interval: Interval ends stay on the log scale · case 05
Bounds are reported as log points while the estimate is a percent change.
Percent change log-ratio interval: Standard errors are scaled by the control mean only · case 01
Intervals are wrong whenever treatment and control means differ.
Percent change log-ratio interval: Standard errors are scaled by the control mean only · case 02
Intervals are wrong whenever treatment and control means differ.
Percent change log-ratio interval: Standard errors are scaled by the control mean only · case 03
Intervals are wrong whenever treatment and control means differ.
Percent change log-ratio interval: Standard errors are scaled by the control mean only · case 04
Intervals are wrong whenever treatment and control means differ.
Percent change log-ratio interval: Standard errors are scaled by the control mean only · case 05
Intervals are wrong whenever treatment and control means differ.
Percent change log-ratio interval: The point estimate is reported in log points · case 01
The headline change disagrees with its own interval.
Percent change log-ratio interval: The point estimate is reported in log points · case 02
The headline change disagrees with its own interval.
Percent change log-ratio interval: The point estimate is reported in log points · case 03
The headline change disagrees with its own interval.
Percent change log-ratio interval: The point estimate is reported in log points · case 04
The headline change disagrees with its own interval.
Percent change log-ratio interval: The point estimate is reported in log points · case 05
The headline change disagrees with its own interval.
G-Counter gossip: repeated sync adds slots instead of taking the maximum · case 01
Counters grow every time the same state is gossiped again, and replicas never agree.
G-Counter gossip: repeated sync adds slots instead of taking the maximum · case 02
Counters grow every time the same state is gossiped again, and replicas never agree.
G-Counter gossip: repeated sync adds slots instead of taking the maximum · case 03
Counters grow every time the same state is gossiped again, and replicas never agree.
G-Counter gossip: repeated sync adds slots instead of taking the maximum · case 04
Counters grow every time the same state is gossiped again, and replicas never agree.
G-Counter gossip: repeated sync adds slots instead of taking the maximum · case 05
Counters grow every time the same state is gossiped again, and replicas never agree.
G-Counter gossip: all replicas increment one shared slot · case 01
Concurrent increments from different replicas disappear after gossip.
G-Counter gossip: all replicas increment one shared slot · case 02
Concurrent increments from different replicas disappear after gossip.
G-Counter gossip: all replicas increment one shared slot · case 03
Concurrent increments from different replicas disappear after gossip.
G-Counter gossip: all replicas increment one shared slot · case 04
Concurrent increments from different replicas disappear after gossip.
G-Counter gossip: all replicas increment one shared slot · case 05
Concurrent increments from different replicas disappear after gossip.
G-Counter gossip: negative increments are admitted into a grow-only slot · case 01
A negative increment lowers a slot, which later merges silently undo, and the rejection count is wrong.
G-Counter gossip: negative increments are admitted into a grow-only slot · case 02
A negative increment lowers a slot, which later merges silently undo, and the rejection count is wrong.
G-Counter gossip: negative increments are admitted into a grow-only slot · case 03
A negative increment lowers a slot, which later merges silently undo, and the rejection count is wrong.
G-Counter gossip: negative increments are admitted into a grow-only slot · case 04
A negative increment lowers a slot, which later merges silently undo, and the rejection count is wrong.
G-Counter gossip: negative increments are admitted into a grow-only slot · case 05
A negative increment lowers a slot, which later merges silently undo, and the rejection count is wrong.
G-Counter gossip: merge only refreshes slots the receiver already knows · case 01
A replica never learns increments from replicas it has not heard from directly before.
G-Counter gossip: merge only refreshes slots the receiver already knows · case 02
A replica never learns increments from replicas it has not heard from directly before.
G-Counter gossip: merge only refreshes slots the receiver already knows · case 03
A replica never learns increments from replicas it has not heard from directly before.
G-Counter gossip: merge only refreshes slots the receiver already knows · case 04
A replica never learns increments from replicas it has not heard from directly before.
G-Counter gossip: merge only refreshes slots the receiver already knows · case 05
A replica never learns increments from replicas it has not heard from directly before.
G-Counter gossip: the value reads only the local slot · case 01
A replica reports only its own increments even after receiving everyone else's state.
G-Counter gossip: the value reads only the local slot · case 02
A replica reports only its own increments even after receiving everyone else's state.
G-Counter gossip: the value reads only the local slot · case 03
A replica reports only its own increments even after receiving everyone else's state.
G-Counter gossip: the value reads only the local slot · case 04
A replica reports only its own increments even after receiving everyone else's state.
G-Counter gossip: the value reads only the local slot · case 05
A replica reports only its own increments even after receiving everyone else's state.
PN-Counter with local floor: decrements lower the positive slot · case 01
Decrements vanish after a merge with a replica that saw the larger positive slot.
PN-Counter with local floor: decrements lower the positive slot · case 02
Decrements vanish after a merge with a replica that saw the larger positive slot.
PN-Counter with local floor: decrements lower the positive slot · case 03
Decrements vanish after a merge with a replica that saw the larger positive slot.
PN-Counter with local floor: decrements lower the positive slot · case 04
Decrements vanish after a merge with a replica that saw the larger positive slot.
PN-Counter with local floor: decrements lower the positive slot · case 05
Decrements vanish after a merge with a replica that saw the larger positive slot.
PN-Counter with local floor: the negative map merges by minimum · case 01
Decrements never propagate; other replicas keep reporting the undecremented value.
PN-Counter with local floor: the negative map merges by minimum · case 02
Decrements never propagate; other replicas keep reporting the undecremented value.
PN-Counter with local floor: the negative map merges by minimum · case 03
Decrements never propagate; other replicas keep reporting the undecremented value.
PN-Counter with local floor: the negative map merges by minimum · case 04
Decrements never propagate; other replicas keep reporting the undecremented value.
PN-Counter with local floor: the negative map merges by minimum · case 05
Decrements never propagate; other replicas keep reporting the undecremented value.
PN-Counter with local floor: the floor check ignores decrements already applied · case 01
Repeated decrements drive a replica's observed value below zero.
PN-Counter with local floor: the floor check ignores decrements already applied · case 02
Repeated decrements drive a replica's observed value below zero.
PN-Counter with local floor: the floor check ignores decrements already applied · case 03
Repeated decrements drive a replica's observed value below zero.
PN-Counter with local floor: the floor check ignores decrements already applied · case 04
Repeated decrements drive a replica's observed value below zero.
PN-Counter with local floor: the floor check ignores decrements already applied · case 05
Repeated decrements drive a replica's observed value below zero.
PN-Counter with local floor: decrementing to exactly zero is refused · case 01
A decrement that would leave the counter at exactly zero is rejected.
PN-Counter with local floor: decrementing to exactly zero is refused · case 02
A decrement that would leave the counter at exactly zero is rejected.
PN-Counter with local floor: decrementing to exactly zero is refused · case 03
A decrement that would leave the counter at exactly zero is rejected.
PN-Counter with local floor: decrementing to exactly zero is refused · case 04
A decrement that would leave the counter at exactly zero is rejected.
PN-Counter with local floor: decrementing to exactly zero is refused · case 05
A decrement that would leave the counter at exactly zero is rejected.
PN-Counter with local floor: negative slots flow against the sync direction · case 01
Receivers do not see the sender's decrements while the sender absorbs the receiver's.
PN-Counter with local floor: negative slots flow against the sync direction · case 02
Receivers do not see the sender's decrements while the sender absorbs the receiver's.
PN-Counter with local floor: negative slots flow against the sync direction · case 03
Receivers do not see the sender's decrements while the sender absorbs the receiver's.
PN-Counter with local floor: negative slots flow against the sync direction · case 04
Receivers do not see the sender's decrements while the sender absorbs the receiver's.
PN-Counter with local floor: negative slots flow against the sync direction · case 05
Receivers do not see the sender's decrements while the sender absorbs the receiver's.
LWW register with Lamport stamps: receive does not advance the local clock · case 01
A write made after receiving a newer value loses to that older value on other replicas.
LWW register with Lamport stamps: receive does not advance the local clock · case 02
A write made after receiving a newer value loses to that older value on other replicas.
LWW register with Lamport stamps: receive does not advance the local clock · case 03
A write made after receiving a newer value loses to that older value on other replicas.
LWW register with Lamport stamps: receive does not advance the local clock · case 04
A write made after receiving a newer value loses to that older value on other replicas.
LWW register with Lamport stamps: receive does not advance the local clock · case 05
A write made after receiving a newer value loses to that older value on other replicas.
LWW register with Lamport stamps: equal clocks keep whatever each replica holds · case 01
Two concurrent writes with the same clock leave replicas permanently disagreeing.
LWW register with Lamport stamps: equal clocks keep whatever each replica holds · case 02
Two concurrent writes with the same clock leave replicas permanently disagreeing.
LWW register with Lamport stamps: equal clocks keep whatever each replica holds · case 03
Two concurrent writes with the same clock leave replicas permanently disagreeing.
LWW register with Lamport stamps: equal clocks keep whatever each replica holds · case 04
Two concurrent writes with the same clock leave replicas permanently disagreeing.
LWW register with Lamport stamps: equal clocks keep whatever each replica holds · case 05
Two concurrent writes with the same clock leave replicas permanently disagreeing.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗