Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Flag targeting precedence: A disabled flag serves its fallthrough · case 01
Turning the flag off leaves users on the rollout variation instead of the safe off variation.
Flag targeting precedence: A disabled flag serves its fallthrough · case 02
Turning the flag off leaves users on the rollout variation instead of the safe off variation.
Flag targeting precedence: A disabled flag serves its fallthrough · case 03
Turning the flag off leaves users on the rollout variation instead of the safe off variation.
Flag targeting precedence: A disabled flag serves its fallthrough · case 04
Turning the flag off leaves users on the rollout variation instead of the safe off variation.
Flag targeting precedence: A disabled flag serves its fallthrough · case 05
Turning the flag off leaves users on the rollout variation instead of the safe off variation.
Flag targeting precedence: Matched rule index is reported one-based · case 01
Evaluation reasons point at the rule after the one that actually matched.
Flag targeting precedence: Matched rule index is reported one-based · case 02
Evaluation reasons point at the rule after the one that actually matched.
Flag targeting precedence: Matched rule index is reported one-based · case 03
Evaluation reasons point at the rule after the one that actually matched.
Flag targeting precedence: Matched rule index is reported one-based · case 04
Evaluation reasons point at the rule after the one that actually matched.
Flag targeting precedence: Matched rule index is reported one-based · case 05
Evaluation reasons point at the rule after the one that actually matched.
Flag targeting precedence: Prefix clauses ignore letter case · case 01
A rule for emails starting with "ann" also captures "Ann@..." addresses.
Flag targeting precedence: Prefix clauses ignore letter case · case 02
A rule for emails starting with "ann" also captures "Ann@..." addresses.
Flag targeting precedence: Prefix clauses ignore letter case · case 03
A rule for emails starting with "ann" also captures "Ann@..." addresses.
Flag targeting precedence: Prefix clauses ignore letter case · case 04
A rule for emails starting with "ann" also captures "Ann@..." addresses.
Flag targeting precedence: Prefix clauses ignore letter case · case 05
A rule for emails starting with "ann" also captures "Ann@..." addresses.
Prerequisite flag chain: An off prerequisite passes when its off value matches · case 01
A dependent flag turns on although its prerequisite flag is switched off.
Prerequisite flag chain: An off prerequisite passes when its off value matches · case 02
A dependent flag turns on although its prerequisite flag is switched off.
Prerequisite flag chain: An off prerequisite passes when its off value matches · case 03
A dependent flag turns on although its prerequisite flag is switched off.
Prerequisite flag chain: An off prerequisite passes when its off value matches · case 04
A dependent flag turns on although its prerequisite flag is switched off.
Prerequisite flag chain: An off prerequisite passes when its off value matches · case 05
A dependent flag turns on although its prerequisite flag is switched off.
Prerequisite flag chain: Failure reason names the dependent flag · case 01
Debug output says a flag failed its own prerequisite instead of naming the blocking dependency.
Prerequisite flag chain: Failure reason names the dependent flag · case 02
Debug output says a flag failed its own prerequisite instead of naming the blocking dependency.
Prerequisite flag chain: Failure reason names the dependent flag · case 03
Debug output says a flag failed its own prerequisite instead of naming the blocking dependency.
Prerequisite flag chain: Failure reason names the dependent flag · case 04
Debug output says a flag failed its own prerequisite instead of naming the blocking dependency.
Prerequisite flag chain: Failure reason names the dependent flag · case 05
Debug output says a flag failed its own prerequisite instead of naming the blocking dependency.
Prerequisite flag chain: Shared prerequisites are mistaken for cycles · case 01
Two flags that both depend on the same base flag report a prerequisite cycle.
Prerequisite flag chain: Shared prerequisites are mistaken for cycles · case 02
Two flags that both depend on the same base flag report a prerequisite cycle.
Prerequisite flag chain: Shared prerequisites are mistaken for cycles · case 03
Two flags that both depend on the same base flag report a prerequisite cycle.
Prerequisite flag chain: Shared prerequisites are mistaken for cycles · case 04
Two flags that both depend on the same base flag report a prerequisite cycle.
Prerequisite flag chain: Shared prerequisites are mistaken for cycles · case 05
Two flags that both depend on the same base flag report a prerequisite cycle.
Prerequisite flag chain: Missing prerequisites look like ordinary failures · case 01
A typo in a prerequisite key is reported as a routine prerequisite failure.
Prerequisite flag chain: Missing prerequisites look like ordinary failures · case 02
A typo in a prerequisite key is reported as a routine prerequisite failure.
Prerequisite flag chain: Missing prerequisites look like ordinary failures · case 03
A typo in a prerequisite key is reported as a routine prerequisite failure.
Prerequisite flag chain: Missing prerequisites look like ordinary failures · case 04
A typo in a prerequisite key is reported as a routine prerequisite failure.
Prerequisite flag chain: Missing prerequisites look like ordinary failures · case 05
A typo in a prerequisite key is reported as a routine prerequisite failure.
Prerequisite flag chain: Evaluation continues past an unmet prerequisite · case 01
The reported blocking prerequisite is the last failing one, and later broken references surface.
Prerequisite flag chain: Evaluation continues past an unmet prerequisite · case 02
The reported blocking prerequisite is the last failing one, and later broken references surface.
Prerequisite flag chain: Evaluation continues past an unmet prerequisite · case 03
The reported blocking prerequisite is the last failing one, and later broken references surface.
Prerequisite flag chain: Evaluation continues past an unmet prerequisite · case 04
The reported blocking prerequisite is the last failing one, and later broken references surface.
Prerequisite flag chain: Evaluation continues past an unmet prerequisite · case 05
The reported blocking prerequisite is the last failing one, and later broken references surface.
Segment membership precedence: Exclusion overrides explicit inclusion · case 01
A user deliberately added to a segment is dropped because an older exclusion list also names them.
Segment membership precedence: Exclusion overrides explicit inclusion · case 02
A user deliberately added to a segment is dropped because an older exclusion list also names them.
Segment membership precedence: Exclusion overrides explicit inclusion · case 03
A user deliberately added to a segment is dropped because an older exclusion list also names them.
Segment membership precedence: Exclusion overrides explicit inclusion · case 04
A user deliberately added to a segment is dropped because an older exclusion list also names them.
Segment membership precedence: Exclusion overrides explicit inclusion · case 05
A user deliberately added to a segment is dropped because an older exclusion list also names them.
Segment membership precedence: A rule with no clauses captures everyone · case 01
A half-configured segment rule suddenly places every remaining user in the segment.
Segment membership precedence: A rule with no clauses captures everyone · case 02
A half-configured segment rule suddenly places every remaining user in the segment.
Segment membership precedence: A rule with no clauses captures everyone · case 03
A half-configured segment rule suddenly places every remaining user in the segment.
Segment membership precedence: A rule with no clauses captures everyone · case 04
A half-configured segment rule suddenly places every remaining user in the segment.
Segment membership precedence: A rule with no clauses captures everyone · case 05
A half-configured segment rule suddenly places every remaining user in the segment.
Segment membership precedence: Absent attributes match null clause values · case 01
Users without a tier attribute join a segment whose clause lists null among its values.
Segment membership precedence: Absent attributes match null clause values · case 02
Users without a tier attribute join a segment whose clause lists null among its values.
Segment membership precedence: Absent attributes match null clause values · case 03
Users without a tier attribute join a segment whose clause lists null among its values.
Segment membership precedence: Absent attributes match null clause values · case 04
Users without a tier attribute join a segment whose clause lists null among its values.
Segment membership precedence: Absent attributes match null clause values · case 05
Users without a tier attribute join a segment whose clause lists null among its values.
Segment membership precedence: List attributes are compared as a whole · case 01
Users whose groups attribute is a list never match group clauses.
Segment membership precedence: List attributes are compared as a whole · case 02
Users whose groups attribute is a list never match group clauses.
Segment membership precedence: List attributes are compared as a whole · case 03
Users whose groups attribute is a list never match group clauses.
Segment membership precedence: List attributes are compared as a whole · case 04
Users whose groups attribute is a list never match group clauses.
Segment membership precedence: List attributes are compared as a whole · case 05
Users whose groups attribute is a list never match group clauses.
Bucketing attribute selection: Boolean attributes become bucketing identities · case 01
All users with org=True share one bucket named "True".
Bucketing attribute selection: Boolean attributes become bucketing identities · case 02
All users with org=True share one bucket named "True".
Bucketing attribute selection: Boolean attributes become bucketing identities · case 03
All users with org=True share one bucket named "True".
Bucketing attribute selection: Boolean attributes become bucketing identities · case 04
All users with org=True share one bucket named "True".
Bucketing attribute selection: Boolean attributes become bucketing identities · case 05
All users with org=True share one bucket named "True".
Bucketing attribute selection: Integral floats keep a decimal suffix · case 01
An org id sent as 5.0 by one client and 5 by another lands in different buckets.
Bucketing attribute selection: Integral floats keep a decimal suffix · case 02
An org id sent as 5.0 by one client and 5 by another lands in different buckets.
Bucketing attribute selection: Integral floats keep a decimal suffix · case 03
An org id sent as 5.0 by one client and 5 by another lands in different buckets.
Bucketing attribute selection: Integral floats keep a decimal suffix · case 04
An org id sent as 5.0 by one client and 5 by another lands in different buckets.
Bucketing attribute selection: Integral floats keep a decimal suffix · case 05
An org id sent as 5.0 by one client and 5 by another lands in different buckets.
Bucketing attribute selection: Missing bucketing attribute yields no identity · case 01
Users without an org attribute are all treated as unbucketable instead of using their key.
Bucketing attribute selection: Missing bucketing attribute yields no identity · case 02
Users without an org attribute are all treated as unbucketable instead of using their key.
Bucketing attribute selection: Missing bucketing attribute yields no identity · case 03
Users without an org attribute are all treated as unbucketable instead of using their key.
Bucketing attribute selection: Missing bucketing attribute yields no identity · case 04
Users without an org attribute are all treated as unbucketable instead of using their key.
Bucketing attribute selection: Missing bucketing attribute yields no identity · case 05
Users without an org attribute are all treated as unbucketable instead of using their key.
Bucketing attribute selection: Fractional floats are truncated into identities · case 01
Values 2.5 and 2 collapse into the same bucketing identity.
Bucketing attribute selection: Fractional floats are truncated into identities · case 02
Values 2.5 and 2 collapse into the same bucketing identity.
Bucketing attribute selection: Fractional floats are truncated into identities · case 03
Values 2.5 and 2 collapse into the same bucketing identity.
Bucketing attribute selection: Fractional floats are truncated into identities · case 04
Values 2.5 and 2 collapse into the same bucketing identity.
Bucketing attribute selection: Fractional floats are truncated into identities · case 05
Values 2.5 and 2 collapse into the same bucketing identity.
Bucketing attribute selection: Numeric-looking strings are normalised · case 01
Keys "05" and "5" share a bucket, merging distinct customers.
Bucketing attribute selection: Numeric-looking strings are normalised · case 02
Keys "05" and "5" share a bucket, merging distinct customers.
Bucketing attribute selection: Numeric-looking strings are normalised · case 03
Keys "05" and "5" share a bucket, merging distinct customers.
Bucketing attribute selection: Numeric-looking strings are normalised · case 04
Keys "05" and "5" share a bucket, merging distinct customers.
Bucketing attribute selection: Numeric-looking strings are normalised · case 05
Keys "05" and "5" share a bucket, merging distinct customers.
Scheduled rollout ramp: A step is not active at its own timestamp · case 01
The ramp stays at the old percentage for the whole second a new step is scheduled.
Scheduled rollout ramp: A step is not active at its own timestamp · case 02
The ramp stays at the old percentage for the whole second a new step is scheduled.
Scheduled rollout ramp: A step is not active at its own timestamp · case 03
The ramp stays at the old percentage for the whole second a new step is scheduled.
Scheduled rollout ramp: A step is not active at its own timestamp · case 04
The ramp stays at the old percentage for the whole second a new step is scheduled.
Scheduled rollout ramp: A step is not active at its own timestamp · case 05
The ramp stays at the old percentage for the whole second a new step is scheduled.
Scheduled rollout ramp: Schedule entries apply in list order · case 01
Appending an earlier correction step to a schedule overrides later steps.
Scheduled rollout ramp: Schedule entries apply in list order · case 02
Appending an earlier correction step to a schedule overrides later steps.
Scheduled rollout ramp: Schedule entries apply in list order · case 03
Appending an earlier correction step to a schedule overrides later steps.
Scheduled rollout ramp: Schedule entries apply in list order · case 04
Appending an earlier correction step to a schedule overrides later steps.
Scheduled rollout ramp: Schedule entries apply in list order · case 05
Appending an earlier correction step to a schedule overrides later steps.
Scheduled rollout ramp: The earliest-listed step wins a timestamp tie · case 01
A correction appended with the same timestamp as a mistaken step never takes effect.
Scheduled rollout ramp: The earliest-listed step wins a timestamp tie · case 02
A correction appended with the same timestamp as a mistaken step never takes effect.
Scheduled rollout ramp: The earliest-listed step wins a timestamp tie · case 03
A correction appended with the same timestamp as a mistaken step never takes effect.
Scheduled rollout ramp: The earliest-listed step wins a timestamp tie · case 04
A correction appended with the same timestamp as a mistaken step never takes effect.
Scheduled rollout ramp: The earliest-listed step wins a timestamp tie · case 05
A correction appended with the same timestamp as a mistaken step never takes effect.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗