FA-73861 / Feature flag rollout bucketing / Open access
Percentage rollout gate: Buckets are reduced into a thousand slots · case 01
Bucket values never exceed 999, so a 10 percent rollout enables every user.
ROOT CAUSE
The digest integer is reduced modulo 1000 while limits are expressed in basis points.
VERIFIED REPAIR
Reduce the digest integer modulo 10000 so buckets share the basis-point scale.
Unsuccessful approach: Keeping modulo 10000 but parsing four hex digits yields a biased 16-bit bucket.
Case contract
bucket = int(first 8 hex digits of sha1(salt + "." + flag + "." + user), 16) % 10000; the rollout percentage pct has at most two decimals and becomes a basis-point limit round(pct * 100); the user is enabled iff bucket < limit. Return [bucket, enabled].
Why this case matters
Percentage gates must be reproducible across services so the same user sees the same state everywhere.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(flag, user, salt, pct):
limit = round(pct * 100)
digest = hashlib.sha1((salt + '.' + flag + '.' + user).encode()).hexdigest()
bucket = int(digest[:8], 16) % 1000
return [bucket, bucket < limit]
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket exactly at limit is excluded (alice)', ['new-checkout', 'alice', 'prod', 66.65], [6665, False]),
('bucket one below limit is included (alice)', ['new-checkout', 'alice', 'prod', 66.66], [6665, True]),
('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('half rollout control', ['search-v2', 'grace', 'ramp-7', 50], [3372, True]),
('hashed user sample 1', ['search-v2', 'u55140', 's1', 100], [5052, True]),
('hashed user sample 2', ['new-checkout', 'u28082', 's1', 50], [928, True])],
[('bucket one below limit is included (alice)', ['new-checkout', 'alice', 'prod', 66.66], [6665, True]),
('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('half rollout control', ['search-v2', 'grace', 'ramp-7', 50], [3372, True]),
('hashed user sample 6', ['new-checkout', 'u58886', 'ramp-7', 50.31], [3873, True])],
[('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('hashed user sample 4', ['fast-pay', 'u43705', 'ramp-7', 32.8], [187, True]),
('hashed user sample 6', ['new-checkout', 'u58886', 'ramp-7', 50.31], [3873, True]),
('hashed user sample 11', ['search-v2', 'u91686', 'prod', 100], [5233, True])],
[('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('hashed user sample 11', ['search-v2', 'u91686', 'prod', 100], [5233, True]),
('hashed user sample 13', ['new-checkout', 'u70823', 'ramp-7', 31.8], [3179, True]),
('hashed user sample 16', ['new-checkout', 'u25854', 'prod', 0.57], [5842, False])],
[('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('full rollout enables everyone', ['dark-mode', 'erin', 's1', 100], [9606, True]),
('hashed user sample 18', ['dark-mode', 'u52498', 'prod', 39.33], [2698, True]),
('hashed user sample 20', ['dark-mode', 'u36438', 'prod', 34.74], [3474, False]),
('hashed user sample 21', ['new-checkout', 'u7954', 'prod', 59.72], [196, True])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| bucket exactly at limit is excluded (alice) | [665, True] | [6665, False] | Failed |
| bucket one below limit is included (alice) | [665, True] | [6665, True] | Failed |
| bucket exactly at limit is excluded (bob) | [755, True] | [2755, False] | Failed |
| bucket one below limit is included (bob) | [755, True] | [2755, True] | Failed |
| bucket exactly at limit is excluded (carol) | [745, True] | [5745, False] | Failed |
| half rollout control | [372, True] | [3372, True] | Failed |
| hashed user sample 1 | [52, True] | [5052, True] | Failed |
| hashed user sample 2 | [928, True] | [928, True] | Passed |
SHA-256 / 64dcfc4828ae23eba0a6d5b5234d1d736f3871a1e9cfd0031542a84e4770de7f
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(flag, user, salt, pct):
limit = round(pct * 100)
digest = hashlib.sha1((salt + '.' + flag + '.' + user).encode()).hexdigest()
bucket = int(digest[:4], 16) % 10000
return [bucket, bucket < limit]
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket exactly at limit is excluded (alice)', ['new-checkout', 'alice', 'prod', 66.65], [6665, False]),
('bucket one below limit is included (alice)', ['new-checkout', 'alice', 'prod', 66.66], [6665, True]),
('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('half rollout control', ['search-v2', 'grace', 'ramp-7', 50], [3372, True]),
('hashed user sample 1', ['search-v2', 'u55140', 's1', 100], [5052, True]),
('hashed user sample 2', ['new-checkout', 'u28082', 's1', 50], [928, True])],
[('bucket one below limit is included (alice)', ['new-checkout', 'alice', 'prod', 66.66], [6665, True]),
('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('half rollout control', ['search-v2', 'grace', 'ramp-7', 50], [3372, True]),
('hashed user sample 6', ['new-checkout', 'u58886', 'ramp-7', 50.31], [3873, True])],
[('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('hashed user sample 4', ['fast-pay', 'u43705', 'ramp-7', 32.8], [187, True]),
('hashed user sample 6', ['new-checkout', 'u58886', 'ramp-7', 50.31], [3873, True]),
('hashed user sample 11', ['search-v2', 'u91686', 'prod', 100], [5233, True])],
[('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('hashed user sample 11', ['search-v2', 'u91686', 'prod', 100], [5233, True]),
('hashed user sample 13', ['new-checkout', 'u70823', 'ramp-7', 31.8], [3179, True]),
('hashed user sample 16', ['new-checkout', 'u25854', 'prod', 0.57], [5842, False])],
[('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('full rollout enables everyone', ['dark-mode', 'erin', 's1', 100], [9606, True]),
('hashed user sample 18', ['dark-mode', 'u52498', 'prod', 39.33], [2698, True]),
('hashed user sample 20', ['dark-mode', 'u36438', 'prod', 34.74], [3474, False]),
('hashed user sample 21', ['new-checkout', 'u7954', 'prod', 59.72], [196, True])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| bucket exactly at limit is excluded (alice) | [2909, True] | [6665, False] | Failed |
| bucket one below limit is included (alice) | [2909, True] | [6665, True] | Failed |
| bucket exactly at limit is excluded (bob) | [4747, False] | [2755, False] | Failed |
| bucket one below limit is included (bob) | [4747, False] | [2755, True] | Failed |
| bucket exactly at limit is excluded (carol) | [3235, True] | [5745, False] | Failed |
| half rollout control | [9696, False] | [3372, True] | Failed |
| hashed user sample 1 | [2285, True] | [5052, True] | Failed |
| hashed user sample 2 | [7330, False] | [928, True] | Failed |
SHA-256 / 93a9e9273a81dac7729924cc2ead01b29baa4ee2c30d3daa66ea55f1f933bd87
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(flag, user, salt, pct):
limit = round(pct * 100)
digest = hashlib.sha1((salt + '.' + flag + '.' + user).encode()).hexdigest()
bucket = int(digest[:8], 16) % 10000
return [bucket, bucket < limit]
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket exactly at limit is excluded (alice)', ['new-checkout', 'alice', 'prod', 66.65], [6665, False]),
('bucket one below limit is included (alice)', ['new-checkout', 'alice', 'prod', 66.66], [6665, True]),
('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('half rollout control', ['search-v2', 'grace', 'ramp-7', 50], [3372, True]),
('hashed user sample 1', ['search-v2', 'u55140', 's1', 100], [5052, True]),
('hashed user sample 2', ['new-checkout', 'u28082', 's1', 50], [928, True])],
[('bucket one below limit is included (alice)', ['new-checkout', 'alice', 'prod', 66.66], [6665, True]),
('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('half rollout control', ['search-v2', 'grace', 'ramp-7', 50], [3372, True]),
('hashed user sample 6', ['new-checkout', 'u58886', 'ramp-7', 50.31], [3873, True])],
[('bucket exactly at limit is excluded (bob)', ['dark-mode', 'bob', 's1', 27.55], [2755, False]),
('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('hashed user sample 4', ['fast-pay', 'u43705', 'ramp-7', 32.8], [187, True]),
('hashed user sample 6', ['new-checkout', 'u58886', 'ramp-7', 50.31], [3873, True]),
('hashed user sample 11', ['search-v2', 'u91686', 'prod', 100], [5233, True])],
[('bucket one below limit is included (bob)', ['dark-mode', 'bob', 's1', 27.56], [2755, True]),
('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('hashed user sample 11', ['search-v2', 'u91686', 'prod', 100], [5233, True]),
('hashed user sample 13', ['new-checkout', 'u70823', 'ramp-7', 31.8], [3179, True]),
('hashed user sample 16', ['new-checkout', 'u25854', 'prod', 0.57], [5842, False])],
[('bucket exactly at limit is excluded (carol)', ['search-v2', 'carol', 'prod', 57.45], [5745, False]),
('bucket one below limit is included (carol)', ['search-v2', 'carol', 'prod', 57.46], [5745, True]),
('float truncation regression at 65.82 percent', ['fast-pay', 'member13', 'prod', 65.82], [6581, True]),
('zero percent disables everyone', ['dark-mode', 'dave', 's1', 0], [4167, False]),
('full rollout enables everyone', ['dark-mode', 'erin', 's1', 100], [9606, True]),
('hashed user sample 18', ['dark-mode', 'u52498', 'prod', 39.33], [2698, True]),
('hashed user sample 20', ['dark-mode', 'u36438', 'prod', 34.74], [3474, False]),
('hashed user sample 21', ['new-checkout', 'u7954', 'prod', 59.72], [196, True])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| bucket exactly at limit is excluded (alice) | [6665, False] | [6665, False] | Passed |
| bucket one below limit is included (alice) | [6665, True] | [6665, True] | Passed |
| bucket exactly at limit is excluded (bob) | [2755, False] | [2755, False] | Passed |
| bucket one below limit is included (bob) | [2755, True] | [2755, True] | Passed |
| bucket exactly at limit is excluded (carol) | [5745, False] | [5745, False] | Passed |
| half rollout control | [3372, True] | [3372, True] | Passed |
| hashed user sample 1 | [5052, True] | [5052, True] | Passed |
| hashed user sample 2 | [928, True] | [928, True] | Passed |
SHA-256 / 8755fd57c77c301e7dd51d1230b0a8a9245fa4756b3e541f8327976cc5487ca3
Verification & scope
A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:48:51.342691+00:00.
Case digest / c35a7e2d4143276ac7ab13c165bd614ea997284a10a80c1b5c5b4ef7cf3373c1