FA-74321 / Feature flag rollout bucketing / Open access
Seeded weighted rollout: Buckets past the weights get no variation · case 01
When weights sum below 100000, some users evaluate to no variation at all.
ROOT CAUSE
After the loop the function returns None instead of the last index.
VERIFIED REPAIR
Assign leftover buckets to the last variation.
Unsuccessful approach: Assigning leftovers to the first variation over-weights it.
Case contract
Hash text is (str(seed) if seed is not None else flag_key + "." + salt) + "." + ident, plus "." + secondary when secondary is not None (an empty string still counts). bucket = int(first 15 hex digits of sha1(text), 16) / 0xFFFFFFFFFFFFFFF. weights are in 1/100000 units; return the first index whose cumulative weight / 100000 exceeds the bucket, else the last index, as [index, round(bucket, 6)].
Why this case matters
Seeds let several flags share one bucketing so a cohort sees consistent variations.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(flag_key, salt, ident, secondary, seed, weights):
prefix = str(seed) if seed is not None else flag_key + '.' + salt
text = prefix + '.' + ident
if secondary is not None:
text += '.' + secondary
bucket = int(hashlib.sha1(text.encode()).hexdigest()[:15], 16) / float(0xFFFFFFFFFFFFFFF)
acc = 0
for index, w in enumerate(weights):
acc += w
if bucket < acc / 100000.0:
return [index, round(bucket, 6)]
return [None, round(bucket, 6)]
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('rollout sample 1', ['flag-b', 'salt2', 'id120', '', 0, [0, 60000, 40000]], [2, 0.88242]),
('rollout sample 2', ['flag-b', 's', 'id212', 'x', 61, [25000, 25000, 25000]], [1, 0.37423]),
('rollout sample 3', ['flag-b', 'salt2', 'id106', None, 7, [50000, 50000]], [0, 0.322784])],
[('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 5', ['flag-b', 's', 'id217', 'x', 61, [30000, 30000]], [1, 0.916419]),
('rollout sample 6', ['flag-b', 'salt2', 'id765', None, 7, [10000, 20000, 70000]], [2, 0.464498]),
('rollout sample 7', ['flag-a', 's', 'id556', None, None, [30000, 30000]], [1, 0.522407])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 10', ['flag-a', 'salt2', 'id392', None, 0, [30000, 30000]], [1, 0.721652]),
('rollout sample 11', ['flag-a', 'salt2', 'id773', '', 7, [0, 60000, 40000]], [1, 0.477906]),
('rollout sample 12', ['flag-b', 'salt2', 'id724', '', None, [0, 60000, 40000]], [1, 0.353275])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 16', ['flag-b', 'salt2', 'id229', 'eu', 61, [10000, 20000, 70000]], [2, 0.315757]),
('rollout sample 17', ['flag-b', 's', 'id105', None, None, [0, 60000, 40000]], [2, 0.791823]),
('rollout sample 26', ['flag-b', 's', 'id731', 'eu', None, [25000, 25000, 25000]], [2, 0.759497])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 21', ['flag-a', 'salt2', 'id352', '', None, [99999]], [0, 0.94531]),
('rollout sample 22', ['flag-a', 'salt2', 'id349', '', 0, [50000, 50000]], [0, 0.298678]),
('rollout sample 28', ['flag-a', 'salt2', 'id79', None, None, [30000, 30000]], [1, 0.719866])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| seed zero is a real seed | [0, 0.331393] | [0, 0.331393] | Passed |
| seed replaces flag key and salt | [1, 0.855827] | [1, 0.855827] | Passed |
| empty secondary is still appended | [1, 0.891663] | [1, 0.891663] | Passed |
| non-empty secondary | [1, 0.975219] | [1, 0.975219] | Passed |
| weights short of total go to last index | [None, 0.233147] | [1, 0.233147] | Failed |
| rollout sample 1 | [2, 0.88242] | [2, 0.88242] | Passed |
| rollout sample 2 | [1, 0.37423] | [1, 0.37423] | Passed |
| rollout sample 3 | [0, 0.322784] | [0, 0.322784] | Passed |
SHA-256 / 34c30c254e9605705bf6a909bcb364f7fbfd8e9799c87d9dcba83685222b2e81
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(flag_key, salt, ident, secondary, seed, weights):
prefix = str(seed) if seed is not None else flag_key + '.' + salt
text = prefix + '.' + ident
if secondary is not None:
text += '.' + secondary
bucket = int(hashlib.sha1(text.encode()).hexdigest()[:15], 16) / float(0xFFFFFFFFFFFFFFF)
acc = 0
for index, w in enumerate(weights):
acc += w
if bucket < acc / 100000.0:
return [index, round(bucket, 6)]
return [0, round(bucket, 6)]
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('rollout sample 1', ['flag-b', 'salt2', 'id120', '', 0, [0, 60000, 40000]], [2, 0.88242]),
('rollout sample 2', ['flag-b', 's', 'id212', 'x', 61, [25000, 25000, 25000]], [1, 0.37423]),
('rollout sample 3', ['flag-b', 'salt2', 'id106', None, 7, [50000, 50000]], [0, 0.322784])],
[('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 5', ['flag-b', 's', 'id217', 'x', 61, [30000, 30000]], [1, 0.916419]),
('rollout sample 6', ['flag-b', 'salt2', 'id765', None, 7, [10000, 20000, 70000]], [2, 0.464498]),
('rollout sample 7', ['flag-a', 's', 'id556', None, None, [30000, 30000]], [1, 0.522407])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 10', ['flag-a', 'salt2', 'id392', None, 0, [30000, 30000]], [1, 0.721652]),
('rollout sample 11', ['flag-a', 'salt2', 'id773', '', 7, [0, 60000, 40000]], [1, 0.477906]),
('rollout sample 12', ['flag-b', 'salt2', 'id724', '', None, [0, 60000, 40000]], [1, 0.353275])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 16', ['flag-b', 'salt2', 'id229', 'eu', 61, [10000, 20000, 70000]], [2, 0.315757]),
('rollout sample 17', ['flag-b', 's', 'id105', None, None, [0, 60000, 40000]], [2, 0.791823]),
('rollout sample 26', ['flag-b', 's', 'id731', 'eu', None, [25000, 25000, 25000]], [2, 0.759497])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 21', ['flag-a', 'salt2', 'id352', '', None, [99999]], [0, 0.94531]),
('rollout sample 22', ['flag-a', 'salt2', 'id349', '', 0, [50000, 50000]], [0, 0.298678]),
('rollout sample 28', ['flag-a', 'salt2', 'id79', None, None, [30000, 30000]], [1, 0.719866])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| seed zero is a real seed | [0, 0.331393] | [0, 0.331393] | Passed |
| seed replaces flag key and salt | [1, 0.855827] | [1, 0.855827] | Passed |
| empty secondary is still appended | [1, 0.891663] | [1, 0.891663] | Passed |
| non-empty secondary | [1, 0.975219] | [1, 0.975219] | Passed |
| weights short of total go to last index | [0, 0.233147] | [1, 0.233147] | Failed |
| rollout sample 1 | [2, 0.88242] | [2, 0.88242] | Passed |
| rollout sample 2 | [1, 0.37423] | [1, 0.37423] | Passed |
| rollout sample 3 | [0, 0.322784] | [0, 0.322784] | Passed |
SHA-256 / 072a6e6b505e5e140b93eaecf4a940a1021fe64efd6fc6bac000cd37cb72273b
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(flag_key, salt, ident, secondary, seed, weights):
prefix = str(seed) if seed is not None else flag_key + '.' + salt
text = prefix + '.' + ident
if secondary is not None:
text += '.' + secondary
bucket = int(hashlib.sha1(text.encode()).hexdigest()[:15], 16) / float(0xFFFFFFFFFFFFFFF)
acc = 0
for index, w in enumerate(weights):
acc += w
if bucket < acc / 100000.0:
return [index, round(bucket, 6)]
return [len(weights) - 1, round(bucket, 6)]
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('rollout sample 1', ['flag-b', 'salt2', 'id120', '', 0, [0, 60000, 40000]], [2, 0.88242]),
('rollout sample 2', ['flag-b', 's', 'id212', 'x', 61, [25000, 25000, 25000]], [1, 0.37423]),
('rollout sample 3', ['flag-b', 'salt2', 'id106', None, 7, [50000, 50000]], [0, 0.322784])],
[('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 5', ['flag-b', 's', 'id217', 'x', 61, [30000, 30000]], [1, 0.916419]),
('rollout sample 6', ['flag-b', 'salt2', 'id765', None, 7, [10000, 20000, 70000]], [2, 0.464498]),
('rollout sample 7', ['flag-a', 's', 'id556', None, None, [30000, 30000]], [1, 0.522407])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 10', ['flag-a', 'salt2', 'id392', None, 0, [30000, 30000]], [1, 0.721652]),
('rollout sample 11', ['flag-a', 'salt2', 'id773', '', 7, [0, 60000, 40000]], [1, 0.477906]),
('rollout sample 12', ['flag-b', 'salt2', 'id724', '', None, [0, 60000, 40000]], [1, 0.353275])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('non-empty secondary', ['flag-a', 's', 'id4', 'eu', None, [10000, 90000]], [1, 0.975219]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 16', ['flag-b', 'salt2', 'id229', 'eu', 61, [10000, 20000, 70000]], [2, 0.315757]),
('rollout sample 17', ['flag-b', 's', 'id105', None, None, [0, 60000, 40000]], [2, 0.791823]),
('rollout sample 26', ['flag-b', 's', 'id731', 'eu', None, [25000, 25000, 25000]], [2, 0.759497])],
[('seed zero is a real seed', ['flag-a', 's', 'id1', None, 0, [50000, 50000]], [0, 0.331393]),
('seed replaces flag key and salt', ['flag-b', 's', 'id2', None, 7, [30000, 70000]], [1, 0.855827]),
('empty secondary is still appended', ['flag-a', 's', 'id3', '', None, [50000, 50000]], [1, 0.891663]),
('weights short of total go to last index',
['flag-b', 'salt2', 'id5', None, None, [10000, 10000]],
[1, 0.233147]),
('zero-weight first variation', ['flag-b', 's', 'id6', None, None, [0, 100000]], [1, 0.255084]),
('rollout sample 21', ['flag-a', 'salt2', 'id352', '', None, [99999]], [0, 0.94531]),
('rollout sample 22', ['flag-a', 'salt2', 'id349', '', 0, [50000, 50000]], [0, 0.298678]),
('rollout sample 28', ['flag-a', 'salt2', 'id79', None, None, [30000, 30000]], [1, 0.719866])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| seed zero is a real seed | [0, 0.331393] | [0, 0.331393] | Passed |
| seed replaces flag key and salt | [1, 0.855827] | [1, 0.855827] | Passed |
| empty secondary is still appended | [1, 0.891663] | [1, 0.891663] | Passed |
| non-empty secondary | [1, 0.975219] | [1, 0.975219] | Passed |
| weights short of total go to last index | [1, 0.233147] | [1, 0.233147] | Passed |
| rollout sample 1 | [2, 0.88242] | [2, 0.88242] | Passed |
| rollout sample 2 | [1, 0.37423] | [1, 0.37423] | Passed |
| rollout sample 3 | [0, 0.322784] | [0, 0.322784] | Passed |
SHA-256 / 6db57c2aa0ed1c611badd848459770b9cf044910ad81fec8859576c75e8354d4
Verification & scope
A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:48:55.697474+00:00.
Case digest / 0e3d5d5b176b41787b1b4e147f9a8a780a235b1ec5b893d26644e721d5d82756