FA-73881 / Feature flag rollout bucketing / Open access
Weighted variant allocation: Variants are walked alphabetically · case 01
Renaming an arm silently moves which bucket ranges it owns.
ROOT CAUSE
The loop iterates sorted(variants) instead of the declared order.
VERIFIED REPAIR
Walk the variants exactly in declared order.
Unsuccessful approach: Sorting by descending weight is still an order the contract never declared.
Case contract
variants is an ordered list of [name, weight percent with at most two decimals]; bucket is an integer in [0, 10000). Walk variants in declared order, skipping non-positive weights, adding round(weight * 100) to a running total, and return the first variant whose total exceeds the bucket. Buckets beyond the total go to the last positively weighted variant; with none, None.
Why this case matters
Variant splits decide which arm of a flag a user receives and must honour declared weights exactly.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(variants, bucket):
acc = 0
last = None
for name, weight in sorted(variants):
if weight <= 0:
continue
last = name
acc += round(weight * 100)
if bucket < acc:
return name
return last
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at first cumulative edge goes to second arm',
[[['control', 50], ['treatment', 50]], 5000],
'treatment'),
('bucket just below first edge stays in first arm',
[[['control', 50], ['treatment', 50]], 4999],
'control'),
('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 1',
[[['green', 1.15], ['holdout', 0], ['alpha', 1.15], ['blue', 50]], 1135],
'blue')],
[('bucket just below first edge stays in first arm',
[[['control', 50], ['treatment', 50]], 4999],
'control'),
('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 13',
[[['alpha', -5], ['green', 12.5], ['treatment', 50], ['blue', 33.34]], 7194],
'blue'),
('weight table sample 28', [[['alpha', 1.15], ['holdout', 4.35]], 1437], 'holdout')],
[('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('weight table sample 11', [[['holdout', 12.5], ['blue', 12.5], ['alpha', 0.57]], 2500], 'alpha'),
('weight table sample 31', [[['green', 1.15], ['blue', 33.33], ['treatment', 4.35]], 3447], 'blue'),
('weight table sample 46',
[[['control', 0], ['alpha', 4.35], ['green', 33.33], ['holdout', 33.33]], 6769],
'holdout')],
[('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 16', [[['green', 0.57], ['alpha', 12.5], ['control', 25]], 1307], 'control'),
('weight table sample 17', [[['treatment', 1.15], ['control', 0.29], ['blue', 0.29]], 2597], 'blue'),
('weight table sample 53',
[[['treatment', 33.33], ['holdout', 33.34], ['control', 4.35]], 0],
'treatment')],
[('bucket at first cumulative edge goes to second arm',
[[['control', 50], ['treatment', 50]], 5000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 3', [[['treatment', 12.5], ['control', 10], ['holdout', 0.57]], 6309], 'holdout'),
('weight table sample 21', [[['green', 33.33], ['treatment', 0]], 0], 'green'),
('weight table sample 31', [[['green', 1.15], ['blue', 33.33], ['treatment', 4.35]], 3447], 'blue')]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| bucket at first cumulative edge goes to second arm | treatment | treatment | Passed |
| bucket just below first edge stays in first arm | control | control | Passed |
| zero weight arm is never selected | treatment | treatment | Passed |
| trailing zero weight arm does not absorb remainder | treatment | treatment | Passed |
| declared order beats alphabetical order | alpha | zeta | Failed |
| fractional weight 0.29 covers 29 buckets | rest | tiny | Failed |
| three way split middle arm | b | b | Passed |
| weight table sample 1 | blue | blue | Passed |
SHA-256 / 499eafd4601363dbb05aa753fc3c96b3b4350911f0216a3d1c90861565a79ab9
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(variants, bucket):
acc = 0
last = None
for name, weight in sorted(variants, key=lambda v: -v[1]):
if weight <= 0:
continue
last = name
acc += round(weight * 100)
if bucket < acc:
return name
return last
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at first cumulative edge goes to second arm',
[[['control', 50], ['treatment', 50]], 5000],
'treatment'),
('bucket just below first edge stays in first arm',
[[['control', 50], ['treatment', 50]], 4999],
'control'),
('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 1',
[[['green', 1.15], ['holdout', 0], ['alpha', 1.15], ['blue', 50]], 1135],
'blue')],
[('bucket just below first edge stays in first arm',
[[['control', 50], ['treatment', 50]], 4999],
'control'),
('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 13',
[[['alpha', -5], ['green', 12.5], ['treatment', 50], ['blue', 33.34]], 7194],
'blue'),
('weight table sample 28', [[['alpha', 1.15], ['holdout', 4.35]], 1437], 'holdout')],
[('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('weight table sample 11', [[['holdout', 12.5], ['blue', 12.5], ['alpha', 0.57]], 2500], 'alpha'),
('weight table sample 31', [[['green', 1.15], ['blue', 33.33], ['treatment', 4.35]], 3447], 'blue'),
('weight table sample 46',
[[['control', 0], ['alpha', 4.35], ['green', 33.33], ['holdout', 33.33]], 6769],
'holdout')],
[('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 16', [[['green', 0.57], ['alpha', 12.5], ['control', 25]], 1307], 'control'),
('weight table sample 17', [[['treatment', 1.15], ['control', 0.29], ['blue', 0.29]], 2597], 'blue'),
('weight table sample 53',
[[['treatment', 33.33], ['holdout', 33.34], ['control', 4.35]], 0],
'treatment')],
[('bucket at first cumulative edge goes to second arm',
[[['control', 50], ['treatment', 50]], 5000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 3', [[['treatment', 12.5], ['control', 10], ['holdout', 0.57]], 6309], 'holdout'),
('weight table sample 21', [[['green', 33.33], ['treatment', 0]], 0], 'green'),
('weight table sample 31', [[['green', 1.15], ['blue', 33.33], ['treatment', 4.35]], 3447], 'blue')]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| bucket at first cumulative edge goes to second arm | treatment | treatment | Passed |
| bucket just below first edge stays in first arm | control | control | Passed |
| zero weight arm is never selected | treatment | treatment | Passed |
| trailing zero weight arm does not absorb remainder | treatment | treatment | Passed |
| declared order beats alphabetical order | alpha | zeta | Failed |
| fractional weight 0.29 covers 29 buckets | rest | tiny | Failed |
| three way split middle arm | a | b | Failed |
| weight table sample 1 | blue | blue | Passed |
SHA-256 / 9ed3765c0425cb52cbbd901cf6a788ba6e26a5c40ce20da9cbd92f374ecb9ce7
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(variants, bucket):
acc = 0
last = None
for name, weight in variants:
if weight <= 0:
continue
last = name
acc += round(weight * 100)
if bucket < acc:
return name
return last
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at first cumulative edge goes to second arm',
[[['control', 50], ['treatment', 50]], 5000],
'treatment'),
('bucket just below first edge stays in first arm',
[[['control', 50], ['treatment', 50]], 4999],
'control'),
('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 1',
[[['green', 1.15], ['holdout', 0], ['alpha', 1.15], ['blue', 50]], 1135],
'blue')],
[('bucket just below first edge stays in first arm',
[[['control', 50], ['treatment', 50]], 4999],
'control'),
('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 13',
[[['alpha', -5], ['green', 12.5], ['treatment', 50], ['blue', 33.34]], 7194],
'blue'),
('weight table sample 28', [[['alpha', 1.15], ['holdout', 4.35]], 1437], 'holdout')],
[('zero weight arm is never selected', [[['control', 0], ['treatment', 100]], 0], 'treatment'),
('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('weight table sample 11', [[['holdout', 12.5], ['blue', 12.5], ['alpha', 0.57]], 2500], 'alpha'),
('weight table sample 31', [[['green', 1.15], ['blue', 33.33], ['treatment', 4.35]], 3447], 'blue'),
('weight table sample 46',
[[['control', 0], ['alpha', 4.35], ['green', 33.33], ['holdout', 33.33]], 6769],
'holdout')],
[('trailing zero weight arm does not absorb remainder',
[[['control', 40], ['treatment', 40], ['off', 0]], 9000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 16', [[['green', 0.57], ['alpha', 12.5], ['control', 25]], 1307], 'control'),
('weight table sample 17', [[['treatment', 1.15], ['control', 0.29], ['blue', 0.29]], 2597], 'blue'),
('weight table sample 53',
[[['treatment', 33.33], ['holdout', 33.34], ['control', 4.35]], 0],
'treatment')],
[('bucket at first cumulative edge goes to second arm',
[[['control', 50], ['treatment', 50]], 5000],
'treatment'),
('declared order beats alphabetical order', [[['zeta', 10], ['alpha', 90]], 500], 'zeta'),
('fractional weight 0.29 covers 29 buckets', [[['tiny', 0.29], ['rest', 99.71]], 28], 'tiny'),
('no positive weights yields None', [[['a', 0], ['b', 0]], 10], None),
('three way split middle arm', [[['a', 33.33], ['b', 33.33], ['c', 33.34]], 5000], 'b'),
('weight table sample 3', [[['treatment', 12.5], ['control', 10], ['holdout', 0.57]], 6309], 'holdout'),
('weight table sample 21', [[['green', 33.33], ['treatment', 0]], 0], 'green'),
('weight table sample 31', [[['green', 1.15], ['blue', 33.33], ['treatment', 4.35]], 3447], 'blue')]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| bucket at first cumulative edge goes to second arm | treatment | treatment | Passed |
| bucket just below first edge stays in first arm | control | control | Passed |
| zero weight arm is never selected | treatment | treatment | Passed |
| trailing zero weight arm does not absorb remainder | treatment | treatment | Passed |
| declared order beats alphabetical order | zeta | zeta | Passed |
| fractional weight 0.29 covers 29 buckets | tiny | tiny | Passed |
| three way split middle arm | b | b | Passed |
| weight table sample 1 | blue | blue | Passed |
SHA-256 / 0a7b39cf2628b86ed5bec5e1c6e2ab4d1ed1a2362c23f2b78e02c76ee4f73e87
Verification & scope
A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:48:51.619243+00:00.
Case digest / 134fe5a5b08fdde3aed4fa30c2243f697749ca6727708ab63ee8187005175b9e