FA-74076 / Feature flag rollout bucketing / Open access

Mutually exclusive layer allocation: The last overlapping range wins · case 01

Adding a new experiment at the end of a layer silently steals users from earlier ones.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The loop keeps overwriting its result instead of stopping at the first containing range.

VERIFIED REPAIR

Return at the first listed range that contains the bucket.

Unsuccessful approach: Choosing the narrowest containing range invents a specificity rule the contract never states.

Case contract

bucket = int(md5(layer_salt + ":" + user) hex, 16) % 1000. experiments is an ordered list of {name, start, end, paused?} with start/end in percent (one decimal) converted to permille by round(x * 10); ranges are half-open [start, end). The first listed range containing the bucket decides: a paused experiment reserves its range (user gets None). Return [bucket, name or None].

Why this case matters

Layers keep concurrent experiments from overlapping on the same users.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(layer_salt, user, experiments):
    bucket = int(hashlib.md5((layer_salt + ':' + user).encode()).hexdigest(), 16) % 1000
    found = [bucket, None]
    for exp in experiments:
        lo = round(exp['start'] * 10)
        hi = round(exp['end'] * 10)
        if lo <= bucket < hi:
            found = [bucket, None] if exp.get('paused') else [bucket, exp['name']]
    return found
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('layer sample 1',
   ['layer-a',
    'u295',
    [{'name': 'exp0', 'start': 50, 'end': 57.7, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 57.7, 'paused': False}]],
   [576, 'exp0']),
  ('layer sample 2',
   ['layer-b',
    'u4435',
    [{'name': 'exp0', 'start': 0, 'end': 75.0, 'paused': True},
     {'name': 'exp1', 'start': 0, 'end': 74.9, 'paused': False}]],
   [750, None]),
  ('layer sample 3',
   ['layer-a',
    'u3828',
    [{'name': 'exp0', 'start': 1.72, 'end': 17.3, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 17.3, 'paused': True}]],
   [172, 'exp0'])],
 [('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('layer sample 6',
   ['layer-a', 'u2869', [{'name': 'exp0', 'start': 28.1, 'end': 50, 'paused': False}]],
   [282, 'exp0']),
  ('layer sample 18',
   ['layer-b',
    'u4000',
    [{'name': 'exp0', 'start': 0, 'end': 2.83, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 100, 'paused': True},
     {'name': 'exp2', 'start': 28.2, 'end': 28.4, 'paused': False}]],
   [283, None]),
  ('layer sample 48',
   ['layer-a',
    'u431',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 20.0, 'end': 20.2, 'paused': True}]],
   [201, 'exp0'])],
 [('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 11',
   ['layer-b',
    'u3995',
    [{'name': 'exp0', 'start': 50, 'end': 84.5, 'paused': True},
     {'name': 'exp1', 'start': 8.45, 'end': 84.5, 'paused': True}]],
   [845, None]),
  ('layer sample 39',
   ['layer-b',
    'u2441',
    [{'name': 'exp0', 'start': 0, 'end': 75.8, 'paused': False},
     {'name': 'exp1', 'start': 43.8, 'end': 100, 'paused': False}]],
   [439, 'exp0']),
  ('layer sample 47',
   ['layer-b',
    'u2172',
    [{'name': 'exp0', 'start': 89.1, 'end': 89.3, 'paused': False},
     {'name': 'exp1', 'start': 89.1, 'end': 100, 'paused': False}]],
   [892, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 4',
   ['layer-a',
    'u309',
    [{'name': 'exp0', 'start': 6.0, 'end': 50, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 5.9, 'paused': False},
     {'name': 'exp2', 'start': 6.0, 'end': 50, 'paused': False}]],
   [60, 'exp0']),
  ('layer sample 16',
   ['layer-a',
    'u4745',
    [{'name': 'exp0', 'start': 81.9, 'end': 100, 'paused': True},
     {'name': 'exp1', 'start': 8.19, 'end': 82.0, 'paused': True},
     {'name': 'exp2', 'start': 50, 'end': 82.0, 'paused': False}]],
   [819, None]),
  ('layer sample 28',
   ['layer-a',
    'u2535',
    [{'name': 'exp0', 'start': 5.59, 'end': 56.0, 'paused': False},
     {'name': 'exp1', 'start': 5.59, 'end': 100, 'paused': False},
     {'name': 'exp2', 'start': 52.4, 'end': 100, 'paused': False}]],
   [559, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 14',
   ['layer-b',
    'u1256',
    [{'name': 'exp0', 'start': 50, 'end': 89.8, 'paused': False},
     {'name': 'exp1', 'start': 89.7, 'end': 89.8, 'paused': False}]],
   [897, 'exp0']),
  ('layer sample 21',
   ['layer-a',
    'u505',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 54.1, 'end': 100, 'paused': False}]],
   [540, 'exp0']),
  ('layer sample 38',
   ['layer-a',
    'u4816',
    [{'name': 'exp0', 'start': 48.1, 'end': 65.6, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 65.6, 'paused': False}]],
   [655, 'exp0'])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
bucket at range end belongs to next range	`[988, 'y']`	`[988, 'y']`	Passed
bucket at range start is included	`[309, 'x']`	`[309, 'x']`	Passed
paused range reserves its users	`[988, 'y']`	`[988, None]`	Failed
first listed overlapping range wins	`[309, 'narrow']`	`[309, 'wide']`	Failed
percent bounds convert to permille	`[988, 'x']`	`[988, 'x']`	Passed
layer sample 1	`[576, 'exp1']`	`[576, 'exp0']`	Failed
layer sample 2	`[750, None]`	`[750, None]`	Passed
layer sample 3	`[172, None]`	`[172, 'exp0']`	Failed

SHA-256 / 94df9fbfa676153256a68e4fc1814e84c8d7c04132eaeaaecde3280be2a990b7

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(layer_salt, user, experiments):
    bucket = int(hashlib.md5((layer_salt + ':' + user).encode()).hexdigest(), 16) % 1000
    best = None
    for exp in experiments:
        lo = round(exp['start'] * 10)
        hi = round(exp['end'] * 10)
        if lo <= bucket < hi and (best is None or hi - lo < best[0]):
            best = (hi - lo, exp)
    if best is None:
        return [bucket, None]
    if best[1].get('paused'):
        return [bucket, None]
    return [bucket, best[1]['name']]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('layer sample 1',
   ['layer-a',
    'u295',
    [{'name': 'exp0', 'start': 50, 'end': 57.7, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 57.7, 'paused': False}]],
   [576, 'exp0']),
  ('layer sample 2',
   ['layer-b',
    'u4435',
    [{'name': 'exp0', 'start': 0, 'end': 75.0, 'paused': True},
     {'name': 'exp1', 'start': 0, 'end': 74.9, 'paused': False}]],
   [750, None]),
  ('layer sample 3',
   ['layer-a',
    'u3828',
    [{'name': 'exp0', 'start': 1.72, 'end': 17.3, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 17.3, 'paused': True}]],
   [172, 'exp0'])],
 [('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('layer sample 6',
   ['layer-a', 'u2869', [{'name': 'exp0', 'start': 28.1, 'end': 50, 'paused': False}]],
   [282, 'exp0']),
  ('layer sample 18',
   ['layer-b',
    'u4000',
    [{'name': 'exp0', 'start': 0, 'end': 2.83, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 100, 'paused': True},
     {'name': 'exp2', 'start': 28.2, 'end': 28.4, 'paused': False}]],
   [283, None]),
  ('layer sample 48',
   ['layer-a',
    'u431',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 20.0, 'end': 20.2, 'paused': True}]],
   [201, 'exp0'])],
 [('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 11',
   ['layer-b',
    'u3995',
    [{'name': 'exp0', 'start': 50, 'end': 84.5, 'paused': True},
     {'name': 'exp1', 'start': 8.45, 'end': 84.5, 'paused': True}]],
   [845, None]),
  ('layer sample 39',
   ['layer-b',
    'u2441',
    [{'name': 'exp0', 'start': 0, 'end': 75.8, 'paused': False},
     {'name': 'exp1', 'start': 43.8, 'end': 100, 'paused': False}]],
   [439, 'exp0']),
  ('layer sample 47',
   ['layer-b',
    'u2172',
    [{'name': 'exp0', 'start': 89.1, 'end': 89.3, 'paused': False},
     {'name': 'exp1', 'start': 89.1, 'end': 100, 'paused': False}]],
   [892, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 4',
   ['layer-a',
    'u309',
    [{'name': 'exp0', 'start': 6.0, 'end': 50, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 5.9, 'paused': False},
     {'name': 'exp2', 'start': 6.0, 'end': 50, 'paused': False}]],
   [60, 'exp0']),
  ('layer sample 16',
   ['layer-a',
    'u4745',
    [{'name': 'exp0', 'start': 81.9, 'end': 100, 'paused': True},
     {'name': 'exp1', 'start': 8.19, 'end': 82.0, 'paused': True},
     {'name': 'exp2', 'start': 50, 'end': 82.0, 'paused': False}]],
   [819, None]),
  ('layer sample 28',
   ['layer-a',
    'u2535',
    [{'name': 'exp0', 'start': 5.59, 'end': 56.0, 'paused': False},
     {'name': 'exp1', 'start': 5.59, 'end': 100, 'paused': False},
     {'name': 'exp2', 'start': 52.4, 'end': 100, 'paused': False}]],
   [559, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 14',
   ['layer-b',
    'u1256',
    [{'name': 'exp0', 'start': 50, 'end': 89.8, 'paused': False},
     {'name': 'exp1', 'start': 89.7, 'end': 89.8, 'paused': False}]],
   [897, 'exp0']),
  ('layer sample 21',
   ['layer-a',
    'u505',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 54.1, 'end': 100, 'paused': False}]],
   [540, 'exp0']),
  ('layer sample 38',
   ['layer-a',
    'u4816',
    [{'name': 'exp0', 'start': 48.1, 'end': 65.6, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 65.6, 'paused': False}]],
   [655, 'exp0'])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
bucket at range end belongs to next range	`[988, 'y']`	`[988, 'y']`	Passed
bucket at range start is included	`[309, 'x']`	`[309, 'x']`	Passed
paused range reserves its users	`[988, None]`	`[988, None]`	Passed
first listed overlapping range wins	`[309, 'narrow']`	`[309, 'wide']`	Failed
percent bounds convert to permille	`[988, 'x']`	`[988, 'x']`	Passed
layer sample 1	`[576, 'exp0']`	`[576, 'exp0']`	Passed
layer sample 2	`[750, None]`	`[750, None]`	Passed
layer sample 3	`[172, 'exp0']`	`[172, 'exp0']`	Passed

SHA-256 / e78e6a802a98a7f53512f2302b046e17c7276d76fc1d557470155276d63f68ea

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(layer_salt, user, experiments):
    bucket = int(hashlib.md5((layer_salt + ':' + user).encode()).hexdigest(), 16) % 1000
    for exp in experiments:
        lo = round(exp['start'] * 10)
        hi = round(exp['end'] * 10)
        if lo <= bucket < hi:
            if exp.get('paused'):
                return [bucket, None]
            return [bucket, exp['name']]
    return [bucket, None]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('layer sample 1',
   ['layer-a',
    'u295',
    [{'name': 'exp0', 'start': 50, 'end': 57.7, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 57.7, 'paused': False}]],
   [576, 'exp0']),
  ('layer sample 2',
   ['layer-b',
    'u4435',
    [{'name': 'exp0', 'start': 0, 'end': 75.0, 'paused': True},
     {'name': 'exp1', 'start': 0, 'end': 74.9, 'paused': False}]],
   [750, None]),
  ('layer sample 3',
   ['layer-a',
    'u3828',
    [{'name': 'exp0', 'start': 1.72, 'end': 17.3, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 17.3, 'paused': True}]],
   [172, 'exp0'])],
 [('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('layer sample 6',
   ['layer-a', 'u2869', [{'name': 'exp0', 'start': 28.1, 'end': 50, 'paused': False}]],
   [282, 'exp0']),
  ('layer sample 18',
   ['layer-b',
    'u4000',
    [{'name': 'exp0', 'start': 0, 'end': 2.83, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 100, 'paused': True},
     {'name': 'exp2', 'start': 28.2, 'end': 28.4, 'paused': False}]],
   [283, None]),
  ('layer sample 48',
   ['layer-a',
    'u431',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 20.0, 'end': 20.2, 'paused': True}]],
   [201, 'exp0'])],
 [('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 11',
   ['layer-b',
    'u3995',
    [{'name': 'exp0', 'start': 50, 'end': 84.5, 'paused': True},
     {'name': 'exp1', 'start': 8.45, 'end': 84.5, 'paused': True}]],
   [845, None]),
  ('layer sample 39',
   ['layer-b',
    'u2441',
    [{'name': 'exp0', 'start': 0, 'end': 75.8, 'paused': False},
     {'name': 'exp1', 'start': 43.8, 'end': 100, 'paused': False}]],
   [439, 'exp0']),
  ('layer sample 47',
   ['layer-b',
    'u2172',
    [{'name': 'exp0', 'start': 89.1, 'end': 89.3, 'paused': False},
     {'name': 'exp1', 'start': 89.1, 'end': 100, 'paused': False}]],
   [892, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 4',
   ['layer-a',
    'u309',
    [{'name': 'exp0', 'start': 6.0, 'end': 50, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 5.9, 'paused': False},
     {'name': 'exp2', 'start': 6.0, 'end': 50, 'paused': False}]],
   [60, 'exp0']),
  ('layer sample 16',
   ['layer-a',
    'u4745',
    [{'name': 'exp0', 'start': 81.9, 'end': 100, 'paused': True},
     {'name': 'exp1', 'start': 8.19, 'end': 82.0, 'paused': True},
     {'name': 'exp2', 'start': 50, 'end': 82.0, 'paused': False}]],
   [819, None]),
  ('layer sample 28',
   ['layer-a',
    'u2535',
    [{'name': 'exp0', 'start': 5.59, 'end': 56.0, 'paused': False},
     {'name': 'exp1', 'start': 5.59, 'end': 100, 'paused': False},
     {'name': 'exp2', 'start': 52.4, 'end': 100, 'paused': False}]],
   [559, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 14',
   ['layer-b',
    'u1256',
    [{'name': 'exp0', 'start': 50, 'end': 89.8, 'paused': False},
     {'name': 'exp1', 'start': 89.7, 'end': 89.8, 'paused': False}]],
   [897, 'exp0']),
  ('layer sample 21',
   ['layer-a',
    'u505',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 54.1, 'end': 100, 'paused': False}]],
   [540, 'exp0']),
  ('layer sample 38',
   ['layer-a',
    'u4816',
    [{'name': 'exp0', 'start': 48.1, 'end': 65.6, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 65.6, 'paused': False}]],
   [655, 'exp0'])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
bucket at range end belongs to next range	`[988, 'y']`	`[988, 'y']`	Passed
bucket at range start is included	`[309, 'x']`	`[309, 'x']`	Passed
paused range reserves its users	`[988, None]`	`[988, None]`	Passed
first listed overlapping range wins	`[309, 'wide']`	`[309, 'wide']`	Passed
percent bounds convert to permille	`[988, 'x']`	`[988, 'x']`	Passed
layer sample 1	`[576, 'exp0']`	`[576, 'exp0']`	Passed
layer sample 2	`[750, None]`	`[750, None]`	Passed
layer sample 3	`[172, 'exp0']`	`[172, 'exp0']`	Passed

SHA-256 / dcbe563ae8bb78574c4fe0858f3539fd3f7ee95b0a0c159491cdf74c188a9f8b

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:53.253971+00:00.

Case digest / 16221ce2f8a62a3bc646f106ba6ae6e1a72fee9e4a7d47fbb73e17305132dec9