FAILURE MAP
← Case archive

FA-74081 / Feature flag rollout bucketing / Open access

Mutually exclusive layer allocation: Percent bounds are scaled to basis points · case 01

Every range reaches ten times too far, so the first experiment captures nearly the whole layer.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Range bounds are multiplied by 100 while buckets are permille.

VERIFIED REPAIR

Multiply percent bounds by 10 to express them in permille.

Unsuccessful approach: Using the raw percent values treats them as permille and shrinks ranges tenfold.

Case contract

bucket = int(md5(layer_salt + ":" + user) hex, 16) % 1000. experiments is an ordered list of {name, start, end, paused?} with start/end in percent (one decimal) converted to permille by round(x * 10); ranges are half-open [start, end). The first listed range containing the bucket decides: a paused experiment reserves its range (user gets None). Return [bucket, name or None].

Why this case matters

Layers keep concurrent experiments from overlapping on the same users.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(layer_salt, user, experiments):
    bucket = int(hashlib.md5((layer_salt + ':' + user).encode()).hexdigest(), 16) % 1000
    for exp in experiments:
        lo = round(exp['start'] * 100)
        hi = round(exp['end'] * 100)
        if lo <= bucket < hi:
            if exp.get('paused'):
                return [bucket, None]
            return [bucket, exp['name']]
    return [bucket, None]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('layer sample 1',
   ['layer-a',
    'u295',
    [{'name': 'exp0', 'start': 50, 'end': 57.7, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 57.7, 'paused': False}]],
   [576, 'exp0']),
  ('layer sample 2',
   ['layer-b',
    'u4435',
    [{'name': 'exp0', 'start': 0, 'end': 75.0, 'paused': True},
     {'name': 'exp1', 'start': 0, 'end': 74.9, 'paused': False}]],
   [750, None]),
  ('layer sample 3',
   ['layer-a',
    'u3828',
    [{'name': 'exp0', 'start': 1.72, 'end': 17.3, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 17.3, 'paused': True}]],
   [172, 'exp0'])],
 [('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('layer sample 5',
   ['layer-b',
    'u45',
    [{'name': 'exp0', 'start': 0, 'end': 88.1, 'paused': True},
     {'name': 'exp1', 'start': 8.81, 'end': 88.2, 'paused': False},
     {'name': 'exp2', 'start': 0, 'end': 88.0, 'paused': False}]],
   [881, 'exp1']),
  ('layer sample 6',
   ['layer-a', 'u2869', [{'name': 'exp0', 'start': 28.1, 'end': 50, 'paused': False}]],
   [282, 'exp0']),
  ('layer sample 9',
   ['layer-b', 'u1279', [{'name': 'exp0', 'start': 38.6, 'end': 100, 'paused': False}]],
   [386, 'exp0'])],
 [('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 11',
   ['layer-b',
    'u3995',
    [{'name': 'exp0', 'start': 50, 'end': 84.5, 'paused': True},
     {'name': 'exp1', 'start': 8.45, 'end': 84.5, 'paused': True}]],
   [845, None]),
  ('layer sample 15',
   ['layer-a',
    'u2371',
    [{'name': 'exp0', 'start': 74.2, 'end': 74.3, 'paused': False},
     {'name': 'exp1', 'start': 56.3, 'end': 74.4, 'paused': False}]],
   [743, 'exp1']),
  ('layer sample 38',
   ['layer-a',
    'u4816',
    [{'name': 'exp0', 'start': 48.1, 'end': 65.6, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 65.6, 'paused': False}]],
   [655, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 16',
   ['layer-a',
    'u4745',
    [{'name': 'exp0', 'start': 81.9, 'end': 100, 'paused': True},
     {'name': 'exp1', 'start': 8.19, 'end': 82.0, 'paused': True},
     {'name': 'exp2', 'start': 50, 'end': 82.0, 'paused': False}]],
   [819, None]),
  ('layer sample 17',
   ['layer-a',
    'u1058',
    [{'name': 'exp0', 'start': 39.5, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 39.3, 'end': 39.4, 'paused': False}]],
   [394, None]),
  ('layer sample 32',
   ['layer-b', 'u4026', [{'name': 'exp0', 'start': 89.5, 'end': 100, 'paused': False}]],
   [895, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 9',
   ['layer-b', 'u1279', [{'name': 'exp0', 'start': 38.6, 'end': 100, 'paused': False}]],
   [386, 'exp0']),
  ('layer sample 21',
   ['layer-a',
    'u505',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 54.1, 'end': 100, 'paused': False}]],
   [540, 'exp0']),
  ('layer sample 47',
   ['layer-b',
    'u2172',
    [{'name': 'exp0', 'start': 89.1, 'end': 89.3, 'paused': False},
     {'name': 'exp1', 'start': 89.1, 'end': 100, 'paused': False}]],
   [892, 'exp0'])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
bucket at range end belongs to next range[988, 'x'][988, 'y']Failed
bucket at range start is included[309, None][309, 'x']Failed
paused range reserves its users[988, None][988, None]Passed
first listed overlapping range wins[309, 'wide'][309, 'wide']Passed
percent bounds convert to permille[988, None][988, 'x']Failed
layer sample 1[576, 'exp1'][576, 'exp0']Failed
layer sample 2[750, None][750, None]Passed
layer sample 3[172, 'exp0'][172, 'exp0']Passed

SHA-256 / c092a4a152aefebfe60de9b10f108e3b16ff9868cc1e84ee4fe7a50e7958f7d7

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(layer_salt, user, experiments):
    bucket = int(hashlib.md5((layer_salt + ':' + user).encode()).hexdigest(), 16) % 1000
    for exp in experiments:
        lo = round(exp['start'])
        hi = round(exp['end'])
        if lo <= bucket < hi:
            if exp.get('paused'):
                return [bucket, None]
            return [bucket, exp['name']]
    return [bucket, None]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('layer sample 1',
   ['layer-a',
    'u295',
    [{'name': 'exp0', 'start': 50, 'end': 57.7, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 57.7, 'paused': False}]],
   [576, 'exp0']),
  ('layer sample 2',
   ['layer-b',
    'u4435',
    [{'name': 'exp0', 'start': 0, 'end': 75.0, 'paused': True},
     {'name': 'exp1', 'start': 0, 'end': 74.9, 'paused': False}]],
   [750, None]),
  ('layer sample 3',
   ['layer-a',
    'u3828',
    [{'name': 'exp0', 'start': 1.72, 'end': 17.3, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 17.3, 'paused': True}]],
   [172, 'exp0'])],
 [('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('layer sample 5',
   ['layer-b',
    'u45',
    [{'name': 'exp0', 'start': 0, 'end': 88.1, 'paused': True},
     {'name': 'exp1', 'start': 8.81, 'end': 88.2, 'paused': False},
     {'name': 'exp2', 'start': 0, 'end': 88.0, 'paused': False}]],
   [881, 'exp1']),
  ('layer sample 6',
   ['layer-a', 'u2869', [{'name': 'exp0', 'start': 28.1, 'end': 50, 'paused': False}]],
   [282, 'exp0']),
  ('layer sample 9',
   ['layer-b', 'u1279', [{'name': 'exp0', 'start': 38.6, 'end': 100, 'paused': False}]],
   [386, 'exp0'])],
 [('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 11',
   ['layer-b',
    'u3995',
    [{'name': 'exp0', 'start': 50, 'end': 84.5, 'paused': True},
     {'name': 'exp1', 'start': 8.45, 'end': 84.5, 'paused': True}]],
   [845, None]),
  ('layer sample 15',
   ['layer-a',
    'u2371',
    [{'name': 'exp0', 'start': 74.2, 'end': 74.3, 'paused': False},
     {'name': 'exp1', 'start': 56.3, 'end': 74.4, 'paused': False}]],
   [743, 'exp1']),
  ('layer sample 38',
   ['layer-a',
    'u4816',
    [{'name': 'exp0', 'start': 48.1, 'end': 65.6, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 65.6, 'paused': False}]],
   [655, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 16',
   ['layer-a',
    'u4745',
    [{'name': 'exp0', 'start': 81.9, 'end': 100, 'paused': True},
     {'name': 'exp1', 'start': 8.19, 'end': 82.0, 'paused': True},
     {'name': 'exp2', 'start': 50, 'end': 82.0, 'paused': False}]],
   [819, None]),
  ('layer sample 17',
   ['layer-a',
    'u1058',
    [{'name': 'exp0', 'start': 39.5, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 39.3, 'end': 39.4, 'paused': False}]],
   [394, None]),
  ('layer sample 32',
   ['layer-b', 'u4026', [{'name': 'exp0', 'start': 89.5, 'end': 100, 'paused': False}]],
   [895, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 9',
   ['layer-b', 'u1279', [{'name': 'exp0', 'start': 38.6, 'end': 100, 'paused': False}]],
   [386, 'exp0']),
  ('layer sample 21',
   ['layer-a',
    'u505',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 54.1, 'end': 100, 'paused': False}]],
   [540, 'exp0']),
  ('layer sample 47',
   ['layer-b',
    'u2172',
    [{'name': 'exp0', 'start': 89.1, 'end': 89.3, 'paused': False},
     {'name': 'exp1', 'start': 89.1, 'end': 100, 'paused': False}]],
   [892, 'exp0'])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
bucket at range end belongs to next range[988, None][988, 'y']Failed
bucket at range start is included[309, None][309, 'x']Failed
paused range reserves its users[988, None][988, None]Passed
first listed overlapping range wins[309, None][309, 'wide']Failed
percent bounds convert to permille[988, None][988, 'x']Failed
layer sample 1[576, None][576, 'exp0']Failed
layer sample 2[750, None][750, None]Passed
layer sample 3[172, None][172, 'exp0']Failed

SHA-256 / 62b0801318df9fa27824d91e33840c32eaaea2017deb6b01c610315bf1bd4261

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json
import hashlib
N = 1
observations = []
def solve(layer_salt, user, experiments):
    bucket = int(hashlib.md5((layer_salt + ':' + user).encode()).hexdigest(), 16) % 1000
    for exp in experiments:
        lo = round(exp['start'] * 10)
        hi = round(exp['end'] * 10)
        if lo <= bucket < hi:
            if exp.get('paused'):
                return [bucket, None]
            return [bucket, exp['name']]
    return [bucket, None]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('layer sample 1',
   ['layer-a',
    'u295',
    [{'name': 'exp0', 'start': 50, 'end': 57.7, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 57.7, 'paused': False}]],
   [576, 'exp0']),
  ('layer sample 2',
   ['layer-b',
    'u4435',
    [{'name': 'exp0', 'start': 0, 'end': 75.0, 'paused': True},
     {'name': 'exp1', 'start': 0, 'end': 74.9, 'paused': False}]],
   [750, None]),
  ('layer sample 3',
   ['layer-a',
    'u3828',
    [{'name': 'exp0', 'start': 1.72, 'end': 17.3, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 17.3, 'paused': True}]],
   [172, 'exp0'])],
 [('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('layer sample 5',
   ['layer-b',
    'u45',
    [{'name': 'exp0', 'start': 0, 'end': 88.1, 'paused': True},
     {'name': 'exp1', 'start': 8.81, 'end': 88.2, 'paused': False},
     {'name': 'exp2', 'start': 0, 'end': 88.0, 'paused': False}]],
   [881, 'exp1']),
  ('layer sample 6',
   ['layer-a', 'u2869', [{'name': 'exp0', 'start': 28.1, 'end': 50, 'paused': False}]],
   [282, 'exp0']),
  ('layer sample 9',
   ['layer-b', 'u1279', [{'name': 'exp0', 'start': 38.6, 'end': 100, 'paused': False}]],
   [386, 'exp0'])],
 [('paused range reserves its users',
   ['layer-a',
    'alice',
    [{'name': 'x', 'start': 0, 'end': 100, 'paused': True}, {'name': 'y', 'start': 0, 'end': 100}]],
   [988, None]),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 11',
   ['layer-b',
    'u3995',
    [{'name': 'exp0', 'start': 50, 'end': 84.5, 'paused': True},
     {'name': 'exp1', 'start': 8.45, 'end': 84.5, 'paused': True}]],
   [845, None]),
  ('layer sample 15',
   ['layer-a',
    'u2371',
    [{'name': 'exp0', 'start': 74.2, 'end': 74.3, 'paused': False},
     {'name': 'exp1', 'start': 56.3, 'end': 74.4, 'paused': False}]],
   [743, 'exp1']),
  ('layer sample 38',
   ['layer-a',
    'u4816',
    [{'name': 'exp0', 'start': 48.1, 'end': 65.6, 'paused': False},
     {'name': 'exp1', 'start': 0, 'end': 65.6, 'paused': False}]],
   [655, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('first listed overlapping range wins',
   ['layer-b',
    'bob',
    [{'name': 'wide', 'start': 0, 'end': 100}, {'name': 'narrow', 'start': 30.9, 'end': 31.0}]],
   [309, 'wide']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 16',
   ['layer-a',
    'u4745',
    [{'name': 'exp0', 'start': 81.9, 'end': 100, 'paused': True},
     {'name': 'exp1', 'start': 8.19, 'end': 82.0, 'paused': True},
     {'name': 'exp2', 'start': 50, 'end': 82.0, 'paused': False}]],
   [819, None]),
  ('layer sample 17',
   ['layer-a',
    'u1058',
    [{'name': 'exp0', 'start': 39.5, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 39.3, 'end': 39.4, 'paused': False}]],
   [394, None]),
  ('layer sample 32',
   ['layer-b', 'u4026', [{'name': 'exp0', 'start': 89.5, 'end': 100, 'paused': False}]],
   [895, 'exp0'])],
 [('bucket at range end belongs to next range',
   ['layer-a', 'alice', [{'name': 'x', 'start': 0, 'end': 98.8}, {'name': 'y', 'start': 98.8, 'end': 100}]],
   [988, 'y']),
  ('bucket at range start is included',
   ['layer-b', 'bob', [{'name': 'x', 'start': 30.9, 'end': 100}]],
   [309, 'x']),
  ('percent bounds convert to permille',
   ['layer-a', 'alice', [{'name': 'x', 'start': 98.3, 'end': 99.3}]],
   [988, 'x']),
  ('no range matches', ['layer-b', 'bob', [{'name': 'x', 'start': 0, 'end': 0}]], [309, None]),
  ('empty experiment list', ['layer-a', 'carol', []], [970, None]),
  ('layer sample 9',
   ['layer-b', 'u1279', [{'name': 'exp0', 'start': 38.6, 'end': 100, 'paused': False}]],
   [386, 'exp0']),
  ('layer sample 21',
   ['layer-a',
    'u505',
    [{'name': 'exp0', 'start': 0, 'end': 100, 'paused': False},
     {'name': 'exp1', 'start': 54.1, 'end': 100, 'paused': False}]],
   [540, 'exp0']),
  ('layer sample 47',
   ['layer-b',
    'u2172',
    [{'name': 'exp0', 'start': 89.1, 'end': 89.3, 'paused': False},
     {'name': 'exp1', 'start': 89.1, 'end': 100, 'paused': False}]],
   [892, 'exp0'])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
bucket at range end belongs to next range[988, 'y'][988, 'y']Passed
bucket at range start is included[309, 'x'][309, 'x']Passed
paused range reserves its users[988, None][988, None]Passed
first listed overlapping range wins[309, 'wide'][309, 'wide']Passed
percent bounds convert to permille[988, 'x'][988, 'x']Passed
layer sample 1[576, 'exp0'][576, 'exp0']Passed
layer sample 2[750, None][750, None]Passed
layer sample 3[172, 'exp0'][172, 'exp0']Passed

SHA-256 / 7fdd34ab72f4e445595db5c3127957fe1176660545db27780c2e837980a43c47

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:53.298920+00:00.

Case digest / d586bfc96e856d06f113a2d510184ea26fa7a86475e2e74ddb16681a22567e50