FAILURE MAP
← Case archive

FA-74041 / Feature flag rollout bucketing / Open access

Sticky variant persistence: Records from older experiment versions stay sticky · case 01

After an experiment restart users keep the arm from the previous version.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The stored version is never compared with the current experiment version.

VERIFIED REPAIR

Honour a stored record only when its version equals the current version.

Unsuccessful approach: Defaulting a missing version to the current one treats unversioned legacy records as current.

Case contract

stored is None or {variant, version}. A stored record is honoured (reason sticky, record unchanged) only if its version equals the current experiment version and its variant is still active; this takes precedence over traffic exclusion. Otherwise a None bucket_variant means excluded and the stored record is kept untouched; else the bucketed variant is assigned and a new record {variant, version} is returned. Return [variant, reason, record].

Why this case matters

Sticky bucketing keeps users on one arm across sessions; stale or retired records must not leak.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('variant') in variants:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 3',
   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 10',
   [{'variant': 'b'}, ['c', 'b', 'a'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 60',
   [{'variant': 'c'}, ['a', 'c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}]),
  ('assignment sample 58',
   [{'variant': 'c'}, ['c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 47',
   [{'variant': 'b'}, ['c', 'b'], 'b', 3],
   ['b', 'assigned', {'variant': 'b', 'version': 3}]),
  ('assignment sample 60',
   [{'variant': 'c'}, ['a', 'c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 7',
   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],
   ['c', 'assigned', {'variant': 'c', 'version': 1}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['b', 'sticky', {'variant': 'b', 'version': 1}]['a', 'assigned', {'variant': 'a', 'version': 2}]Failed
legacy record without version is reassigned['b', 'sticky', {'variant': 'b'}]['a', 'assigned', {'variant': 'a', 'version': 2}]Failed
retired variant is reassigned['b', 'assigned', {'variant': 'b', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Passed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['b', 'sticky', {'variant': 'b', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 2}]Failed
assignment sample 1['b', 'sticky', {'variant': 'b', 'version': 3}]['c', 'assigned', {'variant': 'c', 'version': 2}]Failed
assignment sample 2['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 3}]Failed
assignment sample 3['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed

SHA-256 / e81cb1cab2b52ea34981986745d6f210460f87ae74d2cd2bdf8d344104eebb55

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('version', version) == version and stored.get('variant') in variants:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 3',
   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 10',
   [{'variant': 'b'}, ['c', 'b', 'a'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 60',
   [{'variant': 'c'}, ['a', 'c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}]),
  ('assignment sample 58',
   [{'variant': 'c'}, ['c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 47',
   [{'variant': 'b'}, ['c', 'b'], 'b', 3],
   ['b', 'assigned', {'variant': 'b', 'version': 3}]),
  ('assignment sample 60',
   [{'variant': 'c'}, ['a', 'c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 7',
   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],
   ['c', 'assigned', {'variant': 'c', 'version': 1}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['b', 'sticky', {'variant': 'b'}]['a', 'assigned', {'variant': 'a', 'version': 2}]Failed
retired variant is reassigned['b', 'assigned', {'variant': 'b', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Passed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed
assignment sample 3['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed

SHA-256 / 0817ab7e4b9b25bbfbd64e5fe231c729b041f748fa57164050153abc246ffdde

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('version') == version and stored.get('variant') in variants:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 3',
   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 10',
   [{'variant': 'b'}, ['c', 'b', 'a'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 60',
   [{'variant': 'c'}, ['a', 'c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}]),
  ('assignment sample 58',
   [{'variant': 'c'}, ['c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 47',
   [{'variant': 'b'}, ['c', 'b'], 'b', 3],
   ['b', 'assigned', {'variant': 'b', 'version': 3}]),
  ('assignment sample 60',
   [{'variant': 'c'}, ['a', 'c'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 7',
   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],
   ['c', 'assigned', {'variant': 'c', 'version': 1}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
retired variant is reassigned['b', 'assigned', {'variant': 'b', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Passed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed
assignment sample 3['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed

SHA-256 / 7a2b204c0891068b0b07d5ac830bf29a14639bb4aa8203ec2fc8f4f6cd2ddb4e

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:53.121537+00:00.

Case digest / f653393e8bbaefbb6abd96b2498b35d3f828f544e91b431e90a33ea46caae909