FAILURE MAP
← Case archive

FA-74046 / Feature flag rollout bucketing / Open access

Sticky variant persistence: Retired variants remain sticky · case 01

Users keep receiving a variant that has been removed from the experiment.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The stored variant is not checked against the list of active variants.

VERIFIED REPAIR

Require the stored variant to still be an active variant.

Unsuccessful approach: Checking only that a variant is recorded still accepts retired variants.

Case contract

stored is None or {variant, version}. A stored record is honoured (reason sticky, record unchanged) only if its version equals the current experiment version and its variant is still active; this takes precedence over traffic exclusion. Otherwise a None bucket_variant means excluded and the stored record is kept untouched; else the bucketed variant is assigned and a new record {variant, version} is returned. Return [variant, reason, record].

Why this case matters

Sticky bucketing keeps users on one arm across sessions; stale or retired records must not leak.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('version') == version:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 3',
   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 7',
   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],
   ['c', 'assigned', {'variant': 'c', 'version': 1}]),
  ('assignment sample 39',
   [{'variant': 'b', 'version': 2}, ['c'], None, 2],
   [None, 'excluded', {'variant': 'b', 'version': 2}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 12',
   [None, ['a', 'c', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 31',
   [{'variant': 'a', 'version': 3}, ['b'], 'b', 3],
   ['b', 'assigned', {'variant': 'b', 'version': 3}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 17', [None, ['b', 'a'], 'a', 1], ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 28',
   [{'variant': 'a', 'version': 2}, ['c', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 4',
   [{'variant': 'b', 'version': 3}, ['c'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 22',
   [{'variant': 'c', 'version': 0}, ['b'], 'b', 1],
   ['b', 'assigned', {'variant': 'b', 'version': 1}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
retired variant is reassigned['c', 'sticky', {'variant': 'c', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Failed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed
assignment sample 3['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed

SHA-256 / dd719d0eda745d19678e228a508b299bc3538bd37304beba0bcc35d77862cbbc

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('version') == version and stored.get('variant') is not None:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 3',
   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 7',
   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],
   ['c', 'assigned', {'variant': 'c', 'version': 1}]),
  ('assignment sample 39',
   [{'variant': 'b', 'version': 2}, ['c'], None, 2],
   [None, 'excluded', {'variant': 'b', 'version': 2}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 12',
   [None, ['a', 'c', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 31',
   [{'variant': 'a', 'version': 3}, ['b'], 'b', 3],
   ['b', 'assigned', {'variant': 'b', 'version': 3}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 17', [None, ['b', 'a'], 'a', 1], ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 28',
   [{'variant': 'a', 'version': 2}, ['c', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 4',
   [{'variant': 'b', 'version': 3}, ['c'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 22',
   [{'variant': 'c', 'version': 0}, ['b'], 'b', 1],
   ['b', 'assigned', {'variant': 'b', 'version': 1}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
retired variant is reassigned['c', 'sticky', {'variant': 'c', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Failed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed
assignment sample 3['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed

SHA-256 / be43df880789b3b7176f2d2c2d7a215c1ef7708d0fd723793ebc539d88a12c5e

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('version') == version and stored.get('variant') in variants:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 3',
   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 7',
   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],
   ['c', 'assigned', {'variant': 'c', 'version': 1}]),
  ('assignment sample 39',
   [{'variant': 'b', 'version': 2}, ['c'], None, 2],
   [None, 'excluded', {'variant': 'b', 'version': 2}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 12',
   [None, ['a', 'c', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('assignment sample 31',
   [{'variant': 'a', 'version': 3}, ['b'], 'b', 3],
   ['b', 'assigned', {'variant': 'b', 'version': 3}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 17', [None, ['b', 'a'], 'a', 1], ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 28',
   [{'variant': 'a', 'version': 2}, ['c', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 4',
   [{'variant': 'b', 'version': 3}, ['c'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 22',
   [{'variant': 'c', 'version': 0}, ['b'], 'b', 1],
   ['b', 'assigned', {'variant': 'b', 'version': 1}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
retired variant is reassigned['b', 'assigned', {'variant': 'b', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Passed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed
assignment sample 3['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed

SHA-256 / 0903277ef2ced3d97126fd569959e9cfa88a7c7008d3b4855c01e1e113aa1087

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:53.118972+00:00.

Case digest / b5cf8dce7635e59f645e4ccc1d980cd1c4e1c5100a54d7de8730061a60a1280c