FAILURE MAP
← Case archive

FA-74061 / Feature flag rollout bucketing / Open access

Sticky variant persistence: Exclusion is checked before sticky records · case 01

Reducing traffic allocation drops users who already had a valid sticky assignment.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The excluded branch runs before the sticky record is examined.

THE FAILURE

The excluded branch runs before the sticky record is examined.

Unsuccessful approach: Excluding only record-less users still assigns a None variant when the record is invalid.

Case contract

stored is None or {variant, version}. A stored record is honoured (reason sticky, record unchanged) only if its version equals the current experiment version and its variant is still active; this takes precedence over traffic exclusion. Otherwise a None bucket_variant means excluded and the stored record is kept untouched; else the bucketed variant is assigned and a new record {variant, version} is returned. Return [variant, reason, record].

Why this case matters

Sticky bucketing keeps users on one arm across sessions; stale or retired records must not leak.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if bucket_variant is None:
        return [None, 'excluded', stored]
    if stored and stored.get('version') == version and stored.get('variant') in variants:
        return [stored['variant'], 'sticky', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 14',
   [{'variant': 'c', 'version': 2}, ['c'], None, 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 9',
   [{'variant': 'a', 'version': 2}, ['c'], None, 2],
   [None, 'excluded', {'variant': 'a', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 19',
   [{'variant': 'c', 'version': 2}, ['c'], None, 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 17', [None, ['b', 'a'], 'a', 1], ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 45',
   [{'variant': 'b', 'version': 2}, ['a', 'c'], None, 1],
   [None, 'excluded', {'variant': 'b', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 14',
   [{'variant': 'c', 'version': 2}, ['c'], None, 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 29',
   [{'variant': 'c', 'version': 4}, ['a'], None, 3],
   [None, 'excluded', {'variant': 'c', 'version': 4}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
retired variant is reassigned['b', 'assigned', {'variant': 'b', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Passed
exclusion keeps the stored record[None, 'excluded', {'variant': 'c', 'version': 1}][None, 'excluded', {'variant': 'c', 'version': 1}]Passed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
sticky beats traffic exclusion[None, 'excluded', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Failed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed

SHA-256 / 06d58cdc094ae3825c167548e0a76048160d083ecb4b46d198ccc29ed789219c

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(stored, variants, bucket_variant, version):
    if stored and stored.get('version') == version and stored.get('variant') in variants:
        return [stored['variant'], 'sticky', stored]
    if bucket_variant is None and not stored:
        return [None, 'excluded', stored]
    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('record from previous version is reassigned',
   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 1',
   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],
   ['c', 'assigned', {'variant': 'c', 'version': 2}]),
  ('assignment sample 2',
   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}])],
 [('legacy record without version is reassigned',
   [{'variant': 'b'}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('assignment sample 6',
   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],
   ['a', 'assigned', {'variant': 'a', 'version': 3}]),
  ('assignment sample 14',
   [{'variant': 'c', 'version': 2}, ['c'], None, 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])],
 [('retired variant is reassigned',
   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],
   ['b', 'assigned', {'variant': 'b', 'version': 2}]),
  ('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('assignment sample 9',
   [{'variant': 'a', 'version': 2}, ['c'], None, 2],
   [None, 'excluded', {'variant': 'a', 'version': 2}]),
  ('assignment sample 11',
   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 19',
   [{'variant': 'c', 'version': 2}, ['c'], None, 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}])],
 [('exclusion keeps the stored record',
   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],
   [None, 'excluded', {'variant': 'c', 'version': 1}]),
  ('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 16',
   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],
   ['c', 'assigned', {'variant': 'c', 'version': 3}]),
  ('assignment sample 17', [None, ['b', 'a'], 'a', 1], ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('assignment sample 45',
   [{'variant': 'b', 'version': 2}, ['a', 'c'], None, 1],
   [None, 'excluded', {'variant': 'b', 'version': 2}])],
 [('record from a newer version is reassigned',
   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],
   ['a', 'assigned', {'variant': 'a', 'version': 2}]),
  ('sticky beats traffic exclusion',
   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],
   ['a', 'sticky', {'variant': 'a', 'version': 2}]),
  ('valid record stays sticky',
   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],
   ['b', 'sticky', {'variant': 'b', 'version': 2}]),
  ('first assignment writes a record',
   [None, ['a', 'b'], 'a', 1],
   ['a', 'assigned', {'variant': 'a', 'version': 1}]),
  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),
  ('assignment sample 14',
   [{'variant': 'c', 'version': 2}, ['c'], None, 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 21',
   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],
   ['c', 'sticky', {'variant': 'c', 'version': 2}]),
  ('assignment sample 29',
   [{'variant': 'c', 'version': 4}, ['a'], None, 3],
   [None, 'excluded', {'variant': 'c', 'version': 4}])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
record from previous version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
legacy record without version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
retired variant is reassigned['b', 'assigned', {'variant': 'b', 'version': 2}]['b', 'assigned', {'variant': 'b', 'version': 2}]Passed
exclusion keeps the stored record[None, 'assigned', {'variant': None, 'version': 2}][None, 'excluded', {'variant': 'c', 'version': 1}]Failed
record from a newer version is reassigned['a', 'assigned', {'variant': 'a', 'version': 2}]['a', 'assigned', {'variant': 'a', 'version': 2}]Passed
sticky beats traffic exclusion['a', 'sticky', {'variant': 'a', 'version': 2}]['a', 'sticky', {'variant': 'a', 'version': 2}]Passed
assignment sample 1['c', 'assigned', {'variant': 'c', 'version': 2}]['c', 'assigned', {'variant': 'c', 'version': 2}]Passed
assignment sample 2['a', 'assigned', {'variant': 'a', 'version': 3}]['a', 'assigned', {'variant': 'a', 'version': 3}]Passed

SHA-256 / 044796ec17a7da6db5f37edc17e03b3429ac1d1cd4be77abe67e99d0595524c7

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 8 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Sign in to the archive ↗

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:53.242933+00:00.

Case digest / f16ad1068fe531d82ab36caa811ef016de191e745c6a22e62c96704dddf7001c