{"abstract":"After an experiment restart users keep the arm from the previous version.","category":"Feature flag rollout bucketing","checks":8,"contract":"stored is None or {variant, version}. A stored record is honoured (reason sticky, record unchanged) only if its version equals the current experiment version and its variant is still active; this takes precedence over traffic exclusion. Otherwise a None bucket_variant means excluded and the stored record is kept untouched; else the bucketed variant is assigned and a new record {variant, version} is returned. Return [variant, reason, record].","evaluation_group":"w2-feature-flag-rollout-bucketing-sticky-assignment","failed_approach":"Defaulting a missing version to the current one treats unversioned legacy records as current.","family":"w2-feature-flag-rollout-bucketing-sticky-assignment-version-check","id":"FA-74041","implementations":{"attempt":{"sha256":"0817ab7e4b9b25bbfbd64e5fe231c729b041f748fa57164050153abc246ffdde","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(stored, variants, bucket_variant, version):\n    if stored and stored.get('version', version) == version and stored.get('variant') in variants:\n        return [stored['variant'], 'sticky', stored]\n    if bucket_variant is None:\n        return [None, 'excluded', stored]\n    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('record from previous version is reassigned',\n   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('legacy record without version is reassigned',\n   [{'variant': 'b'}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('assignment sample 1',\n   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}]),\n  ('assignment sample 2',\n   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],\n   ['a', 'assigned', {'variant': 'a', 'version': 3}]),\n  ('assignment sample 3',\n   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}])],\n [('legacy record without version is reassigned',\n   [{'variant': 'b'}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('assignment sample 6',\n   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],\n   ['a', 'assigned', {'variant': 'a', 'version': 3}]),\n  ('assignment sample 10',\n   [{'variant': 'b'}, ['c', 'b', 'a'], 'c', 3],\n   ['c', 'assigned', {'variant': 'c', 'version': 3}]),\n  ('assignment sample 60',\n   [{'variant': 'c'}, ['a', 'c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('assignment sample 11',\n   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}]),\n  ('assignment sample 58',\n   [{'variant': 'c'}, ['c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('first assignment writes a record',\n   [None, ['a', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('assignment sample 16',\n   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],\n   ['c', 'assigned', {'variant': 'c', 'version': 3}]),\n  ('assignment sample 47',\n   [{'variant': 'b'}, ['c', 'b'], 'b', 3],\n   ['b', 'assigned', {'variant': 'b', 'version': 3}]),\n  ('assignment sample 60',\n   [{'variant': 'c'}, ['a', 'c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('first assignment writes a record',\n   [None, ['a', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),\n  ('assignment sample 7',\n   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],\n   ['c', 'assigned', {'variant': 'c', 'version': 1}]),\n  ('assignment sample 21',\n   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],\n   ['c', 'sticky', {'variant': 'c', 'version': 2}]),\n  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"e81cb1cab2b52ea34981986745d6f210460f87ae74d2cd2bdf8d344104eebb55","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(stored, variants, bucket_variant, version):\n    if stored and stored.get('variant') in variants:\n        return [stored['variant'], 'sticky', stored]\n    if bucket_variant is None:\n        return [None, 'excluded', stored]\n    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('record from previous version is reassigned',\n   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('legacy record without version is reassigned',\n   [{'variant': 'b'}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('assignment sample 1',\n   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}]),\n  ('assignment sample 2',\n   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],\n   ['a', 'assigned', {'variant': 'a', 'version': 3}]),\n  ('assignment sample 3',\n   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}])],\n [('legacy record without version is reassigned',\n   [{'variant': 'b'}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('assignment sample 6',\n   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],\n   ['a', 'assigned', {'variant': 'a', 'version': 3}]),\n  ('assignment sample 10',\n   [{'variant': 'b'}, ['c', 'b', 'a'], 'c', 3],\n   ['c', 'assigned', {'variant': 'c', 'version': 3}]),\n  ('assignment sample 60',\n   [{'variant': 'c'}, ['a', 'c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('assignment sample 11',\n   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}]),\n  ('assignment sample 58',\n   [{'variant': 'c'}, ['c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('first assignment writes a record',\n   [None, ['a', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('assignment sample 16',\n   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],\n   ['c', 'assigned', {'variant': 'c', 'version': 3}]),\n  ('assignment sample 47',\n   [{'variant': 'b'}, ['c', 'b'], 'b', 3],\n   ['b', 'assigned', {'variant': 'b', 'version': 3}]),\n  ('assignment sample 60',\n   [{'variant': 'c'}, ['a', 'c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('first assignment writes a record',\n   [None, ['a', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),\n  ('assignment sample 7',\n   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],\n   ['c', 'assigned', {'variant': 'c', 'version': 1}]),\n  ('assignment sample 21',\n   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],\n   ['c', 'sticky', {'variant': 'c', 'version': 2}]),\n  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"fixed":{"sha256":"7a2b204c0891068b0b07d5ac830bf29a14639bb4aa8203ec2fc8f4f6cd2ddb4e","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(stored, variants, bucket_variant, version):\n    if stored and stored.get('version') == version and stored.get('variant') in variants:\n        return [stored['variant'], 'sticky', stored]\n    if bucket_variant is None:\n        return [None, 'excluded', stored]\n    return [bucket_variant, 'assigned', {'variant': bucket_variant, 'version': version}]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('record from previous version is reassigned',\n   [{'variant': 'b', 'version': 1}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('legacy record without version is reassigned',\n   [{'variant': 'b'}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('assignment sample 1',\n   [{'variant': 'b', 'version': 3}, ['b', 'c', 'a'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}]),\n  ('assignment sample 2',\n   [{'variant': 'a', 'version': 2}, ['b', 'c', 'a'], 'a', 3],\n   ['a', 'assigned', {'variant': 'a', 'version': 3}]),\n  ('assignment sample 3',\n   [{'variant': 'a', 'version': 2}, ['a', 'c'], 'c', 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}])],\n [('legacy record without version is reassigned',\n   [{'variant': 'b'}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('assignment sample 6',\n   [{'variant': 'a', 'version': 4}, ['a'], 'a', 3],\n   ['a', 'assigned', {'variant': 'a', 'version': 3}]),\n  ('assignment sample 10',\n   [{'variant': 'b'}, ['c', 'b', 'a'], 'c', 3],\n   ['c', 'assigned', {'variant': 'c', 'version': 3}]),\n  ('assignment sample 60',\n   [{'variant': 'c'}, ['a', 'c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('retired variant is reassigned',\n   [{'variant': 'c', 'version': 2}, ['a', 'b'], 'b', 2],\n   ['b', 'assigned', {'variant': 'b', 'version': 2}]),\n  ('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('assignment sample 11',\n   [{'variant': 'a', 'version': 2}, ['a', 'c', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}]),\n  ('assignment sample 58',\n   [{'variant': 'c'}, ['c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('exclusion keeps the stored record',\n   [{'variant': 'c', 'version': 1}, ['a', 'b'], None, 2],\n   [None, 'excluded', {'variant': 'c', 'version': 1}]),\n  ('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('first assignment writes a record',\n   [None, ['a', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('assignment sample 16',\n   [{'variant': 'a'}, ['a', 'c', 'b'], 'c', 3],\n   ['c', 'assigned', {'variant': 'c', 'version': 3}]),\n  ('assignment sample 47',\n   [{'variant': 'b'}, ['c', 'b'], 'b', 3],\n   ['b', 'assigned', {'variant': 'b', 'version': 3}]),\n  ('assignment sample 60',\n   [{'variant': 'c'}, ['a', 'c'], 'c', 2],\n   ['c', 'assigned', {'variant': 'c', 'version': 2}])],\n [('record from a newer version is reassigned',\n   [{'variant': 'b', 'version': 3}, ['a', 'b'], 'a', 2],\n   ['a', 'assigned', {'variant': 'a', 'version': 2}]),\n  ('sticky beats traffic exclusion',\n   [{'variant': 'a', 'version': 2}, ['a', 'b'], None, 2],\n   ['a', 'sticky', {'variant': 'a', 'version': 2}]),\n  ('valid record stays sticky',\n   [{'variant': 'b', 'version': 2}, ['a', 'b'], 'a', 2],\n   ['b', 'sticky', {'variant': 'b', 'version': 2}]),\n  ('first assignment writes a record',\n   [None, ['a', 'b'], 'a', 1],\n   ['a', 'assigned', {'variant': 'a', 'version': 1}]),\n  ('excluded user without record', [None, ['a'], None, 1], [None, 'excluded', None]),\n  ('assignment sample 7',\n   [{'variant': 'c'}, ['b', 'c', 'a'], 'c', 1],\n   ['c', 'assigned', {'variant': 'c', 'version': 1}]),\n  ('assignment sample 21',\n   [{'variant': 'c', 'version': 2}, ['c', 'b', 'a'], 'c', 2],\n   ['c', 'sticky', {'variant': 'c', 'version': 2}]),\n  ('assignment sample 40', [{'variant': 'c'}, ['c', 'a'], None, 2], [None, 'excluded', {'variant': 'c'}])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w2-feature-flag-rollout-bucketing-sticky-assignment-version-check","generated_at":"2026-09-29T14:48:53.121537+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"Sticky bucketing keeps users on one arm across sessions; stale or retired records must not leak.","repair":"Honour a stored record only when its version equals the current version.","root_cause":"The stored version is never compared with the current experiment version.","sha256":"f653393e8bbaefbb6abd96b2498b35d3f828f544e91b431e90a33ea46caae909","title":"Sticky variant persistence: Records from older experiment versions stay sticky · case 01","variant":1,"variant_policy":"Five numbered records share a model and may reuse boundary fixtures.","verification":{"attempt":{"elapsed_ms":41.285,"exit_code":1,"observations":[{"actual":["a","assigned",{"variant":"a","version":2}],"check":"record from previous version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":true},{"actual":["b","sticky",{"variant":"b"}],"check":"legacy record without version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":false},{"actual":["b","assigned",{"variant":"b","version":2}],"check":"retired variant is reassigned","expected":["b","assigned",{"variant":"b","version":2}],"passed":true},{"actual":[null,"excluded",{"variant":"c","version":1}],"check":"exclusion keeps the stored record","expected":[null,"excluded",{"variant":"c","version":1}],"passed":true},{"actual":["a","assigned",{"variant":"a","version":2}],"check":"record from a newer version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":true},{"actual":["c","assigned",{"variant":"c","version":2}],"check":"assignment sample 1","expected":["c","assigned",{"variant":"c","version":2}],"passed":true},{"actual":["a","assigned",{"variant":"a","version":3}],"check":"assignment sample 2","expected":["a","assigned",{"variant":"a","version":3}],"passed":true},{"actual":["a","sticky",{"variant":"a","version":2}],"check":"assignment sample 3","expected":["a","sticky",{"variant":"a","version":2}],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"record from previous version is reassigned\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}, {\"check\": \"legacy record without version is reassigned\", \"actual\": [\"b\", \"sticky\", {\"variant\": \"b\"}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": false}, {\"check\": \"retired variant is reassigned\", \"actual\": [\"b\", \"assigned\", {\"variant\": \"b\", \"version\": 2}], \"expected\": [\"b\", \"assigned\", {\"variant\": \"b\", \"version\": 2}], \"passed\": true}, {\"check\": \"exclusion keeps the stored record\", \"actual\": [null, \"excluded\", {\"variant\": \"c\", \"version\": 1}], \"expected\": [null, \"excluded\", {\"variant\": \"c\", \"version\": 1}], \"passed\": true}, {\"check\": \"record from a newer version is reassigned\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}, {\"check\": \"assignment sample 1\", \"actual\": [\"c\", \"assigned\", {\"variant\": \"c\", \"version\": 2}], \"expected\": [\"c\", \"assigned\", {\"variant\": \"c\", \"version\": 2}], \"passed\": true}, {\"check\": \"assignment sample 2\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 3}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 3}], \"passed\": true}, {\"check\": \"assignment sample 3\", \"actual\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":39.512,"exit_code":1,"observations":[{"actual":["b","sticky",{"variant":"b","version":1}],"check":"record from previous version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":false},{"actual":["b","sticky",{"variant":"b"}],"check":"legacy record without version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":false},{"actual":["b","assigned",{"variant":"b","version":2}],"check":"retired variant is reassigned","expected":["b","assigned",{"variant":"b","version":2}],"passed":true},{"actual":[null,"excluded",{"variant":"c","version":1}],"check":"exclusion keeps the stored record","expected":[null,"excluded",{"variant":"c","version":1}],"passed":true},{"actual":["b","sticky",{"variant":"b","version":3}],"check":"record from a newer version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":false},{"actual":["b","sticky",{"variant":"b","version":3}],"check":"assignment sample 1","expected":["c","assigned",{"variant":"c","version":2}],"passed":false},{"actual":["a","sticky",{"variant":"a","version":2}],"check":"assignment sample 2","expected":["a","assigned",{"variant":"a","version":3}],"passed":false},{"actual":["a","sticky",{"variant":"a","version":2}],"check":"assignment sample 3","expected":["a","sticky",{"variant":"a","version":2}],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"record from previous version is reassigned\", \"actual\": [\"b\", \"sticky\", {\"variant\": \"b\", \"version\": 1}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": false}, {\"check\": \"legacy record without version is reassigned\", \"actual\": [\"b\", \"sticky\", {\"variant\": \"b\"}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": false}, {\"check\": \"retired variant is reassigned\", \"actual\": [\"b\", \"assigned\", {\"variant\": \"b\", \"version\": 2}], \"expected\": [\"b\", \"assigned\", {\"variant\": \"b\", \"version\": 2}], \"passed\": true}, {\"check\": \"exclusion keeps the stored record\", \"actual\": [null, \"excluded\", {\"variant\": \"c\", \"version\": 1}], \"expected\": [null, \"excluded\", {\"variant\": \"c\", \"version\": 1}], \"passed\": true}, {\"check\": \"record from a newer version is reassigned\", \"actual\": [\"b\", \"sticky\", {\"variant\": \"b\", \"version\": 3}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": false}, {\"check\": \"assignment sample 1\", \"actual\": [\"b\", \"sticky\", {\"variant\": \"b\", \"version\": 3}], \"expected\": [\"c\", \"assigned\", {\"variant\": \"c\", \"version\": 2}], \"passed\": false}, {\"check\": \"assignment sample 2\", \"actual\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 3}], \"passed\": false}, {\"check\": \"assignment sample 3\", \"actual\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}], \"passed\": false}\n"},"fixed":{"elapsed_ms":38.038,"exit_code":0,"observations":[{"actual":["a","assigned",{"variant":"a","version":2}],"check":"record from previous version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":true},{"actual":["a","assigned",{"variant":"a","version":2}],"check":"legacy record without version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":true},{"actual":["b","assigned",{"variant":"b","version":2}],"check":"retired variant is reassigned","expected":["b","assigned",{"variant":"b","version":2}],"passed":true},{"actual":[null,"excluded",{"variant":"c","version":1}],"check":"exclusion keeps the stored record","expected":[null,"excluded",{"variant":"c","version":1}],"passed":true},{"actual":["a","assigned",{"variant":"a","version":2}],"check":"record from a newer version is reassigned","expected":["a","assigned",{"variant":"a","version":2}],"passed":true},{"actual":["c","assigned",{"variant":"c","version":2}],"check":"assignment sample 1","expected":["c","assigned",{"variant":"c","version":2}],"passed":true},{"actual":["a","assigned",{"variant":"a","version":3}],"check":"assignment sample 2","expected":["a","assigned",{"variant":"a","version":3}],"passed":true},{"actual":["a","sticky",{"variant":"a","version":2}],"check":"assignment sample 3","expected":["a","sticky",{"variant":"a","version":2}],"passed":true}],"passed":true,"stderr":"","stdout":"{\"observations\": [{\"check\": \"record from previous version is reassigned\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}, {\"check\": \"legacy record without version is reassigned\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}, {\"check\": \"retired variant is reassigned\", \"actual\": [\"b\", \"assigned\", {\"variant\": \"b\", \"version\": 2}], \"expected\": [\"b\", \"assigned\", {\"variant\": \"b\", \"version\": 2}], \"passed\": true}, {\"check\": \"exclusion keeps the stored record\", \"actual\": [null, \"excluded\", {\"variant\": \"c\", \"version\": 1}], \"expected\": [null, \"excluded\", {\"variant\": \"c\", \"version\": 1}], \"passed\": true}, {\"check\": \"record from a newer version is reassigned\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}, {\"check\": \"assignment sample 1\", \"actual\": [\"c\", \"assigned\", {\"variant\": \"c\", \"version\": 2}], \"expected\": [\"c\", \"assigned\", {\"variant\": \"c\", \"version\": 2}], \"passed\": true}, {\"check\": \"assignment sample 2\", \"actual\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 3}], \"expected\": [\"a\", \"assigned\", {\"variant\": \"a\", \"version\": 3}], \"passed\": true}, {\"check\": \"assignment sample 3\", \"actual\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"expected\": [\"a\", \"sticky\", {\"variant\": \"a\", \"version\": 2}], \"passed\": true}], \"passed\": true}\n"}},"verified":true,"visibility":"public"}