{"abstract":"Clients go stale 30 seconds after startup even though they refetch continuously.","category":"Feature flag rollout bucketing","checks":8,"contract":"Process events in order. [\"fetch\", t, payload] replaces the whole cache with payload, records t and clears kill switches; [\"kill\", t, flag] forces that flag to its default; [\"get\", t, flag, default] returns [default, killed] for killed flags, else [default, stale] if nothing was fetched or t - last fetch >= 30 seconds, else [default, missing] if the flag is not cached, else [value, cache]. Return the list of get results.","evaluation_group":"w2-feature-flag-rollout-bucketing-cache-freshness","failed_approach":"Skipping the timestamp for empty payloads still lets a successful empty refresh go stale.","family":"w2-feature-flag-rollout-bucketing-cache-freshness-refresh-timestamp","id":"FA-74036","implementations":{"attempt":{"sha256":"f722119137b8cf89d9addb2fa6b5897c4742fc7b79aa40faa4c5af45a462ae89","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(events):\n    cache = {}\n    fetched_at = None\n    killed = set()\n    out = []\n    for ev in events:\n        kind, t = ev[0], ev[1]\n        if kind == 'fetch':\n            cache = dict(ev[2])\n            fetched_at = t if ev[2] else fetched_at\n            killed = set()\n        elif kind == 'kill':\n            killed.add(ev[2])\n        else:\n            flag, default = ev[2], ev[3]\n            if flag in killed:\n                out.append([default, 'killed'])\n            elif fetched_at is None or t - fetched_at >= 30:\n                out.append([default, 'stale'])\n            elif flag not in cache:\n                out.append([default, 'missing'])\n            else:\n                out.append([cache[flag], 'cache'])\n    return out\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('value is stale at exactly the ttl',\n   [[['fetch', 0, {'a': True}], ['get', 30, 'a', False]]],\n   [[False, 'stale']]),\n  ('fractional age just under the ttl is fresh',\n   [[['fetch', 0, {'a': True}], ['get', 29.5, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('second fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {'a': 2}], ['get', 40, 'a', 0]]],\n   [[2, 'cache']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 1',\n   [[['kill', 30, 'a'], ['fetch', 40, {'c': 3, 'b': False}], ['kill', 41, 'a']]],\n   [])],\n [('fractional age just under the ttl is fresh',\n   [[['fetch', 0, {'a': True}], ['get', 29.5, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 6',\n   [[['fetch', 5, {'a': 'b', 'c': 3}],\n     ['fetch', 6, {'c': True}],\n     ['get', 6.5, 'b', False],\n     ['get', 7.0, 'a', 'dflt'],\n     ['get', 17.0, 'a', False],\n     ['fetch', 27.0, {}],\n     ['get', 57.0, 'c', 'dflt'],\n     ['get', 87.0, 'b', False]]],\n   [[False, 'missing'], ['dflt', 'missing'], [False, 'missing'], ['dflt', 'stale'], [False, 'stale']]),\n  ('event stream sample 57',\n   [[['get', 1, 'a', 0],\n     ['get', 31, 'a', 'dflt'],\n     ['kill', 31.5, 'c'],\n     ['fetch', 32.5, {'c': False, 'b': True}],\n     ['get', 62.0, 'b', False],\n     ['fetch', 92.0, {}],\n     ['get', 97.0, 'c', 'dflt'],\n     ['get', 107.0, 'a', 0]]],\n   [[0, 'stale'], ['dflt', 'stale'], [True, 'cache'], ['dflt', 'missing'], [0, 'missing']])],\n [('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 11',\n   [[['get', 10, 'a', 0], ['get', 11, 'c', 'dflt'], ['get', 11.5, 'c', False]]],\n   [[0, 'stale'], ['dflt', 'stale'], [False, 'stale']]),\n  ('event stream sample 46',\n   [[['fetch', 5, {'b': False, 'c': False}],\n     ['fetch', 10, {'c': 'b', 'a': False}],\n     ['kill', 20, 'b'],\n     ['get', 30, 'a', 0],\n     ['get', 35, 'b', 'dflt'],\n     ['get', 35, 'a', False]]],\n   [[False, 'cache'], ['dflt', 'killed'], [False, 'cache']])],\n [('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('killed flag beats staleness',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['get', 40, 'a', False]]],\n   [[False, 'killed']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 16',\n   [[['get', 10, 'b', 0],\n     ['get', 40, 'c', 'dflt'],\n     ['get', 69.5, 'a', 0],\n     ['fetch', 79.5, {'a': 'b'}],\n     ['fetch', 84.5, {'c': 'b'}],\n     ['kill', 85.0, 'b'],\n     ['fetch', 95.0, {'b': 3, 'c': False}]]],\n   [[0, 'stale'], ['dflt', 'stale'], [0, 'stale']]),\n  ('event stream sample 44',\n   [[['kill', 0.5, 'a'],\n     ['get', 5.5, 'b', 'dflt'],\n     ['fetch', 15.5, {}],\n     ['fetch', 45.5, {}],\n     ['get', 75.0, 'b', 'dflt'],\n     ['get', 75.0, 'b', 0]]],\n   [['dflt', 'stale'], ['dflt', 'missing'], [0, 'missing']])],\n [('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('killed flag beats staleness',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['get', 40, 'a', False]]],\n   [[False, 'killed']]),\n  ('second fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {'a': 2}], ['get', 40, 'a', 0]]],\n   [[2, 'cache']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 21',\n   [[['get', 5, 'a', 0], ['fetch', 5, {'a': False, 'b': True}], ['get', 5, 'b', 0]]],\n   [[0, 'stale'], [True, 'cache']]),\n  ('event stream sample 38',\n   [[['get', 5, 'c', 0],\n     ['fetch', 34.5, {'c': True}],\n     ['kill', 44.5, 'b'],\n     ['kill', 54.5, 'a'],\n     ['fetch', 55.0, {'c': 3, 'b': True}],\n     ['kill', 55.5, 'c'],\n     ['fetch', 85.5, {}],\n     ['get', 95.5, 'a', 'dflt']]],\n   [[0, 'stale'], ['dflt', 'missing']])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"4247e95c0786e19ba120ed12cde4eb0294d0e1042dbd02da6b820df2b941434a","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(events):\n    cache = {}\n    fetched_at = None\n    killed = set()\n    out = []\n    for ev in events:\n        kind, t = ev[0], ev[1]\n        if kind == 'fetch':\n            cache = dict(ev[2])\n            if fetched_at is None:\n                fetched_at = t\n            killed = set()\n        elif kind == 'kill':\n            killed.add(ev[2])\n        else:\n            flag, default = ev[2], ev[3]\n            if flag in killed:\n                out.append([default, 'killed'])\n            elif fetched_at is None or t - fetched_at >= 30:\n                out.append([default, 'stale'])\n            elif flag not in cache:\n                out.append([default, 'missing'])\n            else:\n                out.append([cache[flag], 'cache'])\n    return out\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('value is stale at exactly the ttl',\n   [[['fetch', 0, {'a': True}], ['get', 30, 'a', False]]],\n   [[False, 'stale']]),\n  ('fractional age just under the ttl is fresh',\n   [[['fetch', 0, {'a': True}], ['get', 29.5, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('second fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {'a': 2}], ['get', 40, 'a', 0]]],\n   [[2, 'cache']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 1',\n   [[['kill', 30, 'a'], ['fetch', 40, {'c': 3, 'b': False}], ['kill', 41, 'a']]],\n   [])],\n [('fractional age just under the ttl is fresh',\n   [[['fetch', 0, {'a': True}], ['get', 29.5, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 6',\n   [[['fetch', 5, {'a': 'b', 'c': 3}],\n     ['fetch', 6, {'c': True}],\n     ['get', 6.5, 'b', False],\n     ['get', 7.0, 'a', 'dflt'],\n     ['get', 17.0, 'a', False],\n     ['fetch', 27.0, {}],\n     ['get', 57.0, 'c', 'dflt'],\n     ['get', 87.0, 'b', False]]],\n   [[False, 'missing'], ['dflt', 'missing'], [False, 'missing'], ['dflt', 'stale'], [False, 'stale']]),\n  ('event stream sample 57',\n   [[['get', 1, 'a', 0],\n     ['get', 31, 'a', 'dflt'],\n     ['kill', 31.5, 'c'],\n     ['fetch', 32.5, {'c': False, 'b': True}],\n     ['get', 62.0, 'b', False],\n     ['fetch', 92.0, {}],\n     ['get', 97.0, 'c', 'dflt'],\n     ['get', 107.0, 'a', 0]]],\n   [[0, 'stale'], ['dflt', 'stale'], [True, 'cache'], ['dflt', 'missing'], [0, 'missing']])],\n [('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 11',\n   [[['get', 10, 'a', 0], ['get', 11, 'c', 'dflt'], ['get', 11.5, 'c', False]]],\n   [[0, 'stale'], ['dflt', 'stale'], [False, 'stale']]),\n  ('event stream sample 46',\n   [[['fetch', 5, {'b': False, 'c': False}],\n     ['fetch', 10, {'c': 'b', 'a': False}],\n     ['kill', 20, 'b'],\n     ['get', 30, 'a', 0],\n     ['get', 35, 'b', 'dflt'],\n     ['get', 35, 'a', False]]],\n   [[False, 'cache'], ['dflt', 'killed'], [False, 'cache']])],\n [('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('killed flag beats staleness',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['get', 40, 'a', False]]],\n   [[False, 'killed']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 16',\n   [[['get', 10, 'b', 0],\n     ['get', 40, 'c', 'dflt'],\n     ['get', 69.5, 'a', 0],\n     ['fetch', 79.5, {'a': 'b'}],\n     ['fetch', 84.5, {'c': 'b'}],\n     ['kill', 85.0, 'b'],\n     ['fetch', 95.0, {'b': 3, 'c': False}]]],\n   [[0, 'stale'], ['dflt', 'stale'], [0, 'stale']]),\n  ('event stream sample 44',\n   [[['kill', 0.5, 'a'],\n     ['get', 5.5, 'b', 'dflt'],\n     ['fetch', 15.5, {}],\n     ['fetch', 45.5, {}],\n     ['get', 75.0, 'b', 'dflt'],\n     ['get', 75.0, 'b', 0]]],\n   [['dflt', 'stale'], ['dflt', 'missing'], [0, 'missing']])],\n [('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('killed flag beats staleness',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['get', 40, 'a', False]]],\n   [[False, 'killed']]),\n  ('second fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {'a': 2}], ['get', 40, 'a', 0]]],\n   [[2, 'cache']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 21',\n   [[['get', 5, 'a', 0], ['fetch', 5, {'a': False, 'b': True}], ['get', 5, 'b', 0]]],\n   [[0, 'stale'], [True, 'cache']]),\n  ('event stream sample 38',\n   [[['get', 5, 'c', 0],\n     ['fetch', 34.5, {'c': True}],\n     ['kill', 44.5, 'b'],\n     ['kill', 54.5, 'a'],\n     ['fetch', 55.0, {'c': 3, 'b': True}],\n     ['kill', 55.5, 'c'],\n     ['fetch', 85.5, {}],\n     ['get', 95.5, 'a', 'dflt']]],\n   [[0, 'stale'], ['dflt', 'missing']])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"fixed":{"sha256":"b120078615e571b6141f78c91e6589402d0ebfda0df92127ce4618b0e37865d0","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(events):\n    cache = {}\n    fetched_at = None\n    killed = set()\n    out = []\n    for ev in events:\n        kind, t = ev[0], ev[1]\n        if kind == 'fetch':\n            cache = dict(ev[2])\n            fetched_at = t\n            killed = set()\n        elif kind == 'kill':\n            killed.add(ev[2])\n        else:\n            flag, default = ev[2], ev[3]\n            if flag in killed:\n                out.append([default, 'killed'])\n            elif fetched_at is None or t - fetched_at >= 30:\n                out.append([default, 'stale'])\n            elif flag not in cache:\n                out.append([default, 'missing'])\n            else:\n                out.append([cache[flag], 'cache'])\n    return out\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('value is stale at exactly the ttl',\n   [[['fetch', 0, {'a': True}], ['get', 30, 'a', False]]],\n   [[False, 'stale']]),\n  ('fractional age just under the ttl is fresh',\n   [[['fetch', 0, {'a': True}], ['get', 29.5, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('second fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {'a': 2}], ['get', 40, 'a', 0]]],\n   [[2, 'cache']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 1',\n   [[['kill', 30, 'a'], ['fetch', 40, {'c': 3, 'b': False}], ['kill', 41, 'a']]],\n   [])],\n [('fractional age just under the ttl is fresh',\n   [[['fetch', 0, {'a': True}], ['get', 29.5, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 6',\n   [[['fetch', 5, {'a': 'b', 'c': 3}],\n     ['fetch', 6, {'c': True}],\n     ['get', 6.5, 'b', False],\n     ['get', 7.0, 'a', 'dflt'],\n     ['get', 17.0, 'a', False],\n     ['fetch', 27.0, {}],\n     ['get', 57.0, 'c', 'dflt'],\n     ['get', 87.0, 'b', False]]],\n   [[False, 'missing'], ['dflt', 'missing'], [False, 'missing'], ['dflt', 'stale'], [False, 'stale']]),\n  ('event stream sample 57',\n   [[['get', 1, 'a', 0],\n     ['get', 31, 'a', 'dflt'],\n     ['kill', 31.5, 'c'],\n     ['fetch', 32.5, {'c': False, 'b': True}],\n     ['get', 62.0, 'b', False],\n     ['fetch', 92.0, {}],\n     ['get', 97.0, 'c', 'dflt'],\n     ['get', 107.0, 'a', 0]]],\n   [[0, 'stale'], ['dflt', 'stale'], [True, 'cache'], ['dflt', 'missing'], [0, 'missing']])],\n [('refetch removes flags absent from payload',\n   [[['fetch', 0, {'a': True, 'b': 'x'}], ['fetch', 5, {'a': False}], ['get', 6, 'b', 'd']]],\n   [['d', 'missing']]),\n  ('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 11',\n   [[['get', 10, 'a', 0], ['get', 11, 'c', 'dflt'], ['get', 11.5, 'c', False]]],\n   [[0, 'stale'], ['dflt', 'stale'], [False, 'stale']]),\n  ('event stream sample 46',\n   [[['fetch', 5, {'b': False, 'c': False}],\n     ['fetch', 10, {'c': 'b', 'a': False}],\n     ['kill', 20, 'b'],\n     ['get', 30, 'a', 0],\n     ['get', 35, 'b', 'dflt'],\n     ['get', 35, 'a', False]]],\n   [[False, 'cache'], ['dflt', 'killed'], [False, 'cache']])],\n [('empty payload clears the cache',\n   [[['fetch', 0, {'a': True}], ['fetch', 5, {}], ['get', 6, 'a', False]]],\n   [[False, 'missing']]),\n  ('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('killed flag beats staleness',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['get', 40, 'a', False]]],\n   [[False, 'killed']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 16',\n   [[['get', 10, 'b', 0],\n     ['get', 40, 'c', 'dflt'],\n     ['get', 69.5, 'a', 0],\n     ['fetch', 79.5, {'a': 'b'}],\n     ['fetch', 84.5, {'c': 'b'}],\n     ['kill', 85.0, 'b'],\n     ['fetch', 95.0, {'b': 3, 'c': False}]]],\n   [[0, 'stale'], ['dflt', 'stale'], [0, 'stale']]),\n  ('event stream sample 44',\n   [[['kill', 0.5, 'a'],\n     ['get', 5.5, 'b', 'dflt'],\n     ['fetch', 15.5, {}],\n     ['fetch', 45.5, {}],\n     ['get', 75.0, 'b', 'dflt'],\n     ['get', 75.0, 'b', 0]]],\n   [['dflt', 'stale'], ['dflt', 'missing'], [0, 'missing']])],\n [('refetch clears kill switches',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['fetch', 2, {'a': True}], ['get', 3, 'a', False]]],\n   [[True, 'cache']]),\n  ('refetch clears a kill on an uncached flag',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['fetch', 2, {'a': True}], ['get', 4, 'b', 0]]],\n   [[0, 'missing']]),\n  ('kill of a flag missing from the payload',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'b'], ['get', 2, 'b', 'd']]],\n   [['d', 'killed']]),\n  ('killed flag beats staleness',\n   [[['fetch', 0, {'a': True}], ['kill', 1, 'a'], ['get', 40, 'a', False]]],\n   [[False, 'killed']]),\n  ('second fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {'a': 2}], ['get', 40, 'a', 0]]],\n   [[2, 'cache']]),\n  ('empty fetch refreshes the ttl clock',\n   [[['fetch', 0, {'a': 1}], ['fetch', 25, {}], ['get', 40, 'a', 0]]],\n   [[0, 'missing']]),\n  ('event stream sample 21',\n   [[['get', 5, 'a', 0], ['fetch', 5, {'a': False, 'b': True}], ['get', 5, 'b', 0]]],\n   [[0, 'stale'], [True, 'cache']]),\n  ('event stream sample 38',\n   [[['get', 5, 'c', 0],\n     ['fetch', 34.5, {'c': True}],\n     ['kill', 44.5, 'b'],\n     ['kill', 54.5, 'a'],\n     ['fetch', 55.0, {'c': 3, 'b': True}],\n     ['kill', 55.5, 'c'],\n     ['fetch', 85.5, {}],\n     ['get', 95.5, 'a', 'dflt']]],\n   [[0, 'stale'], ['dflt', 'missing']])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w2-feature-flag-rollout-bucketing-cache-freshness-refresh-timestamp","generated_at":"2026-09-29T14:48:52.996655+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"Client SDKs serve cached flags offline; kill switches and staleness decide whether a bad flag keeps serving.","repair":"Record the time of every fetch.","root_cause":"fetched_at is assigned only when it is still None.","sha256":"9d84ed8fbf55e257ad0c106a1cf0a6e8bcd1b3fa56b9d88c88d9eb388ea75645","title":"Flag value cache freshness: Only the first fetch starts the ttl clock · case 01","variant":1,"variant_policy":"Five numbered records share a model and may reuse boundary fixtures.","verification":{"attempt":{"elapsed_ms":40.938,"exit_code":1,"observations":[{"actual":[[false,"stale"]],"check":"value is stale at exactly the ttl","expected":[[false,"stale"]],"passed":true},{"actual":[[true,"cache"]],"check":"fractional age just under the ttl is fresh","expected":[[true,"cache"]],"passed":true},{"actual":[["d","missing"]],"check":"refetch removes flags absent from payload","expected":[["d","missing"]],"passed":true},{"actual":[[false,"missing"]],"check":"empty payload clears the cache","expected":[[false,"missing"]],"passed":true},{"actual":[[true,"cache"]],"check":"refetch clears kill switches","expected":[[true,"cache"]],"passed":true},{"actual":[[2,"cache"]],"check":"second fetch refreshes the ttl clock","expected":[[2,"cache"]],"passed":true},{"actual":[[0,"stale"]],"check":"empty fetch refreshes the ttl clock","expected":[[0,"missing"]],"passed":false},{"actual":[],"check":"event stream sample 1","expected":[],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"value is stale at exactly the ttl\", \"actual\": [[false, \"stale\"]], \"expected\": [[false, \"stale\"]], \"passed\": true}, {\"check\": \"fractional age just under the ttl is fresh\", \"actual\": [[true, \"cache\"]], \"expected\": [[true, \"cache\"]], \"passed\": true}, {\"check\": \"refetch removes flags absent from payload\", \"actual\": [[\"d\", \"missing\"]], \"expected\": [[\"d\", \"missing\"]], \"passed\": true}, {\"check\": \"empty payload clears the cache\", \"actual\": [[false, \"missing\"]], \"expected\": [[false, \"missing\"]], \"passed\": true}, {\"check\": \"refetch clears kill switches\", \"actual\": [[true, \"cache\"]], \"expected\": [[true, \"cache\"]], \"passed\": true}, {\"check\": \"second fetch refreshes the ttl clock\", \"actual\": [[2, \"cache\"]], \"expected\": [[2, \"cache\"]], \"passed\": true}, {\"check\": \"empty fetch refreshes the ttl clock\", \"actual\": [[0, \"stale\"]], \"expected\": [[0, \"missing\"]], \"passed\": false}, {\"check\": \"event stream sample 1\", \"actual\": [], \"expected\": [], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":36.772,"exit_code":1,"observations":[{"actual":[[false,"stale"]],"check":"value is stale at exactly the ttl","expected":[[false,"stale"]],"passed":true},{"actual":[[true,"cache"]],"check":"fractional age just under the ttl is fresh","expected":[[true,"cache"]],"passed":true},{"actual":[["d","missing"]],"check":"refetch removes flags absent from payload","expected":[["d","missing"]],"passed":true},{"actual":[[false,"missing"]],"check":"empty payload clears the cache","expected":[[false,"missing"]],"passed":true},{"actual":[[true,"cache"]],"check":"refetch clears kill switches","expected":[[true,"cache"]],"passed":true},{"actual":[[0,"stale"]],"check":"second fetch refreshes the ttl clock","expected":[[2,"cache"]],"passed":false},{"actual":[[0,"stale"]],"check":"empty fetch refreshes the ttl clock","expected":[[0,"missing"]],"passed":false},{"actual":[],"check":"event stream sample 1","expected":[],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"value is stale at exactly the ttl\", \"actual\": [[false, \"stale\"]], \"expected\": [[false, \"stale\"]], \"passed\": true}, {\"check\": \"fractional age just under the ttl is fresh\", \"actual\": [[true, \"cache\"]], \"expected\": [[true, \"cache\"]], \"passed\": true}, {\"check\": \"refetch removes flags absent from payload\", \"actual\": [[\"d\", \"missing\"]], \"expected\": [[\"d\", \"missing\"]], \"passed\": true}, {\"check\": \"empty payload clears the cache\", \"actual\": [[false, \"missing\"]], \"expected\": [[false, \"missing\"]], \"passed\": true}, {\"check\": \"refetch clears kill switches\", \"actual\": [[true, \"cache\"]], \"expected\": [[true, \"cache\"]], \"passed\": true}, {\"check\": \"second fetch refreshes the ttl clock\", \"actual\": [[0, \"stale\"]], \"expected\": [[2, \"cache\"]], \"passed\": false}, {\"check\": \"empty fetch refreshes the ttl clock\", \"actual\": [[0, \"stale\"]], \"expected\": [[0, \"missing\"]], \"passed\": false}, {\"check\": \"event stream sample 1\", \"actual\": [], \"expected\": [], \"passed\": true}], \"passed\": false}\n"},"fixed":{"elapsed_ms":39.373,"exit_code":0,"observations":[{"actual":[[false,"stale"]],"check":"value is stale at exactly the ttl","expected":[[false,"stale"]],"passed":true},{"actual":[[true,"cache"]],"check":"fractional age just under the ttl is fresh","expected":[[true,"cache"]],"passed":true},{"actual":[["d","missing"]],"check":"refetch removes flags absent from payload","expected":[["d","missing"]],"passed":true},{"actual":[[false,"missing"]],"check":"empty payload clears the cache","expected":[[false,"missing"]],"passed":true},{"actual":[[true,"cache"]],"check":"refetch clears kill switches","expected":[[true,"cache"]],"passed":true},{"actual":[[2,"cache"]],"check":"second fetch refreshes the ttl clock","expected":[[2,"cache"]],"passed":true},{"actual":[[0,"missing"]],"check":"empty fetch refreshes the ttl clock","expected":[[0,"missing"]],"passed":true},{"actual":[],"check":"event stream sample 1","expected":[],"passed":true}],"passed":true,"stderr":"","stdout":"{\"observations\": [{\"check\": \"value is stale at exactly the ttl\", \"actual\": [[false, \"stale\"]], \"expected\": [[false, \"stale\"]], \"passed\": true}, {\"check\": \"fractional age just under the ttl is fresh\", \"actual\": [[true, \"cache\"]], \"expected\": [[true, \"cache\"]], \"passed\": true}, {\"check\": \"refetch removes flags absent from payload\", \"actual\": [[\"d\", \"missing\"]], \"expected\": [[\"d\", \"missing\"]], \"passed\": true}, {\"check\": \"empty payload clears the cache\", \"actual\": [[false, \"missing\"]], \"expected\": [[false, \"missing\"]], \"passed\": true}, {\"check\": \"refetch clears kill switches\", \"actual\": [[true, \"cache\"]], \"expected\": [[true, \"cache\"]], \"passed\": true}, {\"check\": \"second fetch refreshes the ttl clock\", \"actual\": [[2, \"cache\"]], \"expected\": [[2, \"cache\"]], \"passed\": true}, {\"check\": \"empty fetch refreshes the ttl clock\", \"actual\": [[0, \"missing\"]], \"expected\": [[0, \"missing\"]], \"passed\": true}, {\"check\": \"event stream sample 1\", \"actual\": [], \"expected\": [], \"passed\": true}], \"passed\": true}\n"}},"verified":true,"visibility":"public"}