FAILURE MAP
← Case archive

FA-73896 / Feature flag rollout bucketing / Open access

Flag targeting precedence: Negated clauses match users lacking the attribute · case 01

Anonymous users without a plan attribute are served the "not pro" variation.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

A missing attribute is read as None, which fails membership and is then inverted by negate.

THE FAILURE

A missing attribute is read as None, which fails membership and is then inverted by negate.

Unsuccessful approach: Defaulting the missing attribute to an empty string still fails membership and is still negated.

Case contract

An off flag serves its off variation (reason OFF). Otherwise individual targets (checked by sorted variation name) win with reason TARGET, then rules in order, then the fallthrough. A rule matches only when the user has its attribute: op in = membership, startsWith = case-sensitive string prefix, lt = numeric less-than any value; negate inverts a present attribute result. Return [variation, reason, zero-based rule index or -1].

Why this case matters

Evaluation order decides whether QA allowlists, segment rules and defaults behave predictably.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(flag, user):
    if not flag.get('on'):
        return [flag.get('off'), 'OFF', -1]
    key = user.get('key')
    for variation in sorted(flag.get('targets', {})):
        if key in flag['targets'][variation]:
            return [variation, 'TARGET', -1]
    for index, rule in enumerate(flag.get('rules', [])):
        value = user.get(rule['attr'])
        if rule['op'] == 'in':
            hit = value in rule['values']
        elif rule['op'] == 'startsWith':
            hit = isinstance(value, str) and any(value.startswith(v) for v in rule['values'])
        else:
            hit = isinstance(value, (int, float)) and not isinstance(value, bool) and any(value < v for v in rule['values'])
        if rule.get('negate'):
            hit = not hit
        if hit:
            return [rule['variation'], 'RULE', index]
    return [flag.get('fallthrough'), 'FALLTHROUGH', -1]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('individual target beats a matching rule',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': ['k1']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k1', 'country': 'US'}],
   ['v1', 'TARGET', -1]),
  ('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 1',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': []},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'},
               {'attr': 'age', 'op': 'lt', 'values': [21, 18], 'negate': False, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'}]},
    {'key': 'k2', 'country': 'DE', 'age': 20, 'plan': 'team'}],
   ['v0', 'RULE', 1]),
  ('targeting sample 2',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v2': []}, 'rules': []},
    {'key': 'k6', 'age': 40, 'plan': 'pro'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 3',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v1': ['k3'], 'v3': ['k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k1', 'country': 'US', 'email': 'ann@x.io', 'age': 40, 'plan': 'pro'}],
   ['v1', 'RULE', 0])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('targeting sample 6',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 65], 'negate': False, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k5', 'country': 'DE', 'plan': 'team'}],
   ['v2', 'RULE', 1]),
  ('targeting sample 7',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['FR', 'US'], 'negate': False, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 17, 'plan': 'free'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 57',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v1': ['k2', 'k3'], 'v2': ['k5']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k4', 'email': 'bob@y.io'}],
   ['v2', 'FALLTHROUGH', -1])],
 [('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('targeting sample 11',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': ['k2'], 'v2': ['k4', 'k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'FR'], 'negate': True, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'email': 'annie', 'age': 40, 'plan': 'free'}],
   ['v1', 'RULE', 0]),
  ('targeting sample 12',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v1': ['k1', 'k4'], 'v2': []}, 'rules': []},
    {'key': 'k1', 'age': 30, 'plan': 'free'}],
   ['v1', 'TARGET', -1]),
  ('targeting sample 36',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': [], 'v2': ['k2', 'k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'qa-'],
                'negate': True,
                'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['pro', 'free'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k1', 'country': 'US', 'email': 'qa-1@z.io', 'age': 70}],
   ['v0', 'FALLTHROUGH', -1])],
 [('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('targeting sample 15',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v2',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': True,
                'variation': 'v0'},
               {'attr': 'plan', 'op': 'in', 'values': ['free', 'US'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k5', 'country': 'DE', 'age': 17}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 16',
   [{'on': True, 'off': 'v0', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k5', 'age': 40, 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 17',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['ann', 'a'],
                'negate': True,
                'variation': 'v0'},
               {'attr': 'plan', 'op': 'lt', 'values': [30, 18], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'ann@x.io', 'plan': 'team'}],
   ['v3', 'FALLTHROUGH', -1])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix operand order matters',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['annie@corp'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k2', 'email': 'annie'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 21',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k3', 'email': 'qa-1@z.io', 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 22',
   [{'on': False, 'off': 'v0', 'fallthrough': 'v2', 'targets': {'v1': [], 'v3': ['k3']}, 'rules': []},
    {'key': 'k6', 'country': 'DE', 'email': 'Ann@x.io', 'plan': 'team'}],
   ['v0', 'OFF', -1])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
individual target beats a matching rule['v1', 'TARGET', -1]['v1', 'TARGET', -1]Passed
negated rule does not match a missing attribute['v2', 'RULE', 0]['v3', 'FALLTHROUGH', -1]Failed
off flag serves off variation['v1', 'OFF', -1]['v1', 'OFF', -1]Passed
second rule reports index one['v2', 'RULE', 1]['v2', 'RULE', 1]Passed
prefix match is case sensitive['v0', 'FALLTHROUGH', -1]['v0', 'FALLTHROUGH', -1]Passed
targeting sample 1['v0', 'RULE', 1]['v0', 'RULE', 1]Passed
targeting sample 2['v0', 'FALLTHROUGH', -1]['v0', 'FALLTHROUGH', -1]Passed
targeting sample 3['v1', 'RULE', 0]['v1', 'RULE', 0]Passed

SHA-256 / e005bf42830eab741fd7984a5e88a7acd8761b32fc16500ef6c231385a240fe8

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(flag, user):
    if not flag.get('on'):
        return [flag.get('off'), 'OFF', -1]
    key = user.get('key')
    for variation in sorted(flag.get('targets', {})):
        if key in flag['targets'][variation]:
            return [variation, 'TARGET', -1]
    for index, rule in enumerate(flag.get('rules', [])):
        value = user.get(rule['attr'], '')
        if rule['op'] == 'in':
            hit = value in rule['values']
        elif rule['op'] == 'startsWith':
            hit = isinstance(value, str) and any(value.startswith(v) for v in rule['values'])
        else:
            hit = isinstance(value, (int, float)) and not isinstance(value, bool) and any(value < v for v in rule['values'])
        if rule.get('negate'):
            hit = not hit
        if hit:
            return [rule['variation'], 'RULE', index]
    return [flag.get('fallthrough'), 'FALLTHROUGH', -1]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('individual target beats a matching rule',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': ['k1']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k1', 'country': 'US'}],
   ['v1', 'TARGET', -1]),
  ('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 1',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': []},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'},
               {'attr': 'age', 'op': 'lt', 'values': [21, 18], 'negate': False, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'}]},
    {'key': 'k2', 'country': 'DE', 'age': 20, 'plan': 'team'}],
   ['v0', 'RULE', 1]),
  ('targeting sample 2',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v2': []}, 'rules': []},
    {'key': 'k6', 'age': 40, 'plan': 'pro'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 3',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v1': ['k3'], 'v3': ['k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k1', 'country': 'US', 'email': 'ann@x.io', 'age': 40, 'plan': 'pro'}],
   ['v1', 'RULE', 0])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('targeting sample 6',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 65], 'negate': False, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k5', 'country': 'DE', 'plan': 'team'}],
   ['v2', 'RULE', 1]),
  ('targeting sample 7',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['FR', 'US'], 'negate': False, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 17, 'plan': 'free'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 57',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v1': ['k2', 'k3'], 'v2': ['k5']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k4', 'email': 'bob@y.io'}],
   ['v2', 'FALLTHROUGH', -1])],
 [('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('targeting sample 11',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': ['k2'], 'v2': ['k4', 'k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'FR'], 'negate': True, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'email': 'annie', 'age': 40, 'plan': 'free'}],
   ['v1', 'RULE', 0]),
  ('targeting sample 12',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v1': ['k1', 'k4'], 'v2': []}, 'rules': []},
    {'key': 'k1', 'age': 30, 'plan': 'free'}],
   ['v1', 'TARGET', -1]),
  ('targeting sample 36',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': [], 'v2': ['k2', 'k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'qa-'],
                'negate': True,
                'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['pro', 'free'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k1', 'country': 'US', 'email': 'qa-1@z.io', 'age': 70}],
   ['v0', 'FALLTHROUGH', -1])],
 [('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('targeting sample 15',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v2',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': True,
                'variation': 'v0'},
               {'attr': 'plan', 'op': 'in', 'values': ['free', 'US'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k5', 'country': 'DE', 'age': 17}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 16',
   [{'on': True, 'off': 'v0', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k5', 'age': 40, 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 17',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['ann', 'a'],
                'negate': True,
                'variation': 'v0'},
               {'attr': 'plan', 'op': 'lt', 'values': [30, 18], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'ann@x.io', 'plan': 'team'}],
   ['v3', 'FALLTHROUGH', -1])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix operand order matters',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['annie@corp'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k2', 'email': 'annie'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 21',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k3', 'email': 'qa-1@z.io', 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 22',
   [{'on': False, 'off': 'v0', 'fallthrough': 'v2', 'targets': {'v1': [], 'v3': ['k3']}, 'rules': []},
    {'key': 'k6', 'country': 'DE', 'email': 'Ann@x.io', 'plan': 'team'}],
   ['v0', 'OFF', -1])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
individual target beats a matching rule['v1', 'TARGET', -1]['v1', 'TARGET', -1]Passed
negated rule does not match a missing attribute['v2', 'RULE', 0]['v3', 'FALLTHROUGH', -1]Failed
off flag serves off variation['v1', 'OFF', -1]['v1', 'OFF', -1]Passed
second rule reports index one['v2', 'RULE', 1]['v2', 'RULE', 1]Passed
prefix match is case sensitive['v0', 'FALLTHROUGH', -1]['v0', 'FALLTHROUGH', -1]Passed
targeting sample 1['v0', 'RULE', 1]['v0', 'RULE', 1]Passed
targeting sample 2['v0', 'FALLTHROUGH', -1]['v0', 'FALLTHROUGH', -1]Passed
targeting sample 3['v1', 'RULE', 0]['v1', 'RULE', 0]Passed

SHA-256 / a4d693a320a06293c838c0ca8adb99c04e927a1cf4679e72343bdd06eb669f63

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 8 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Sign in to the archive ↗

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:51.757940+00:00.

Case digest / 564b8f767e0b9e2ea0915cd053949056482bffe91e2e90b6497ce7d4c24221e0