FA-73901 / Feature flag rollout bucketing / Open access

Flag targeting precedence: A disabled flag serves its fallthrough · case 01

Turning the flag off leaves users on the rollout variation instead of the safe off variation.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The off branch returns flag["fallthrough"] while still labelling the reason OFF.

VERIFIED REPAIR

Return the configured off variation when the flag is off.

Unsuccessful approach: Returning None when off ignores the configured off variation.

Case contract

An off flag serves its off variation (reason OFF). Otherwise individual targets (checked by sorted variation name) win with reason TARGET, then rules in order, then the fallthrough. A rule matches only when the user has its attribute: op in = membership, startsWith = case-sensitive string prefix, lt = numeric less-than any value; negate inverts a present attribute result. Return [variation, reason, zero-based rule index or -1].

Why this case matters

Evaluation order decides whether QA allowlists, segment rules and defaults behave predictably.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(flag, user):
    if not flag.get('on'):
        return [flag.get('fallthrough'), 'OFF', -1]
    key = user.get('key')
    for variation in sorted(flag.get('targets', {})):
        if key in flag['targets'][variation]:
            return [variation, 'TARGET', -1]
    for index, rule in enumerate(flag.get('rules', [])):
        if rule['attr'] not in user:
            continue
        value = user[rule['attr']]
        if rule['op'] == 'in':
            hit = value in rule['values']
        elif rule['op'] == 'startsWith':
            hit = isinstance(value, str) and any(value.startswith(v) for v in rule['values'])
        else:
            hit = isinstance(value, (int, float)) and not isinstance(value, bool) and any(value < v for v in rule['values'])
        if rule.get('negate'):
            hit = not hit
        if hit:
            return [rule['variation'], 'RULE', index]
    return [flag.get('fallthrough'), 'FALLTHROUGH', -1]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('individual target beats a matching rule',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': ['k1']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k1', 'country': 'US'}],
   ['v1', 'TARGET', -1]),
  ('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 1',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': []},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'},
               {'attr': 'age', 'op': 'lt', 'values': [21, 18], 'negate': False, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'}]},
    {'key': 'k2', 'country': 'DE', 'age': 20, 'plan': 'team'}],
   ['v0', 'RULE', 1]),
  ('targeting sample 2',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v2': []}, 'rules': []},
    {'key': 'k6', 'age': 40, 'plan': 'pro'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 3',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v1': ['k3'], 'v3': ['k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k1', 'country': 'US', 'email': 'ann@x.io', 'age': 40, 'plan': 'pro'}],
   ['v1', 'RULE', 0])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('targeting sample 6',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 65], 'negate': False, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k5', 'country': 'DE', 'plan': 'team'}],
   ['v2', 'RULE', 1]),
  ('targeting sample 7',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['FR', 'US'], 'negate': False, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 17, 'plan': 'free'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 32',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': [], 'v1': ['k4']},
     'rules': [{'attr': 'plan', 'op': 'lt', 'values': [21, 65], 'negate': True, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v0'},
               {'attr': 'country', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v2'},
               {'attr': 'plan', 'op': 'lt', 'values': [21, 65], 'negate': True, 'variation': 'v1'}]},
    {'key': 'k5', 'plan': 'pro'}],
   ['v1', 'OFF', -1])],
 [('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('targeting sample 9',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v3': ['k1', 'k4'], 'v1': ['k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['bob', 'ann'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['bob', 'ann'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'ann@x.io', 'age': 70, 'plan': 'team'}],
   ['v1', 'OFF', -1]),
  ('targeting sample 10',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v3': ['k5'], 'v1': ['k2', 'k3']},
     'rules': []},
    {'key': 'k2', 'country': 'US', 'email': 'Ann@x.io'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 11',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': ['k2'], 'v2': ['k4', 'k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'FR'], 'negate': True, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'email': 'annie', 'age': 40, 'plan': 'free'}],
   ['v1', 'RULE', 0])],
 [('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('targeting sample 16',
   [{'on': True, 'off': 'v0', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k5', 'age': 40, 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 40',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1', 'k2'], 'v1': ['k1', 'k2']},
     'rules': []},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 40, 'plan': 'pro'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 42',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v1': []},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v3'}]},
    {'key': 'k5', 'email': 'bob@y.io', 'age': 20, 'plan': 'team'}],
   ['v0', 'OFF', -1])],
 [('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix operand order matters',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['annie@corp'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k2', 'email': 'annie'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 10',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v3': ['k5'], 'v1': ['k2', 'k3']},
     'rules': []},
    {'key': 'k2', 'country': 'US', 'email': 'Ann@x.io'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 21',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k3', 'email': 'qa-1@z.io', 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 25',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v1': []},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': False, 'variation': 'v2'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'qa-1@z.io', 'age': 17}],
   ['v1', 'OFF', -1])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
individual target beats a matching rule	`['v1', 'TARGET', -1]`	`['v1', 'TARGET', -1]`	Passed
negated rule does not match a missing attribute	`['v3', 'FALLTHROUGH', -1]`	`['v3', 'FALLTHROUGH', -1]`	Passed
off flag serves off variation	`['v2', 'OFF', -1]`	`['v1', 'OFF', -1]`	Failed
second rule reports index one	`['v2', 'RULE', 1]`	`['v2', 'RULE', 1]`	Passed
prefix match is case sensitive	`['v0', 'FALLTHROUGH', -1]`	`['v0', 'FALLTHROUGH', -1]`	Passed
targeting sample 1	`['v0', 'RULE', 1]`	`['v0', 'RULE', 1]`	Passed
targeting sample 2	`['v0', 'FALLTHROUGH', -1]`	`['v0', 'FALLTHROUGH', -1]`	Passed
targeting sample 3	`['v1', 'RULE', 0]`	`['v1', 'RULE', 0]`	Passed

SHA-256 / 117c5c4c2633a0cf2a0fd88d9a5937897bd1d8c53d7175b205b4d98579425b48

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(flag, user):
    if not flag.get('on'):
        return [None, 'OFF', -1]
    key = user.get('key')
    for variation in sorted(flag.get('targets', {})):
        if key in flag['targets'][variation]:
            return [variation, 'TARGET', -1]
    for index, rule in enumerate(flag.get('rules', [])):
        if rule['attr'] not in user:
            continue
        value = user[rule['attr']]
        if rule['op'] == 'in':
            hit = value in rule['values']
        elif rule['op'] == 'startsWith':
            hit = isinstance(value, str) and any(value.startswith(v) for v in rule['values'])
        else:
            hit = isinstance(value, (int, float)) and not isinstance(value, bool) and any(value < v for v in rule['values'])
        if rule.get('negate'):
            hit = not hit
        if hit:
            return [rule['variation'], 'RULE', index]
    return [flag.get('fallthrough'), 'FALLTHROUGH', -1]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('individual target beats a matching rule',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': ['k1']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k1', 'country': 'US'}],
   ['v1', 'TARGET', -1]),
  ('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 1',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': []},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'},
               {'attr': 'age', 'op': 'lt', 'values': [21, 18], 'negate': False, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'}]},
    {'key': 'k2', 'country': 'DE', 'age': 20, 'plan': 'team'}],
   ['v0', 'RULE', 1]),
  ('targeting sample 2',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v2': []}, 'rules': []},
    {'key': 'k6', 'age': 40, 'plan': 'pro'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 3',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v1': ['k3'], 'v3': ['k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k1', 'country': 'US', 'email': 'ann@x.io', 'age': 40, 'plan': 'pro'}],
   ['v1', 'RULE', 0])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('targeting sample 6',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 65], 'negate': False, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k5', 'country': 'DE', 'plan': 'team'}],
   ['v2', 'RULE', 1]),
  ('targeting sample 7',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['FR', 'US'], 'negate': False, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 17, 'plan': 'free'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 32',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': [], 'v1': ['k4']},
     'rules': [{'attr': 'plan', 'op': 'lt', 'values': [21, 65], 'negate': True, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v0'},
               {'attr': 'country', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v2'},
               {'attr': 'plan', 'op': 'lt', 'values': [21, 65], 'negate': True, 'variation': 'v1'}]},
    {'key': 'k5', 'plan': 'pro'}],
   ['v1', 'OFF', -1])],
 [('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('targeting sample 9',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v3': ['k1', 'k4'], 'v1': ['k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['bob', 'ann'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['bob', 'ann'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'ann@x.io', 'age': 70, 'plan': 'team'}],
   ['v1', 'OFF', -1]),
  ('targeting sample 10',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v3': ['k5'], 'v1': ['k2', 'k3']},
     'rules': []},
    {'key': 'k2', 'country': 'US', 'email': 'Ann@x.io'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 11',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': ['k2'], 'v2': ['k4', 'k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'FR'], 'negate': True, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'email': 'annie', 'age': 40, 'plan': 'free'}],
   ['v1', 'RULE', 0])],
 [('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('targeting sample 16',
   [{'on': True, 'off': 'v0', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k5', 'age': 40, 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 40',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1', 'k2'], 'v1': ['k1', 'k2']},
     'rules': []},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 40, 'plan': 'pro'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 42',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v1': []},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v3'}]},
    {'key': 'k5', 'email': 'bob@y.io', 'age': 20, 'plan': 'team'}],
   ['v0', 'OFF', -1])],
 [('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix operand order matters',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['annie@corp'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k2', 'email': 'annie'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 10',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v3': ['k5'], 'v1': ['k2', 'k3']},
     'rules': []},
    {'key': 'k2', 'country': 'US', 'email': 'Ann@x.io'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 21',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k3', 'email': 'qa-1@z.io', 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 25',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v1': []},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': False, 'variation': 'v2'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'qa-1@z.io', 'age': 17}],
   ['v1', 'OFF', -1])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
individual target beats a matching rule	`['v1', 'TARGET', -1]`	`['v1', 'TARGET', -1]`	Passed
negated rule does not match a missing attribute	`['v3', 'FALLTHROUGH', -1]`	`['v3', 'FALLTHROUGH', -1]`	Passed
off flag serves off variation	`[None, 'OFF', -1]`	`['v1', 'OFF', -1]`	Failed
second rule reports index one	`['v2', 'RULE', 1]`	`['v2', 'RULE', 1]`	Passed
prefix match is case sensitive	`['v0', 'FALLTHROUGH', -1]`	`['v0', 'FALLTHROUGH', -1]`	Passed
targeting sample 1	`['v0', 'RULE', 1]`	`['v0', 'RULE', 1]`	Passed
targeting sample 2	`['v0', 'FALLTHROUGH', -1]`	`['v0', 'FALLTHROUGH', -1]`	Passed
targeting sample 3	`['v1', 'RULE', 0]`	`['v1', 'RULE', 0]`	Passed

SHA-256 / e37f18d1bffb181a6880fedd77742e8bbb65865190cd22d1a5cef44595586072

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(flag, user):
    if not flag.get('on'):
        return [flag.get('off'), 'OFF', -1]
    key = user.get('key')
    for variation in sorted(flag.get('targets', {})):
        if key in flag['targets'][variation]:
            return [variation, 'TARGET', -1]
    for index, rule in enumerate(flag.get('rules', [])):
        if rule['attr'] not in user:
            continue
        value = user[rule['attr']]
        if rule['op'] == 'in':
            hit = value in rule['values']
        elif rule['op'] == 'startsWith':
            hit = isinstance(value, str) and any(value.startswith(v) for v in rule['values'])
        else:
            hit = isinstance(value, (int, float)) and not isinstance(value, bool) and any(value < v for v in rule['values'])
        if rule.get('negate'):
            hit = not hit
        if hit:
            return [rule['variation'], 'RULE', index]
    return [flag.get('fallthrough'), 'FALLTHROUGH', -1]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('individual target beats a matching rule',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': ['k1']},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k1', 'country': 'US'}],
   ['v1', 'TARGET', -1]),
  ('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 1',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {'v1': []},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'},
               {'attr': 'age', 'op': 'lt', 'values': [21, 18], 'negate': False, 'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': True, 'variation': 'v2'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['a', 'Ann'],
                'negate': False,
                'variation': 'v2'}]},
    {'key': 'k2', 'country': 'DE', 'age': 20, 'plan': 'team'}],
   ['v0', 'RULE', 1]),
  ('targeting sample 2',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v0', 'targets': {'v2': []}, 'rules': []},
    {'key': 'k6', 'age': 40, 'plan': 'pro'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 3',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v1': ['k3'], 'v3': ['k3']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['Ann', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k1', 'country': 'US', 'email': 'ann@x.io', 'age': 40, 'plan': 'pro'}],
   ['v1', 'RULE', 0])],
 [('negated rule does not match a missing attribute',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k2'}],
   ['v3', 'FALLTHROUGH', -1]),
  ('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('targeting sample 6',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 65], 'negate': False, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v2'}]},
    {'key': 'k5', 'country': 'DE', 'plan': 'team'}],
   ['v2', 'RULE', 1]),
  ('targeting sample 7',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v1'},
               {'attr': 'plan', 'op': 'in', 'values': ['FR', 'US'], 'negate': False, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 17, 'plan': 'free'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 32',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': [], 'v1': ['k4']},
     'rules': [{'attr': 'plan', 'op': 'lt', 'values': [21, 65], 'negate': True, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v0'},
               {'attr': 'country', 'op': 'in', 'values': ['FR', 'free'], 'negate': True, 'variation': 'v2'},
               {'attr': 'plan', 'op': 'lt', 'values': [21, 65], 'negate': True, 'variation': 'v1'}]},
    {'key': 'k5', 'plan': 'pro'}],
   ['v1', 'OFF', -1])],
 [('off flag serves off variation',
   [{'on': False, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []}, {'key': 'k1'}],
   ['v1', 'OFF', -1]),
  ('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('targeting sample 9',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v0',
     'targets': {'v3': ['k1', 'k4'], 'v1': ['k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['bob', 'ann'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['bob', 'ann'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'ann@x.io', 'age': 70, 'plan': 'team'}],
   ['v1', 'OFF', -1]),
  ('targeting sample 10',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v3': ['k5'], 'v1': ['k2', 'k3']},
     'rules': []},
    {'key': 'k2', 'country': 'US', 'email': 'Ann@x.io'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 11',
   [{'on': True,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v3': ['k2'], 'v2': ['k4', 'k2']},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'FR'], 'negate': True, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k6', 'email': 'annie', 'age': 40, 'plan': 'free'}],
   ['v1', 'RULE', 0])],
 [('second rule reports index one',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['FR'], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['DE'], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k3', 'country': 'DE'}],
   ['v2', 'RULE', 1]),
  ('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('targeting sample 16',
   [{'on': True, 'off': 'v0', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k5', 'age': 40, 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 40',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1', 'k2'], 'v1': ['k1', 'k2']},
     'rules': []},
    {'key': 'k6', 'country': 'JP', 'email': 'annie', 'age': 40, 'plan': 'pro'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 42',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v1': []},
     'rules': [{'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v3'},
               {'attr': 'email',
                'op': 'startsWith',
                'values': ['qa-', 'a'],
                'negate': False,
                'variation': 'v0'},
               {'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': False, 'variation': 'v1'},
               {'attr': 'country', 'op': 'in', 'values': ['US', 'pro'], 'negate': False, 'variation': 'v3'}]},
    {'key': 'k5', 'email': 'bob@y.io', 'age': 20, 'plan': 'team'}],
   ['v0', 'OFF', -1])],
 [('prefix match is case sensitive',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email', 'op': 'startsWith', 'values': ['ann'], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k4', 'email': 'Ann@x.io'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('rule index skips rules on absent attributes',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'plan', 'op': 'in', 'values': ['pro'], 'negate': False, 'variation': 'v1'},
               {'attr': 'age', 'op': 'lt', 'values': [21], 'negate': False, 'variation': 'v2'}]},
    {'key': 'k5', 'age': 19}],
   ['v2', 'RULE', 1]),
  ('target to fallthrough variation still reports TARGET',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v2': ['k1']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [30], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 20}],
   ['v2', 'TARGET', -1]),
  ('no rule matches uses fallthrough',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v3',
     'targets': {'v1': ['k9']},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18], 'negate': False, 'variation': 'v1'}]},
    {'key': 'k1', 'age': 40}],
   ['v3', 'FALLTHROUGH', -1]),
  ('prefix operand order matters',
   [{'on': True,
     'off': 'v0',
     'fallthrough': 'v0',
     'targets': {},
     'rules': [{'attr': 'email',
                'op': 'startsWith',
                'values': ['annie@corp'],
                'negate': False,
                'variation': 'v1'}]},
    {'key': 'k2', 'email': 'annie'}],
   ['v0', 'FALLTHROUGH', -1]),
  ('targeting sample 10',
   [{'on': False,
     'off': 'v0',
     'fallthrough': 'v2',
     'targets': {'v3': ['k5'], 'v1': ['k2', 'k3']},
     'rules': []},
    {'key': 'k2', 'country': 'US', 'email': 'Ann@x.io'}],
   ['v0', 'OFF', -1]),
  ('targeting sample 21',
   [{'on': True, 'off': 'v1', 'fallthrough': 'v2', 'targets': {}, 'rules': []},
    {'key': 'k3', 'email': 'qa-1@z.io', 'plan': 'free'}],
   ['v2', 'FALLTHROUGH', -1]),
  ('targeting sample 25',
   [{'on': False,
     'off': 'v1',
     'fallthrough': 'v3',
     'targets': {'v1': []},
     'rules': [{'attr': 'age', 'op': 'lt', 'values': [18, 30], 'negate': False, 'variation': 'v2'},
               {'attr': 'country', 'op': 'in', 'values': ['DE', 'free'], 'negate': True, 'variation': 'v0'}]},
    {'key': 'k6', 'country': 'JP', 'email': 'qa-1@z.io', 'age': 17}],
   ['v1', 'OFF', -1])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
individual target beats a matching rule	`['v1', 'TARGET', -1]`	`['v1', 'TARGET', -1]`	Passed
negated rule does not match a missing attribute	`['v3', 'FALLTHROUGH', -1]`	`['v3', 'FALLTHROUGH', -1]`	Passed
off flag serves off variation	`['v1', 'OFF', -1]`	`['v1', 'OFF', -1]`	Passed
second rule reports index one	`['v2', 'RULE', 1]`	`['v2', 'RULE', 1]`	Passed
prefix match is case sensitive	`['v0', 'FALLTHROUGH', -1]`	`['v0', 'FALLTHROUGH', -1]`	Passed
targeting sample 1	`['v0', 'RULE', 1]`	`['v0', 'RULE', 1]`	Passed
targeting sample 2	`['v0', 'FALLTHROUGH', -1]`	`['v0', 'FALLTHROUGH', -1]`	Passed
targeting sample 3	`['v1', 'RULE', 0]`	`['v1', 'RULE', 0]`	Passed

SHA-256 / 4cd4e659f1402b8a3d1690ae79fc7b85b3d036ffd8529fb8a8b17cefbe86915e

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:51.851622+00:00.

Case digest / 52f2ce3ab1d39ff08efb41ec501022dace42b8f42c7e352cc71e1a79169f9462