FAILURE MAP
← Case archive

FA-74106 / Feature flag rollout bucketing / Open access

Semantic version clause: Numeric prerelease identifiers compare as text · case 01

beta.11 is treated as older than beta.2, so clients on the newer prerelease miss the feature.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Digit-only identifiers are keyed by their string, so comparison is lexical.

VERIFIED REPAIR

Key digit-only identifiers by their integer value and rank them below alphanumeric identifiers.

Unsuccessful approach: Keying numbers by value but alphanumerics by zero ranks numbers above words.

Case contract

Versions are MAJOR[.MINOR[.PATCH]][-prerelease][+build]; missing components are 0, build metadata is ignored, anything else is invalid (return None). A release outranks its prereleases; prerelease identifiers compare numerically when all digits, lexically otherwise, numeric below alphanumeric, and a shorter identifier list ranks lower when it is a prefix. op is semVerLessThan, semVerEqual or semVerGreaterThan; unknown ops return None.

Why this case matters

Targeting by app version gates features to clients that can render them.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(version, op, target):
    def parse(s):
        s = s.split('+', 1)[0]
        core, _, pre = s.partition('-')
        parts = core.split('.')
        if not 1 <= len(parts) <= 3 or not all(p.isdigit() for p in parts):
            return None
        nums = [int(p) for p in parts] + [0] * (3 - len(parts))
        ids = pre.split('.') if pre else []
        return nums, ids
    def key_id(i):
        return (0, 0, i) if i.isdigit() else (1, 0, i)
    def cmp(a, b):
        if a[0] != b[0]:
            return -1 if a[0] < b[0] else 1
        x, y = a[1], b[1]
        if not x or not y:
            return (len(x) == 0) - (len(y) == 0) if (x or y) else 0
        for p, q in zip(x, y):
            kp, kq = key_id(p), key_id(q)
            if kp != kq:
                return -1 if kp < kq else 1
        return (len(x) > len(y)) - (len(x) < len(y))
    a, b = parse(version), parse(target)
    if a is None or b is None:
        return None
    c = cmp(a, b)
    return {'semVerLessThan': c < 0, 'semVerEqual': c == 0, 'semVerGreaterThan': c > 0}.get(op)
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('version pair sample 1', ['1.0.0-beta.11', 'semVerEqual', '1.0.0-alpha'], False),
  ('version pair sample 2', ['1.0.0-alpha', 'semVerEqual', '1.0.0'], False),
  ('version pair sample 3', ['1.0.0-alpha.10', 'semVerLessThan', '1.0.0+build.5'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('version pair sample 6', ['1.0.0.0', 'semVerEqual', '2.1'], None),
  ('version pair sample 43', ['1.0.0-beta.2', 'semVerGreaterThan', '1.0.0-1'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('version pair sample 11', ['1.0.0+build.5', 'semVerLessThan', '1.0.0.0'], None),
  ('version pair sample 34', ['1.0.0-2', 'semVerLessThan', '1.0.0-beta.2'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('minor ten beats minor nine', ['2.10.0', 'semVerGreaterThan', '2.9.0'], True),
  ('version pair sample 16', ['1.0.0-alpha.10', 'semVerEqual', '1.0.0'], False),
  ('version pair sample 20', ['1.0.0-rc.1+b7', 'semVerLessThan', '1.0.0-2'], False)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('minor ten beats minor nine', ['2.10.0', 'semVerGreaterThan', '2.9.0'], True),
  ('four components are invalid', ['1.0.0.0', 'semVerEqual', '1.0.0'], None),
  ('version pair sample 21', ['2.1.0', 'semVerGreaterThan', '2.1'], False)]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
numeric prerelease identifiers compare as numbersFalseTrueFailed
numeric identifier ranks below alphanumericTrueTruePassed
release outranks its prereleaseTrueTruePassed
two-component version pads patchTrueTruePassed
build metadata is ignoredTrueTruePassed
version pair sample 1FalseFalsePassed
version pair sample 2FalseFalsePassed
version pair sample 3TrueTruePassed

SHA-256 / fb2652335ed1641bf5ff094798d15a89ed2ecb746519c96303002364ca811f93

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(version, op, target):
    def parse(s):
        s = s.split('+', 1)[0]
        core, _, pre = s.partition('-')
        parts = core.split('.')
        if not 1 <= len(parts) <= 3 or not all(p.isdigit() for p in parts):
            return None
        nums = [int(p) for p in parts] + [0] * (3 - len(parts))
        ids = pre.split('.') if pre else []
        return nums, ids
    def key_id(i):
        return (int(i), '') if i.isdigit() else (0, i)
    def cmp(a, b):
        if a[0] != b[0]:
            return -1 if a[0] < b[0] else 1
        x, y = a[1], b[1]
        if not x or not y:
            return (len(x) == 0) - (len(y) == 0) if (x or y) else 0
        for p, q in zip(x, y):
            kp, kq = key_id(p), key_id(q)
            if kp != kq:
                return -1 if kp < kq else 1
        return (len(x) > len(y)) - (len(x) < len(y))
    a, b = parse(version), parse(target)
    if a is None or b is None:
        return None
    c = cmp(a, b)
    return {'semVerLessThan': c < 0, 'semVerEqual': c == 0, 'semVerGreaterThan': c > 0}.get(op)
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('version pair sample 1', ['1.0.0-beta.11', 'semVerEqual', '1.0.0-alpha'], False),
  ('version pair sample 2', ['1.0.0-alpha', 'semVerEqual', '1.0.0'], False),
  ('version pair sample 3', ['1.0.0-alpha.10', 'semVerLessThan', '1.0.0+build.5'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('version pair sample 6', ['1.0.0.0', 'semVerEqual', '2.1'], None),
  ('version pair sample 43', ['1.0.0-beta.2', 'semVerGreaterThan', '1.0.0-1'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('version pair sample 11', ['1.0.0+build.5', 'semVerLessThan', '1.0.0.0'], None),
  ('version pair sample 34', ['1.0.0-2', 'semVerLessThan', '1.0.0-beta.2'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('minor ten beats minor nine', ['2.10.0', 'semVerGreaterThan', '2.9.0'], True),
  ('version pair sample 16', ['1.0.0-alpha.10', 'semVerEqual', '1.0.0'], False),
  ('version pair sample 20', ['1.0.0-rc.1+b7', 'semVerLessThan', '1.0.0-2'], False)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('minor ten beats minor nine', ['2.10.0', 'semVerGreaterThan', '2.9.0'], True),
  ('four components are invalid', ['1.0.0.0', 'semVerEqual', '1.0.0'], None),
  ('version pair sample 21', ['2.1.0', 'semVerGreaterThan', '2.1'], False)]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
numeric prerelease identifiers compare as numbersTrueTruePassed
numeric identifier ranks below alphanumericFalseTrueFailed
release outranks its prereleaseTrueTruePassed
two-component version pads patchTrueTruePassed
build metadata is ignoredTrueTruePassed
version pair sample 1FalseFalsePassed
version pair sample 2FalseFalsePassed
version pair sample 3TrueTruePassed

SHA-256 / 3a2a808133d9c52d989ea20787eb2d68919025ef8b04344e0911053f2376f024

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(version, op, target):
    def parse(s):
        s = s.split('+', 1)[0]
        core, _, pre = s.partition('-')
        parts = core.split('.')
        if not 1 <= len(parts) <= 3 or not all(p.isdigit() for p in parts):
            return None
        nums = [int(p) for p in parts] + [0] * (3 - len(parts))
        ids = pre.split('.') if pre else []
        return nums, ids
    def key_id(i):
        return (0, int(i), '') if i.isdigit() else (1, 0, i)
    def cmp(a, b):
        if a[0] != b[0]:
            return -1 if a[0] < b[0] else 1
        x, y = a[1], b[1]
        if not x or not y:
            return (len(x) == 0) - (len(y) == 0) if (x or y) else 0
        for p, q in zip(x, y):
            kp, kq = key_id(p), key_id(q)
            if kp != kq:
                return -1 if kp < kq else 1
        return (len(x) > len(y)) - (len(x) < len(y))
    a, b = parse(version), parse(target)
    if a is None or b is None:
        return None
    c = cmp(a, b)
    return {'semVerLessThan': c < 0, 'semVerEqual': c == 0, 'semVerGreaterThan': c > 0}.get(op)
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('version pair sample 1', ['1.0.0-beta.11', 'semVerEqual', '1.0.0-alpha'], False),
  ('version pair sample 2', ['1.0.0-alpha', 'semVerEqual', '1.0.0'], False),
  ('version pair sample 3', ['1.0.0-alpha.10', 'semVerLessThan', '1.0.0+build.5'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('version pair sample 6', ['1.0.0.0', 'semVerEqual', '2.1'], None),
  ('version pair sample 43', ['1.0.0-beta.2', 'semVerGreaterThan', '1.0.0-1'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('release outranks its prerelease', ['1.0.0', 'semVerGreaterThan', '1.0.0-rc.1'], True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('version pair sample 11', ['1.0.0+build.5', 'semVerLessThan', '1.0.0.0'], None),
  ('version pair sample 34', ['1.0.0-2', 'semVerLessThan', '1.0.0-beta.2'], True)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('two-component version pads patch', ['2.1', 'semVerEqual', '2.1.0'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('minor ten beats minor nine', ['2.10.0', 'semVerGreaterThan', '2.9.0'], True),
  ('version pair sample 16', ['1.0.0-alpha.10', 'semVerEqual', '1.0.0'], False),
  ('version pair sample 20', ['1.0.0-rc.1+b7', 'semVerLessThan', '1.0.0-2'], False)],
 [('numeric prerelease identifiers compare as numbers',
   ['1.0.0-beta.11', 'semVerGreaterThan', '1.0.0-beta.2'],
   True),
  ('numeric identifier ranks below alphanumeric', ['1.0.0-1', 'semVerLessThan', '1.0.0-alpha'], True),
  ('build metadata is ignored', ['1.0.0+build.5', 'semVerEqual', '1.0.0'], True),
  ('build metadata after prerelease is ignored', ['1.0.0-rc.1+b7', 'semVerEqual', '1.0.0-rc.1'], True),
  ('longer prerelease list ranks higher', ['1.0.0-alpha.1', 'semVerGreaterThan', '1.0.0-alpha'], True),
  ('minor ten beats minor nine', ['2.10.0', 'semVerGreaterThan', '2.9.0'], True),
  ('four components are invalid', ['1.0.0.0', 'semVerEqual', '1.0.0'], None),
  ('version pair sample 21', ['2.1.0', 'semVerGreaterThan', '2.1'], False)]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
numeric prerelease identifiers compare as numbersTrueTruePassed
numeric identifier ranks below alphanumericTrueTruePassed
release outranks its prereleaseTrueTruePassed
two-component version pads patchTrueTruePassed
build metadata is ignoredTrueTruePassed
version pair sample 1FalseFalsePassed
version pair sample 2FalseFalsePassed
version pair sample 3TrueTruePassed

SHA-256 / 603dcce820bd69f3599b634d243c0cbe919da8c79bf18e39f43df2d1500800e8

Verification & scope

A deterministic toy flag-evaluation model with a stipulated contract; it does not reproduce any vendor SDK byte for byte. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:53.621081+00:00.

Case digest / 7408df9f8d731945f2abfb554254441c3301c749c5cd60e621cd176b14341ecb