FA-3766 / Tokenization / Open access

Keyword prefix matching splits an identifier · case 01

Keyword prefix matching splits an identifier.

Verified by executionVariant 1 · 4 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The implementation applies an operation whose text or grammar semantics violate this contract: Tokenize ASCII identifiers and label exact if/else as keywords.

THE FAILURE

The implementation applies an operation whose text or grammar semantics violate this contract: Tokenize ASCII identifiers and label exact if/else as keywords.

Unsuccessful approach: The attempted repair handles the primary example but still violates a separate boundary of the same contract.

Case contract

Tokenize ASCII identifiers and label exact if/else as keywords.

Why this case matters

A local executable model for consumers of structured text; oracle values are authored literals, not outputs copied from the repaired implementation.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return [['keyword' if w.startswith(('if','else')) else 'name',w] for w in re.findall(r'[A-Za-z_][A-Za-z0-9_]*',x)]
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve('iffy'), [['name', 'iffy']])
check('boundary 2', solve('shelf_else'), [['name', 'shelf_else']])
check('boundary 3', solve('if else'), [['keyword', 'if'], ['keyword', 'else']])
check('boundary 4', solve(''), [])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
boundary 1	`[['keyword', 'iffy']]`	`[['name', 'iffy']]`	Failed
boundary 2	`[['name', 'shelf_else']]`	`[['name', 'shelf_else']]`	Passed
boundary 3	`[['keyword', 'if'], ['keyword', 'else']]`	`[['keyword', 'if'], ['keyword', 'else']]`	Passed
boundary 4	`[]`	`[]`	Passed

SHA-256 / 9fd976c502c4c4d9feebed180259dd16d168225a8e8ec87ef82e1ea879818f48

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return [['keyword' if 'if' in w or 'else' in w else 'name',w] for w in re.findall(r'[A-Za-z_][A-Za-z0-9_]*',x)]
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve('iffy'), [['name', 'iffy']])
check('boundary 2', solve('shelf_else'), [['name', 'shelf_else']])
check('boundary 3', solve('if else'), [['keyword', 'if'], ['keyword', 'else']])
check('boundary 4', solve(''), [])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
boundary 1	`[['keyword', 'iffy']]`	`[['name', 'iffy']]`	Failed
boundary 2	`[['keyword', 'shelf_else']]`	`[['name', 'shelf_else']]`	Failed
boundary 3	`[['keyword', 'if'], ['keyword', 'else']]`	`[['keyword', 'if'], ['keyword', 'else']]`	Passed
boundary 4	`[]`	`[]`	Passed

SHA-256 / aaaad2c10b71610dcf518c04e1e699adf573b63d441ea2d86e591dc58fd63294

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 4 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:37:30.447727+00:00.

Case digest / 030643eb51bb2c157c5dc78f30bddb318d3f346afd464f3458dccb9bbc1c6fff