FA-3971 / Tokenization / Open access
Word bigrams cross a record boundary · case 01
Word bigrams cross a record boundary.
ROOT CAUSE
The implementation applies an operation whose text or grammar semantics violate this contract: Given a list of word-token records, return bigrams within records only.
VERIFIED REPAIR
Implement the complete stated contract, including the boundary fixtures: Given a list of word-token records, return bigrams within records only.
Unsuccessful approach: The attempted repair handles the primary example but still violates a separate boundary of the same contract.
Case contract
Given a list of word-token records, return bigrams within records only.
Why this case matters
A local executable model for consumers of structured text; oracle values are authored literals, not outputs copied from the repaired implementation.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap
N = 1
observations = []
def solve(x):
try:
return [list(p) for p in zip([v for row in x for v in row],[v for row in x for v in row][1:])]
except Exception as exc:
return {"error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve([['a', 'b'], ['c', 'd']]), [['a', 'b'], ['c', 'd']])
check('boundary 2', solve([['a', 'b', 'c']]), [['a', 'b'], ['b', 'c']])
check('boundary 3', solve([['a'], ['b']]), [])
check('boundary 4', solve([]), [])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| boundary 1 | [['a', 'b'], ['b', 'c'], ['c', 'd']] | [['a', 'b'], ['c', 'd']] | Failed |
| boundary 2 | [['a', 'b'], ['b', 'c']] | [['a', 'b'], ['b', 'c']] | Passed |
| boundary 3 | [['a', 'b']] | [] | Failed |
| boundary 4 | [] | [] | Passed |
SHA-256 / bef3c3bec02421936b302a55861b6527e034a7aa41fca203eaf4cc8e03d2a44c
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap
N = 1
observations = []
def solve(x):
try:
return [[row[i],row[i+1]] for row in x for i in range(0,len(row)-1,2)]
except Exception as exc:
return {"error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve([['a', 'b'], ['c', 'd']]), [['a', 'b'], ['c', 'd']])
check('boundary 2', solve([['a', 'b', 'c']]), [['a', 'b'], ['b', 'c']])
check('boundary 3', solve([['a'], ['b']]), [])
check('boundary 4', solve([]), [])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| boundary 1 | [['a', 'b'], ['c', 'd']] | [['a', 'b'], ['c', 'd']] | Passed |
| boundary 2 | [['a', 'b']] | [['a', 'b'], ['b', 'c']] | Failed |
| boundary 3 | [] | [] | Passed |
| boundary 4 | [] | [] | Passed |
SHA-256 / 998daf9b89444145e3ac63d1453aceb5ed9c1ec05bcc750ecbc0ec2c836ec7f4
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap
N = 1
observations = []
def solve(x):
try:
return [[row[i],row[i+1]] for row in x for i in range(len(row)-1)]
except Exception as exc:
return {"error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve([['a', 'b'], ['c', 'd']]), [['a', 'b'], ['c', 'd']])
check('boundary 2', solve([['a', 'b', 'c']]), [['a', 'b'], ['b', 'c']])
check('boundary 3', solve([['a'], ['b']]), [])
check('boundary 4', solve([]), [])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| boundary 1 | [['a', 'b'], ['c', 'd']] | [['a', 'b'], ['c', 'd']] | Passed |
| boundary 2 | [['a', 'b'], ['b', 'c']] | [['a', 'b'], ['b', 'c']] | Passed |
| boundary 3 | [] | [] | Passed |
| boundary 4 | [] | [] | Passed |
SHA-256 / 0dfe7ac87589046512b8a0cc53db96edba95d0eb7aaf263d319ab5de62f6a2c6
Verification & scope
This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:37:33.636790+00:00.
Case digest / c42a2354b55f26f56afb84633c21b0fd3ca308979cb95ce8d025b8b7cc5547dc