FA-100231 / Annotation span alignment / Open access
Token coverage ratio counts only fully contained tokens · case 01
Token coverage ratio counts only fully contained tokens.
ROOT CAUSE
Coverage numerator includes only tokens wholly contained in the annotation, excluding partial overlaps.
THE FAILURE
Coverage numerator includes only tokens wholly contained in the annotation, excluding partial overlaps.
Unsuccessful approach: Capping coverage by token and span lengths ignores their relative positions and overstates disjoint coverage.
Case contract
Return covered token-character width divided by total token width, rounded to six decimal places; return 0.0 when total width is zero.
Why this case matters
A small offline annotation transformation keeps span coordinates explicit when text or token boundaries change.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(tokens, start, end):
total=sum(b-a for a,b in tokens); covered=sum(b-a for a,b in tokens if start<=a and b<=end); return round(covered/total,6) if total else 0.0
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('partial tokens', solve([[0,4],[5,9]],3,6), 0.25)
check('full coverage', solve([[0,2],[2,5]],0,5), 1.0)
check('no overlap', solve([[0,2]],2,4), 0.0)
check('empty tokens', solve([],0,3), 0.0)
check('zero width tokens', solve([[N,N]],N,N+1), 0.0)
check('half one token', solve([[0,4]],1,3), 0.5)
check('span crosses gap', solve([[0,2],[5,7]],1,6), 0.5)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| partial tokens | 0.0 | 0.25 | Failed |
| full coverage | 1.0 | 1.0 | Passed |
| no overlap | 0.0 | 0.0 | Passed |
| empty tokens | 0.0 | 0.0 | Passed |
| zero width tokens | 0.0 | 0.0 | Passed |
| half one token | 0.0 | 0.5 | Failed |
| span crosses gap | 0.0 | 0.5 | Failed |
SHA-256 / c9b1bc72aca703abe2f92282e072e98f81d011d95760d92ed710a611494abcc7
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(tokens, start, end):
total=sum(b-a for a,b in tokens); covered=sum(min(b-a,max(0,end-start)) for a,b in tokens); return round(covered/total,6) if total else 0.0
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('partial tokens', solve([[0,4],[5,9]],3,6), 0.25)
check('full coverage', solve([[0,2],[2,5]],0,5), 1.0)
check('no overlap', solve([[0,2]],2,4), 0.0)
check('empty tokens', solve([],0,3), 0.0)
check('zero width tokens', solve([[N,N]],N,N+1), 0.0)
check('half one token', solve([[0,4]],1,3), 0.5)
check('span crosses gap', solve([[0,2],[5,7]],1,6), 0.5)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| partial tokens | 0.75 | 0.25 | Failed |
| full coverage | 1.0 | 1.0 | Passed |
| no overlap | 1.0 | 0.0 | Failed |
| empty tokens | 0.0 | 0.0 | Passed |
| zero width tokens | 0.0 | 0.0 | Passed |
| half one token | 0.5 | 0.5 | Passed |
| span crosses gap | 1.0 | 0.5 | Failed |
SHA-256 / 320082769ff4f720b2d1b50f22ea62ea14e3867ee3f58a6931ff08252cc1bd30
HELD IN THE MEMBER ARCHIVE
The verified repair and its recorded checks are member-only.
This mechanism has 7 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.
Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.
Member access is invitation-based. Sign in with your invited account to inspect the repair.
Sign in to the archive ↗Verification & scope
Contracts use integer offsets, explicit edits, and supplied token boundaries; they do not define a tokenizer or production annotation format. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-30T16:48:33.099174+00:00.
Case digest / 8fa47f89e06fd96568832da21410d688acb1d9d01336ff383054cde269af8769