FAILURE MAP
← Case archive

FA-100756 / Model output stopping rules / Open access

Output stopping: Prompt tokens consume the new-token budget · case 01

Prompt tokens consume the new-token budget.

Verified by executionVariant 1 · 6 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The cap subtracts prompt_len from a budget defined only for generated tokens; a partial repair also needs an inclusive reached-limit comparison.

THE FAILURE

The cap subtracts prompt_len from a budget defined only for generated tokens; a partial repair also needs an inclusive reached-limit comparison.

Unsuccessful approach: The attempted slice uses max_new correctly but len(tokens)>max_new still reports an exact-limit output as unfinished.

Case contract

Given prompt length, generated tokens, and nonnegative maximum new tokens, retain at most that many generated tokens. Return retained list and whether the generated limit was reached.

Why this case matters

A deterministic toy output controller makes one stopping-rule contract independently inspectable.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(tokens, prompt_len, max_new):
    kept=tokens[:max(0,max_new-prompt_len)]
    return kept,len(kept)>=max_new
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('long prompt',solve(list(range(N+2)),N,2),([0,1],True))
check('prompt does not count',solve([4,5],N,2),([4,5],True))
check('zero budget',solve([1],N,0),([],True))
check('under budget',solve([1],N,3),([1],False))
check('exact budget',solve([1,2],N,2),([1,2],True))
check('empty',solve([],N,3),([],False))
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
long prompt[[0], False][[0, 1], True]Failed
prompt does not count[[4], False][[4, 5], True]Failed
zero budget[[], True][[], True]Passed
under budget[[1], False][[1], False]Passed
exact budget[[1], False][[1, 2], True]Failed
empty[[], False][[], False]Passed

SHA-256 / 13316e2c10e0eccf1d90e4a3d06df8daea4ed45f450ba252f4d142380b1bac0a

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(tokens, prompt_len, max_new):
    kept=tokens[:max_new]
    return kept,len(tokens)>max_new
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('long prompt',solve(list(range(N+2)),N,2),([0,1],True))
check('prompt does not count',solve([4,5],N,2),([4,5],True))
check('zero budget',solve([1],N,0),([],True))
check('under budget',solve([1],N,3),([1],False))
check('exact budget',solve([1,2],N,2),([1,2],True))
check('empty',solve([],N,3),([],False))
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
long prompt[[0, 1], True][[0, 1], True]Passed
prompt does not count[[4, 5], False][[4, 5], True]Failed
zero budget[[], True][[], True]Passed
under budget[[1], False][[1], False]Passed
exact budget[[1, 2], False][[1, 2], True]Failed
empty[[], False][[], False]Passed

SHA-256 / 6c8586c74d286ab852586353e19ec784054a02075522855ec220489cf6bceb8e

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 6 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Sign in to the archive ↗

Verification & scope

This toy contract covers the supplied token records only; it does not claim compatibility with a particular model runtime or tokenizer. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-30T16:48:39.096245+00:00.

Case digest / 730366271428f3aecaed58b4d79b1090c4f8f43b7f20c2a25d647273f7289cee