{"abstract":"Prompt tokens consume the new-token budget.","category":"Model output stopping rules","checks":6,"contract":"Given prompt length, generated tokens, and nonnegative maximum new tokens, retain at most that many generated tokens. Return retained list and whether the generated limit was reached.","contract_signature":"tokens, prompt_len, max_new","evaluation_group":"w3-output-prompt-exclusion","failed_approach":"The attempted slice uses max_new correctly but len(tokens)>max_new still reports an exact-limit output as unfinished.","family":"w3-model-output-stopping-rules-prompt-exclusion","id":"FA-100756","implementations":{"attempt":{"sha256":"6c8586c74d286ab852586353e19ec784054a02075522855ec220489cf6bceb8e","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(tokens, prompt_len, max_new):\n    kept=tokens[:max_new]\n    return kept,len(tokens)>max_new\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('long prompt',solve(list(range(N+2)),N,2),([0,1],True))\ncheck('prompt does not count',solve([4,5],N,2),([4,5],True))\ncheck('zero budget',solve([1],N,0),([],True))\ncheck('under budget',solve([1],N,3),([1],False))\ncheck('exact budget',solve([1,2],N,2),([1,2],True))\ncheck('empty',solve([],N,3),([],False))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"13316e2c10e0eccf1d90e4a3d06df8daea4ed45f450ba252f4d142380b1bac0a","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(tokens, prompt_len, max_new):\n    kept=tokens[:max(0,max_new-prompt_len)]\n    return kept,len(kept)>=max_new\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('long prompt',solve(list(range(N+2)),N,2),([0,1],True))\ncheck('prompt does not count',solve([4,5],N,2),([4,5],True))\ncheck('zero budget',solve([1],N,0),([],True))\ncheck('under budget',solve([1],N,3),([1],False))\ncheck('exact budget',solve([1,2],N,2),([1,2],True))\ncheck('empty',solve([],N,3),([],False))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"This toy contract covers the supplied token records only; it does not claim compatibility with a particular model runtime or tokenizer. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-model-output-stopping-rules-prompt-exclusion","generated_at":"2026-09-30T16:48:39.096245+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A deterministic toy output controller makes one stopping-rule contract independently inspectable.","root_cause":"The cap subtracts prompt_len from a budget defined only for generated tokens; a partial repair also needs an inclusive reached-limit comparison.","sha256":"730366271428f3aecaed58b4d79b1090c4f8f43b7f20c2a25d647273f7289cee","title":"Output stopping: Prompt tokens consume the new-token budget · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":34.293,"exit_code":1,"observations":[{"actual":[[0,1],true],"check":"long prompt","expected":[[0,1],true],"passed":true},{"actual":[[4,5],false],"check":"prompt does not count","expected":[[4,5],true],"passed":false},{"actual":[[],true],"check":"zero budget","expected":[[],true],"passed":true},{"actual":[[1],false],"check":"under budget","expected":[[1],false],"passed":true},{"actual":[[1,2],false],"check":"exact budget","expected":[[1,2],true],"passed":false},{"actual":[[],false],"check":"empty","expected":[[],false],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"long prompt\", \"actual\": [[0, 1], true], \"expected\": [[0, 1], true], \"passed\": true}, {\"check\": \"prompt does not count\", \"actual\": [[4, 5], false], \"expected\": [[4, 5], true], \"passed\": false}, {\"check\": \"zero budget\", \"actual\": [[], true], \"expected\": [[], true], \"passed\": true}, {\"check\": \"under budget\", \"actual\": [[1], false], \"expected\": [[1], false], \"passed\": true}, {\"check\": \"exact budget\", \"actual\": [[1, 2], false], \"expected\": [[1, 2], true], \"passed\": false}, {\"check\": \"empty\", \"actual\": [[], false], \"expected\": [[], false], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":33.097,"exit_code":1,"observations":[{"actual":[[0],false],"check":"long prompt","expected":[[0,1],true],"passed":false},{"actual":[[4],false],"check":"prompt does not count","expected":[[4,5],true],"passed":false},{"actual":[[],true],"check":"zero budget","expected":[[],true],"passed":true},{"actual":[[1],false],"check":"under budget","expected":[[1],false],"passed":true},{"actual":[[1],false],"check":"exact budget","expected":[[1,2],true],"passed":false},{"actual":[[],false],"check":"empty","expected":[[],false],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"long prompt\", \"actual\": [[0], false], \"expected\": [[0, 1], true], \"passed\": false}, {\"check\": \"prompt does not count\", \"actual\": [[4], false], \"expected\": [[4, 5], true], \"passed\": false}, {\"check\": \"zero budget\", \"actual\": [[], true], \"expected\": [[], true], \"passed\": true}, {\"check\": \"under budget\", \"actual\": [[1], false], \"expected\": [[1], false], \"passed\": true}, {\"check\": \"exact budget\", \"actual\": [[1], false], \"expected\": [[1, 2], true], \"passed\": false}, {\"check\": \"empty\", \"actual\": [[], false], \"expected\": [[], false], \"passed\": true}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}