{"abstract":"A length finish overwrites an observed stop reason.","category":"Model output stopping rules","checks":6,"contract":"Given stop position and generated count at cap, reason is stop if a stop was observed; otherwise length when at cap; otherwise null.","contract_signature":"tokens, stop_ids, limit","evaluation_group":"w3-output-reason-precedence","failed_approach":"Checking length first gives it priority even when a stop marker is present.","family":"w3-model-output-stopping-rules-finish-reason-precedence","id":"FA-100776","implementations":{"attempt":{"sha256":"83414ae8014e3e000c1cc6e8d43122186f72b566a2db212dab5f7fd561456a2c","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(tokens, stop_ids, limit):\n    if len(tokens)>=limit:return 'length'\n    if any(t in stop_ids for t in tokens):return 'stop'\n    return None\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('stop exactly at cap',solve([1,8],[8],2),'stop')\ncheck('length only',solve([1,2],[],2),'length')\ncheck('stop before cap',solve([8], [8],3),'stop')\ncheck('below cap',solve([1],[],2),None)\ncheck('zero cap empty',solve([],[],0),'length')\ncheck('stop after cap input',solve([1,8,2],[8],2),'stop')\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"76f1e3e9a963688c8e41f60d78122f79ff96eb8c47f9839844521a0e00c47168","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(tokens, stop_ids, limit):\n    reason=None\n    for t in tokens:\n        if t in stop_ids:reason='stop'\n    if len(tokens)>=limit:reason='length'\n    return reason\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('stop exactly at cap',solve([1,8],[8],2),'stop')\ncheck('length only',solve([1,2],[],2),'length')\ncheck('stop before cap',solve([8], [8],3),'stop')\ncheck('below cap',solve([1],[],2),None)\ncheck('zero cap empty',solve([],[],0),'length')\ncheck('stop after cap input',solve([1,8,2],[8],2),'stop')\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"This toy contract covers the supplied token records only; it does not claim compatibility with a particular model runtime or tokenizer. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-model-output-stopping-rules-finish-reason-precedence","generated_at":"2026-09-30T16:48:39.593524+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A deterministic output-control model isolates one finish or streaming boundary from any particular runtime.","root_cause":"A later length check overwrites a stop reason already observed in the output.","sha256":"f2e0d6ce8241796b378f3f877d6bc8ba002939c96f0d652686d3303116b87ffd","title":"Output stopping: A length finish overwrites an observed stop reason · case 01","variant":1,"variant_policy":"The same explicitly authored fixtures repeat across all five numbered records; runtime N is unused. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":29.762,"exit_code":1,"observations":[{"actual":"length","check":"stop exactly at cap","expected":"stop","passed":false},{"actual":"length","check":"length only","expected":"length","passed":true},{"actual":"stop","check":"stop before cap","expected":"stop","passed":true},{"actual":null,"check":"below cap","expected":null,"passed":true},{"actual":"length","check":"zero cap empty","expected":"length","passed":true},{"actual":"length","check":"stop after cap input","expected":"stop","passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"stop exactly at cap\", \"actual\": \"length\", \"expected\": \"stop\", \"passed\": false}, {\"check\": \"length only\", \"actual\": \"length\", \"expected\": \"length\", \"passed\": true}, {\"check\": \"stop before cap\", \"actual\": \"stop\", \"expected\": \"stop\", \"passed\": true}, {\"check\": \"below cap\", \"actual\": null, \"expected\": null, \"passed\": true}, {\"check\": \"zero cap empty\", \"actual\": \"length\", \"expected\": \"length\", \"passed\": true}, {\"check\": \"stop after cap input\", \"actual\": \"length\", \"expected\": \"stop\", \"passed\": false}], \"passed\": false}\n"},"broken":{"elapsed_ms":33.49,"exit_code":1,"observations":[{"actual":"length","check":"stop exactly at cap","expected":"stop","passed":false},{"actual":"length","check":"length only","expected":"length","passed":true},{"actual":"stop","check":"stop before cap","expected":"stop","passed":true},{"actual":null,"check":"below cap","expected":null,"passed":true},{"actual":"length","check":"zero cap empty","expected":"length","passed":true},{"actual":"length","check":"stop after cap input","expected":"stop","passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"stop exactly at cap\", \"actual\": \"length\", \"expected\": \"stop\", \"passed\": false}, {\"check\": \"length only\", \"actual\": \"length\", \"expected\": \"length\", \"passed\": true}, {\"check\": \"stop before cap\", \"actual\": \"stop\", \"expected\": \"stop\", \"passed\": true}, {\"check\": \"below cap\", \"actual\": null, \"expected\": null, \"passed\": true}, {\"check\": \"zero cap empty\", \"actual\": \"length\", \"expected\": \"length\", \"passed\": true}, {\"check\": \"stop after cap input\", \"actual\": \"length\", \"expected\": \"stop\", \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}