{"abstract":"Prompt tokens leak into response-only truncation.","category":"Sequence batch packing","checks":6,"contract":"Return response tokens only, preserving the full response when it fits and otherwise retaining its final limit tokens; output (tokens, response_start), where response_start is zero.","contract_signature":"prompt, response, limit","evaluation_group":"w3-seqpack-response-truncate-v1","failed_approach":"Taking a response prefix removes the latest response supervision even though it no longer leaks prompt tokens.","family":"w3-sequence-batch-packing-response-suffix-truncation","id":"FA-98126","implementations":{"attempt":{"sha256":"3bb7bda05d446391e0aca4e3d39960c8c34f53ed5d2174cd89a18c1f20f77e95","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(prompt, response, limit):\n    kept=response[:limit] if limit else []; return kept,0\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np=list(range(N+2)); r=[8,9]\ncheck('response exceeds limit', solve(p,r,1), ([9],0))\ncheck('response fits', solve(p,r,N+4), (r,0))\ncheck('zero limit', solve(p,r,0), ([],0))\ncheck('empty response', solve(p,[],N), ([],0))\ncheck('single final token', solve([1],[7],1), ([7],0))\ncheck('several response tokens', solve([1],[2,3,4],2), ([3,4],0))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"f415ae06f8e9d683aac3f268d7c875a22b3da284a9783ca58d6958c8624edb88","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(prompt, response, limit):\n    joined=prompt+response; kept=joined[-limit:] if limit else []; return kept,max(0,len(prompt)-max(0,len(joined)-limit))\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np=list(range(N+2)); r=[8,9]\ncheck('response exceeds limit', solve(p,r,1), ([9],0))\ncheck('response fits', solve(p,r,N+4), (r,0))\ncheck('zero limit', solve(p,r,0), ([],0))\ncheck('empty response', solve(p,[],N), ([],0))\ncheck('single final token', solve([1],[7],1), ([7],0))\ncheck('several response tokens', solve([1],[2,3,4],2), ([3,4],0))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"Inputs are short integer-token lists under the stated toy contract; results do not claim compatibility with a specific training framework. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-sequence-batch-packing-response-suffix-truncation","generated_at":"2026-09-30T16:48:09.757529+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"This deterministic toy packing rule models a common boundary in preparing bounded training sequences.","root_cause":"Truncation is applied to the concatenated prompt and response, so prompt tokens can leak into the response-only result.","sha256":"1c4fccee395013c7cc13ff549dece1332ee6cd95073f929dd2224044cf00da4e","title":"Prompt tokens leak into response-only truncation · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":28.164,"exit_code":1,"observations":[{"actual":[[8],0],"check":"response exceeds limit","expected":[[9],0],"passed":false},{"actual":[[8,9],0],"check":"response fits","expected":[[8,9],0],"passed":true},{"actual":[[],0],"check":"zero limit","expected":[[],0],"passed":true},{"actual":[[],0],"check":"empty response","expected":[[],0],"passed":true},{"actual":[[7],0],"check":"single final token","expected":[[7],0],"passed":true},{"actual":[[2,3],0],"check":"several response tokens","expected":[[3,4],0],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"response exceeds limit\", \"actual\": [[8], 0], \"expected\": [[9], 0], \"passed\": false}, {\"check\": \"response fits\", \"actual\": [[8, 9], 0], \"expected\": [[8, 9], 0], \"passed\": true}, {\"check\": \"zero limit\", \"actual\": [[], 0], \"expected\": [[], 0], \"passed\": true}, {\"check\": \"empty response\", \"actual\": [[], 0], \"expected\": [[], 0], \"passed\": true}, {\"check\": \"single final token\", \"actual\": [[7], 0], \"expected\": [[7], 0], \"passed\": true}, {\"check\": \"several response tokens\", \"actual\": [[2, 3], 0], \"expected\": [[3, 4], 0], \"passed\": false}], \"passed\": false}\n"},"broken":{"elapsed_ms":26.926,"exit_code":1,"observations":[{"actual":[[9],0],"check":"response exceeds limit","expected":[[9],0],"passed":true},{"actual":[[0,1,2,8,9],3],"check":"response fits","expected":[[8,9],0],"passed":false},{"actual":[[],0],"check":"zero limit","expected":[[],0],"passed":true},{"actual":[[2],1],"check":"empty response","expected":[[],0],"passed":false},{"actual":[[7],0],"check":"single final token","expected":[[7],0],"passed":true},{"actual":[[3,4],0],"check":"several response tokens","expected":[[3,4],0],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"response exceeds limit\", \"actual\": [[9], 0], \"expected\": [[9], 0], \"passed\": true}, {\"check\": \"response fits\", \"actual\": [[0, 1, 2, 8, 9], 3], \"expected\": [[8, 9], 0], \"passed\": false}, {\"check\": \"zero limit\", \"actual\": [[], 0], \"expected\": [[], 0], \"passed\": true}, {\"check\": \"empty response\", \"actual\": [[2], 1], \"expected\": [[], 0], \"passed\": false}, {\"check\": \"single final token\", \"actual\": [[7], 0], \"expected\": [[7], 0], \"passed\": true}, {\"check\": \"several response tokens\", \"actual\": [[3, 4], 0], \"expected\": [[3, 4], 0], \"passed\": true}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}