{"abstract":"Target evaluation count omits the rejected token comparison.","category":"Speculative decoding bookkeeping","checks":7,"contract":"If rejection is at zero-based index k, target evaluations equal k+1; if fully accepted, they equal the draft length.","contract_signature":"draft, rejected_at","evaluation_group":"spec-target-evaluation-count","failed_approach":"Always charging one extra evaluation overcounts fully accepted proposals.","family":"w3-speculative-decoding-bookkeeping-target-evaluation-count","id":"FA-100686","implementations":{"attempt":{"sha256":"c6454338206eef0313d896f4546b51bb8cb6d7a12c16318c803217a4b9c1ca12","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(draft, rejected_at):\n    return len(draft)+1\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('first rejection', solve([1,2],0),1)\ncheck('later rejection', solve([1,2,3],2),3)\ncheck('full verification', solve([1,2],None),2)\ncheck('empty verified batch', solve([],None),0)\ncheck('last rejection', solve([1,2],1),2)\ncheck('singleton rejection', solve([N],0),1)\ncheck('rejection at penultimate slot', solve([1,2,3,4],2),3)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"a00b560a4a49185575b40def21c396d1810e1dffe5dba7bfbb4985dc77926091","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(draft, rejected_at):\n    return rejected_at if rejected_at is not None else len(draft)\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('first rejection', solve([1,2],0),1)\ncheck('later rejection', solve([1,2,3],2),3)\ncheck('full verification', solve([1,2],None),2)\ncheck('empty verified batch', solve([],None),0)\ncheck('last rejection', solve([1,2],1),2)\ncheck('singleton rejection', solve([N],0),1)\ncheck('rejection at penultimate slot', solve([1,2,3,4],2),3)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"These are finite toy sequences with explicit contracts; they make no claim about production decoder behavior or model quality. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-speculative-decoding-bookkeeping-target-evaluation-count","generated_at":"2026-09-30T16:48:38.598891+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A self-contained bookkeeping model makes prefix and token-count semantics observable without a model runtime.","root_cause":"The check at the rejecting position is excluded from work accounting.","sha256":"454898f9237a5b2205e34c859e6eec66cf9551b69481e9bdd4d7e8eb598d8f11","title":"Decoding: Target evaluation count omits the rejected token comparison · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":31.865,"exit_code":1,"observations":[{"actual":3,"check":"first rejection","expected":1,"passed":false},{"actual":4,"check":"later rejection","expected":3,"passed":false},{"actual":3,"check":"full verification","expected":2,"passed":false},{"actual":1,"check":"empty verified batch","expected":0,"passed":false},{"actual":3,"check":"last rejection","expected":2,"passed":false},{"actual":2,"check":"singleton rejection","expected":1,"passed":false},{"actual":5,"check":"rejection at penultimate slot","expected":3,"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"first rejection\", \"actual\": 3, \"expected\": 1, \"passed\": false}, {\"check\": \"later rejection\", \"actual\": 4, \"expected\": 3, \"passed\": false}, {\"check\": \"full verification\", \"actual\": 3, \"expected\": 2, \"passed\": false}, {\"check\": \"empty verified batch\", \"actual\": 1, \"expected\": 0, \"passed\": false}, {\"check\": \"last rejection\", \"actual\": 3, \"expected\": 2, \"passed\": false}, {\"check\": \"singleton rejection\", \"actual\": 2, \"expected\": 1, \"passed\": false}, {\"check\": \"rejection at penultimate slot\", \"actual\": 5, \"expected\": 3, \"passed\": false}], \"passed\": false}\n"},"broken":{"elapsed_ms":32.43,"exit_code":1,"observations":[{"actual":0,"check":"first rejection","expected":1,"passed":false},{"actual":2,"check":"later rejection","expected":3,"passed":false},{"actual":2,"check":"full verification","expected":2,"passed":true},{"actual":0,"check":"empty verified batch","expected":0,"passed":true},{"actual":1,"check":"last rejection","expected":2,"passed":false},{"actual":0,"check":"singleton rejection","expected":1,"passed":false},{"actual":2,"check":"rejection at penultimate slot","expected":3,"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"first rejection\", \"actual\": 0, \"expected\": 1, \"passed\": false}, {\"check\": \"later rejection\", \"actual\": 2, \"expected\": 3, \"passed\": false}, {\"check\": \"full verification\", \"actual\": 2, \"expected\": 2, \"passed\": true}, {\"check\": \"empty verified batch\", \"actual\": 0, \"expected\": 0, \"passed\": true}, {\"check\": \"last rejection\", \"actual\": 1, \"expected\": 2, \"passed\": false}, {\"check\": \"singleton rejection\", \"actual\": 0, \"expected\": 1, \"passed\": false}, {\"check\": \"rejection at penultimate slot\", \"actual\": 2, \"expected\": 3, \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}