{"abstract":"beam stopping ignores the best normalized score a live path can still reach.","category":"Beam search bookkeeping","checks":8,"contract":"Scores are nonpositive upper bounds on cumulative log scores; future token contributions cannot increase them. Active rows are [score_bound,current_generated_length], finished contains already-normalized final scores, width is positive, and max_length is at least each active length. The length factor is 1+alpha*(length-1), alpha>=0. Stop if there are no active paths, or at least width finished paths exist and the best active upper bound normalized at max_length is no better than the width-th best finished score.","contract_signature":"active, finished, width, max_length, alpha","evaluation_group":"beam-length_capped_bound","failed_approach":"Normalizing at the current length still stops too early when a live path can improve its normalized rank by extending.","family":"w3-beam-search-bookkeeping-length-capped-score-bound","id":"FA-98536","implementations":{"attempt":{"sha256":"94aac97190ba247a490c2cfba3a75539f92ffa5a4ed6e56006feb2085960793b","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(active, finished, width, max_length, alpha):\n    if not active: return True\n    if len(finished)<width: return False\n    threshold=sorted(finished,reverse=True)[width-1]\n    return max(score/(1+alpha*(length-1)) for score,length in active)<=threshold\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('extension can improve rank',solve([[-3,1]],[-1,-2],2,4,1),False)\ncheck('bounded below kth result',solve([[-10*N,1]],[-N,-2*N],2,4,1),True)\ncheck('pool short of requested width',solve([[-10,1]],[-1],2,4,1),False)\ncheck('empty live set is complete',solve([],[],1,4,1),True)\ncheck('exact optimistic tie can stop',solve([[-4,1]],[-1],1,3,1),True)\ncheck('current length is not the cap',solve([[-1,2]],[-0.5],1,4,1),False)\ncheck('already at maximum length',solve([[-2,4]],[-0.5],1,4,1),True)\ncheck('zero penalty factor',solve([[-1,1]],[-2],1,4,0),False)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"fc1c405e631120e0735356465c6d8c334f2aae054fb87de1597e9b465026ed8b","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(active, finished, width, max_length, alpha):\n    return bool(finished) and max((score for score,length in active),default=float('-inf'))<=max(finished)\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('extension can improve rank',solve([[-3,1]],[-1,-2],2,4,1),False)\ncheck('bounded below kth result',solve([[-10*N,1]],[-N,-2*N],2,4,1),True)\ncheck('pool short of requested width',solve([[-10,1]],[-1],2,4,1),False)\ncheck('empty live set is complete',solve([],[],1,4,1),True)\ncheck('exact optimistic tie can stop',solve([[-4,1]],[-1],1,3,1),True)\ncheck('current length is not the cap',solve([[-1,2]],[-0.5],1,4,1),False)\ncheck('already at maximum length',solve([[-2,4]],[-0.5],1,4,1),True)\ncheck('zero penalty factor',solve([[-1,1]],[-2],1,4,0),False)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"This is a finite toy beam-state contract and makes no claim about neural model quality or production decoding. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-beam-search-bookkeeping-length-capped-score-bound","generated_at":"2026-09-30T16:48:14.106818+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"Beam decoders rely on consistent hypothesis state while expanding and ranking candidates.","root_cause":"The stop check compares current live totals with finished scores, although a negative total becomes less negative under the configured length normalization as the path grows.","sha256":"fa62a7d4ecc8a72a4b9611940644e83ddabfbd4e8759defa6eda2ec919c7f5a3","title":"Beam search: beam stopping ignores the best normalized score a live path can still reach · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":29.049,"exit_code":1,"observations":[{"actual":true,"check":"extension can improve rank","expected":false,"passed":false},{"actual":true,"check":"bounded below kth result","expected":true,"passed":true},{"actual":false,"check":"pool short of requested width","expected":false,"passed":true},{"actual":true,"check":"empty live set is complete","expected":true,"passed":true},{"actual":true,"check":"exact optimistic tie can stop","expected":true,"passed":true},{"actual":true,"check":"current length is not the cap","expected":false,"passed":false},{"actual":true,"check":"already at maximum length","expected":true,"passed":true},{"actual":false,"check":"zero penalty factor","expected":false,"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"extension can improve rank\", \"actual\": true, \"expected\": false, \"passed\": false}, {\"check\": \"bounded below kth result\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"pool short of requested width\", \"actual\": false, \"expected\": false, \"passed\": true}, {\"check\": \"empty live set is complete\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"exact optimistic tie can stop\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"current length is not the cap\", \"actual\": true, \"expected\": false, \"passed\": false}, {\"check\": \"already at maximum length\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"zero penalty factor\", \"actual\": false, \"expected\": false, \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":26.64,"exit_code":1,"observations":[{"actual":true,"check":"extension can improve rank","expected":false,"passed":false},{"actual":true,"check":"bounded below kth result","expected":true,"passed":true},{"actual":true,"check":"pool short of requested width","expected":false,"passed":false},{"actual":false,"check":"empty live set is complete","expected":true,"passed":false},{"actual":true,"check":"exact optimistic tie can stop","expected":true,"passed":true},{"actual":true,"check":"current length is not the cap","expected":false,"passed":false},{"actual":true,"check":"already at maximum length","expected":true,"passed":true},{"actual":false,"check":"zero penalty factor","expected":false,"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"extension can improve rank\", \"actual\": true, \"expected\": false, \"passed\": false}, {\"check\": \"bounded below kth result\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"pool short of requested width\", \"actual\": true, \"expected\": false, \"passed\": false}, {\"check\": \"empty live set is complete\", \"actual\": false, \"expected\": true, \"passed\": false}, {\"check\": \"exact optimistic tie can stop\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"current length is not the cap\", \"actual\": true, \"expected\": false, \"passed\": false}, {\"check\": \"already at maximum length\", \"actual\": true, \"expected\": true, \"passed\": true}, {\"check\": \"zero penalty factor\", \"actual\": false, \"expected\": false, \"passed\": true}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}