{"abstract":"Frequency penalty counts only distinct history tokens.","category":"Token sampling filters","checks":6,"contract":"Subtract penalty times the number of occurrences of each token in history from that token score.","contract_signature":"scores, history, penalty","evaluation_group":"w3-tsf-frequency-count-history","failed_approach":"Charging once to unseen candidates penalizes tokens absent from history.","family":"w3-token-sampling-filters-frequency-count-history","id":"FA-98381","implementations":{"attempt":{"sha256":"c37d3a47f27a075e69af2997164cc7c0c2ac6dc564c9ada64d044b5edc7b17c5","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\nimport math\nN = 1\nobservations = []\ndef solve(scores, history, penalty):\n    from collections import Counter\n    c=Counter(history)\n    return {t:v-penalty*(c[t] if c[t] else 1) for t,v in scores.items()}\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('repeated token',solve({'a':8.0,'b':4.0},['a']*(N+1),1),{'a':8.0-(N+1),'b':4.0})\ncheck('single token',solve({'a':N},['a'],2),{'a':N-2})\ncheck('empty',solve({'a':N},[],1),{'a':N})\ncheck('unseen remains',solve({'b':3.0},['a','a'],4),{'b':3.0})\ncheck('signed penalty',solve({'a':2.0},['a','a'], -1),{'a':4.0})\ncheck('mixed counts',solve({'a':9.0,'b':5.0},['a','b','a'],2),{'a':5.0,'b':3.0})\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"940857bdfee19aa16ec64c01492968ed61cdb973dd4cbdad2727322a5f7dd566","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\nimport math\nN = 1\nobservations = []\ndef solve(scores, history, penalty):\n    seen=set(history)\n    return {t:v-penalty*(1 if t in seen else 0) for t,v in scores.items()}\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('repeated token',solve({'a':8.0,'b':4.0},['a']*(N+1),1),{'a':8.0-(N+1),'b':4.0})\ncheck('single token',solve({'a':N},['a'],2),{'a':N-2})\ncheck('empty',solve({'a':N},[],1),{'a':N})\ncheck('unseen remains',solve({'b':3.0},['a','a'],4),{'b':3.0})\ncheck('signed penalty',solve({'a':2.0},['a','a'], -1),{'a':4.0})\ncheck('mixed counts',solve({'a':9.0,'b':5.0},['a','b','a'],2),{'a':5.0,'b':3.0})\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"These are toy finite-vocabulary contracts with supplied scores, histories, and draws; they do not claim production model quality or compatibility. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-token-sampling-filters-frequency-count-history","generated_at":"2026-09-30T16:48:12.372518+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A finite, deterministic token-selection contract isolates one decoding operation from model inference and runtime behavior.","root_cause":"Only membership is used, losing repeated-token frequency.","sha256":"6471e58d85d6f541d2b80f9e16f3b8c5c523635643fd8eeaeec52aaf299bb54e","title":"Token sampling: Frequency penalty counts only distinct history tokens · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":27.586,"exit_code":1,"observations":[{"actual":{"a":6.0,"b":3.0},"check":"repeated token","expected":{"a":6.0,"b":4.0},"passed":false},{"actual":{"a":-1},"check":"single token","expected":{"a":-1},"passed":true},{"actual":{"a":0},"check":"empty","expected":{"a":1},"passed":false},{"actual":{"b":-1.0},"check":"unseen remains","expected":{"b":3.0},"passed":false},{"actual":{"a":4.0},"check":"signed penalty","expected":{"a":4.0},"passed":true},{"actual":{"a":5.0,"b":3.0},"check":"mixed counts","expected":{"a":5.0,"b":3.0},"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"repeated token\", \"actual\": {\"a\": 6.0, \"b\": 3.0}, \"expected\": {\"a\": 6.0, \"b\": 4.0}, \"passed\": false}, {\"check\": \"single token\", \"actual\": {\"a\": -1}, \"expected\": {\"a\": -1}, \"passed\": true}, {\"check\": \"empty\", \"actual\": {\"a\": 0}, \"expected\": {\"a\": 1}, \"passed\": false}, {\"check\": \"unseen remains\", \"actual\": {\"b\": -1.0}, \"expected\": {\"b\": 3.0}, \"passed\": false}, {\"check\": \"signed penalty\", \"actual\": {\"a\": 4.0}, \"expected\": {\"a\": 4.0}, \"passed\": true}, {\"check\": \"mixed counts\", \"actual\": {\"a\": 5.0, \"b\": 3.0}, \"expected\": {\"a\": 5.0, \"b\": 3.0}, \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":28.851,"exit_code":1,"observations":[{"actual":{"a":7.0,"b":4.0},"check":"repeated token","expected":{"a":6.0,"b":4.0},"passed":false},{"actual":{"a":-1},"check":"single token","expected":{"a":-1},"passed":true},{"actual":{"a":1},"check":"empty","expected":{"a":1},"passed":true},{"actual":{"b":3.0},"check":"unseen remains","expected":{"b":3.0},"passed":true},{"actual":{"a":3.0},"check":"signed penalty","expected":{"a":4.0},"passed":false},{"actual":{"a":7.0,"b":3.0},"check":"mixed counts","expected":{"a":5.0,"b":3.0},"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"repeated token\", \"actual\": {\"a\": 7.0, \"b\": 4.0}, \"expected\": {\"a\": 6.0, \"b\": 4.0}, \"passed\": false}, {\"check\": \"single token\", \"actual\": {\"a\": -1}, \"expected\": {\"a\": -1}, \"passed\": true}, {\"check\": \"empty\", \"actual\": {\"a\": 1}, \"expected\": {\"a\": 1}, \"passed\": true}, {\"check\": \"unseen remains\", \"actual\": {\"b\": 3.0}, \"expected\": {\"b\": 3.0}, \"passed\": true}, {\"check\": \"signed penalty\", \"actual\": {\"a\": 3.0}, \"expected\": {\"a\": 4.0}, \"passed\": false}, {\"check\": \"mixed counts\", \"actual\": {\"a\": 7.0, \"b\": 3.0}, \"expected\": {\"a\": 5.0, \"b\": 3.0}, \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}