{"abstract":"weights are normalized across unrelated prompts.","category":"Preference pair construction","checks":6,"contract":"Rows are [prompt,id,weight]. Return [prompt,id,numerator,denominator] for positive weights normalized by prompt sum, preserving input order.","contract_signature":"rows","evaluation_group":"preference-per-prompt-weighting","failed_approach":"Normalizing each row to one ignores other eligible pairs for the same prompt.","family":"w3-preference-pair-construction-per-prompt-weighting","id":"FA-100081","implementations":{"attempt":{"sha256":"30ed27929224f3c8954cfff89a41ba948c1f0badc7101d19936fb25312f1a920","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    return [[r[0],r[1],1,1] for r in rows if r[2]>0]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np='p'+str(N); q='q'+str(N)\ncheck('separate denominators',solve([[p,'a',N],[p,'b',N],[q,'c',2*N]]),[[p,'a',1,2],[p,'b',1,2],[q,'c',1,1]])\ncheck('zero excluded',solve([[p,'a',0],[p,'b',N]]),[[p,'b',1,1]])\ncheck('empty',solve([]),[])\ncheck('all zero',solve([[p,'a',0]]),[])\ncheck('singleton',solve([[p,'a',N]]),[[p,'a',1,1]])\ncheck('interleaved groups',solve([[p,'a',1],[q,'b',1],[p,'c',3]]),[[p,'a',1,4],[q,'b',1,1],[p,'c',3,4]])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"0841dc420f610fd0a7b07cc52f306be8fc30ae902911d74d8db1daeac4d5d2b5","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    total=sum(r[2] for r in rows)\n    return [[r[0],r[1],r[2],total] for r in rows if r[2]>0]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np='p'+str(N); q='q'+str(N)\ncheck('separate denominators',solve([[p,'a',N],[p,'b',N],[q,'c',2*N]]),[[p,'a',1,2],[p,'b',1,2],[q,'c',1,1]])\ncheck('zero excluded',solve([[p,'a',0],[p,'b',N]]),[[p,'b',1,1]])\ncheck('empty',solve([]),[])\ncheck('all zero',solve([[p,'a',0]]),[])\ncheck('singleton',solve([[p,'a',N]]),[[p,'a',1,1]])\ncheck('interleaved groups',solve([[p,'a',1],[q,'b',1],[p,'c',3]]),[[p,'a',1,4],[q,'b',1,1],[p,'c',3,4]])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"These are explicit toy record contracts; they do not claim compatibility with a trainer or improved model behavior. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-preference-pair-construction-per-prompt-weighting","generated_at":"2026-09-30T16:48:31.717323+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A small deterministic preference-data transform makes pair semantics inspectable without a training framework.","root_cause":"The denominator is computed over the entire dataset, coupling independent prompt groups.","sha256":"59b307abf159cdd229e1a3c6bddc0d3c9fafe2990069bf0cc6427f24639a3f61","title":"Preference pairs: weights are normalized across unrelated prompts · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":28.889,"exit_code":1,"observations":[{"actual":[["p1","a",1,1],["p1","b",1,1],["q1","c",1,1]],"check":"separate denominators","expected":[["p1","a",1,2],["p1","b",1,2],["q1","c",1,1]],"passed":false},{"actual":[["p1","b",1,1]],"check":"zero excluded","expected":[["p1","b",1,1]],"passed":true},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":[],"check":"all zero","expected":[],"passed":true},{"actual":[["p1","a",1,1]],"check":"singleton","expected":[["p1","a",1,1]],"passed":true},{"actual":[["p1","a",1,1],["q1","b",1,1],["p1","c",1,1]],"check":"interleaved groups","expected":[["p1","a",1,4],["q1","b",1,1],["p1","c",3,4]],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"separate denominators\", \"actual\": [[\"p1\", \"a\", 1, 1], [\"p1\", \"b\", 1, 1], [\"q1\", \"c\", 1, 1]], \"expected\": [[\"p1\", \"a\", 1, 2], [\"p1\", \"b\", 1, 2], [\"q1\", \"c\", 1, 1]], \"passed\": false}, {\"check\": \"zero excluded\", \"actual\": [[\"p1\", \"b\", 1, 1]], \"expected\": [[\"p1\", \"b\", 1, 1]], \"passed\": true}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"all zero\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"singleton\", \"actual\": [[\"p1\", \"a\", 1, 1]], \"expected\": [[\"p1\", \"a\", 1, 1]], \"passed\": true}, {\"check\": \"interleaved groups\", \"actual\": [[\"p1\", \"a\", 1, 1], [\"q1\", \"b\", 1, 1], [\"p1\", \"c\", 1, 1]], \"expected\": [[\"p1\", \"a\", 1, 4], [\"q1\", \"b\", 1, 1], [\"p1\", \"c\", 3, 4]], \"passed\": false}], \"passed\": false}\n"},"broken":{"elapsed_ms":28.723,"exit_code":1,"observations":[{"actual":[["p1","a",1,4],["p1","b",1,4],["q1","c",2,4]],"check":"separate denominators","expected":[["p1","a",1,2],["p1","b",1,2],["q1","c",1,1]],"passed":false},{"actual":[["p1","b",1,1]],"check":"zero excluded","expected":[["p1","b",1,1]],"passed":true},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":[],"check":"all zero","expected":[],"passed":true},{"actual":[["p1","a",1,1]],"check":"singleton","expected":[["p1","a",1,1]],"passed":true},{"actual":[["p1","a",1,5],["q1","b",1,5],["p1","c",3,5]],"check":"interleaved groups","expected":[["p1","a",1,4],["q1","b",1,1],["p1","c",3,4]],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"separate denominators\", \"actual\": [[\"p1\", \"a\", 1, 4], [\"p1\", \"b\", 1, 4], [\"q1\", \"c\", 2, 4]], \"expected\": [[\"p1\", \"a\", 1, 2], [\"p1\", \"b\", 1, 2], [\"q1\", \"c\", 1, 1]], \"passed\": false}, {\"check\": \"zero excluded\", \"actual\": [[\"p1\", \"b\", 1, 1]], \"expected\": [[\"p1\", \"b\", 1, 1]], \"passed\": true}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"all zero\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"singleton\", \"actual\": [[\"p1\", \"a\", 1, 1]], \"expected\": [[\"p1\", \"a\", 1, 1]], \"passed\": true}, {\"check\": \"interleaved groups\", \"actual\": [[\"p1\", \"a\", 1, 5], [\"q1\", \"b\", 1, 5], [\"p1\", \"c\", 3, 5]], \"expected\": [[\"p1\", \"a\", 1, 4], [\"q1\", \"b\", 1, 1], [\"p1\", \"c\", 3, 4]], \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}