{"abstract":"responses from different prompts are compared.","category":"Preference pair construction","checks":6,"contract":"Rows are [prompt,id,score]. For each prompt with at least two rows, return [prompt,max_id,min_id] in first-seen prompt order; ties break by lexical ID.","contract_signature":"rows","evaluation_group":"preference-prompt-grouping","failed_approach":"Grouping by prompt but relying on input order for equal scores makes tied endpoints depend on row order.","family":"w3-preference-pair-construction-prompt-grouping","id":"FA-100056","implementations":{"attempt":{"sha256":"8fff7a060a1557b8a82f20c1bf41283a2d7859209bc1ebcbe3db01b0645a339b","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    groups={}\n    for r in rows: groups.setdefault(r[0],[]).append(r)\n    sorted_rows=sorted(rows,key=lambda r:(r[2],r[1]))\n    return [[p,max(g,key=lambda r:r[2])[1],min(g,key=lambda r:r[2])[1]] for p,g in groups.items() if len(g)>1]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np='p'+str(N); q='q'+str(N)\nrows=[[p,'a',N],[q,'x',N+5],[p,'b',0],[q,'y',1]]\ncheck('separate prompts',solve(rows),[[p,'a','b'],[q,'x','y']])\ncheck('singleton group omitted',solve([[p,'a',N],[q,'x',1],[q,'y',0]]),[[q,'x','y']])\ncheck('empty',solve([]),[])\ncheck('single row',solve([[p,'a',N]]),[])\ncheck('tie lexical ids',solve([[p,'a',N],[p,'z',N]]),[[p,'z','a']])\ncheck('input group order',solve([[q,'b',N],[p,'a',N+1],[q,'a',0]]),[[q,'b','a']])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"d75cb8e80bb07f4810749abce49b9f1ca61fb8f3cc0aa504255566b8baf77071","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    s=sorted(rows,key=lambda r:(r[2],r[1]))\n    return [[s[-1][0],s[-1][1],s[0][1]]] if len(s)>1 else []\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np='p'+str(N); q='q'+str(N)\nrows=[[p,'a',N],[q,'x',N+5],[p,'b',0],[q,'y',1]]\ncheck('separate prompts',solve(rows),[[p,'a','b'],[q,'x','y']])\ncheck('singleton group omitted',solve([[p,'a',N],[q,'x',1],[q,'y',0]]),[[q,'x','y']])\ncheck('empty',solve([]),[])\ncheck('single row',solve([[p,'a',N]]),[])\ncheck('tie lexical ids',solve([[p,'a',N],[p,'z',N]]),[[p,'z','a']])\ncheck('input group order',solve([[q,'b',N],[p,'a',N+1],[q,'a',0]]),[[q,'b','a']])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"These are explicit toy record contracts; they do not claim compatibility with a trainer or improved model behavior. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-preference-pair-construction-prompt-grouping","generated_at":"2026-09-30T16:48:31.262459+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A small deterministic preference-data transform makes pair semantics inspectable without a training framework.","root_cause":"A global score sort ignores the prompt identity that defines comparability.","sha256":"aca9680eec188be6166219bf793c4d18595ec9e774ac12bf6402054af19020d8","title":"Preference pairs: responses from different prompts are compared · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":28.214,"exit_code":1,"observations":[{"actual":[["p1","a","b"],["q1","x","y"]],"check":"separate prompts","expected":[["p1","a","b"],["q1","x","y"]],"passed":true},{"actual":[["q1","x","y"]],"check":"singleton group omitted","expected":[["q1","x","y"]],"passed":true},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":[],"check":"single row","expected":[],"passed":true},{"actual":[["p1","a","a"]],"check":"tie lexical ids","expected":[["p1","z","a"]],"passed":false},{"actual":[["q1","b","a"]],"check":"input group order","expected":[["q1","b","a"]],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"separate prompts\", \"actual\": [[\"p1\", \"a\", \"b\"], [\"q1\", \"x\", \"y\"]], \"expected\": [[\"p1\", \"a\", \"b\"], [\"q1\", \"x\", \"y\"]], \"passed\": true}, {\"check\": \"singleton group omitted\", \"actual\": [[\"q1\", \"x\", \"y\"]], \"expected\": [[\"q1\", \"x\", \"y\"]], \"passed\": true}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"single row\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"tie lexical ids\", \"actual\": [[\"p1\", \"a\", \"a\"]], \"expected\": [[\"p1\", \"z\", \"a\"]], \"passed\": false}, {\"check\": \"input group order\", \"actual\": [[\"q1\", \"b\", \"a\"]], \"expected\": [[\"q1\", \"b\", \"a\"]], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":28.346,"exit_code":1,"observations":[{"actual":[["q1","x","b"]],"check":"separate prompts","expected":[["p1","a","b"],["q1","x","y"]],"passed":false},{"actual":[["q1","x","y"]],"check":"singleton group omitted","expected":[["q1","x","y"]],"passed":true},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":[],"check":"single row","expected":[],"passed":true},{"actual":[["p1","z","a"]],"check":"tie lexical ids","expected":[["p1","z","a"]],"passed":true},{"actual":[["p1","a","a"]],"check":"input group order","expected":[["q1","b","a"]],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"separate prompts\", \"actual\": [[\"q1\", \"x\", \"b\"]], \"expected\": [[\"p1\", \"a\", \"b\"], [\"q1\", \"x\", \"y\"]], \"passed\": false}, {\"check\": \"singleton group omitted\", \"actual\": [[\"q1\", \"x\", \"y\"]], \"expected\": [[\"q1\", \"x\", \"y\"]], \"passed\": true}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"single row\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"tie lexical ids\", \"actual\": [[\"p1\", \"z\", \"a\"]], \"expected\": [[\"p1\", \"z\", \"a\"]], \"passed\": true}, {\"check\": \"input group order\", \"actual\": [[\"p1\", \"a\", \"a\"]], \"expected\": [[\"q1\", \"b\", \"a\"]], \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}