{"abstract":"stale annotation revisions remain active.","category":"Preference pair construction","checks":6,"contract":"Rows are [pair_id,revision,chosen,rejected,valid], with unique revisions per pair. Latest revision wins regardless of validity; emit chosen/rejected only when that latest row is valid, otherwise None. Pair order is first-seen.","contract_signature":"rows","evaluation_group":"preference-latest-annotation","failed_approach":"Selecting the latest winning revision ignores a later tie or reversal.","family":"w3-preference-pair-construction-latest-annotation","id":"FA-100111","implementations":{"attempt":{"sha256":"a94f59352e52702f8b5ebef43206307f646907f1933c655ceec776bd067d1f24","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    latest={}\n    for p,v,c,r,ok in rows:\n     if ok and (p not in latest or v>latest[p][0]): latest[p]=(v,c,r)\n    return [[p,*latest[p][1:]] for p in latest]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np='p'+str(N); q='q'+str(N)\ncheck('later reversal',solve([[p,1,'a','b',True],[p,2,'b','a',True]]),[[p,'b','a']])\ncheck('later invalid',solve([[p,1,'a','b',True],[p,2,'x','y',False]]),[[p,None,None]])\ncheck('older invalid later valid',solve([[p,1,'x','y',False],[p,2,'a','b',True]]),[[p,'a','b']])\ncheck('empty',solve([]),[])\ncheck('independent pair',solve([[q,2,'c','d',True],[p,1,'a','b',True]]),[[q,'c','d'],[p,'a','b']])\ncheck('revision order arbitrary',solve([[p,3,'c','d',True],[p,1,'a','b',True]]),[[p,'c','d']])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"4a1958282ef89d60e3c7a91a2f0a20afd803c07dab545f7b692b81296657bf5c","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    out=[]; seen=set()\n    for p,v,c,r,ok in rows:\n     if p not in seen: seen.add(p); out.append([p,c,r] if ok else [p,None,None])\n    return out\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\np='p'+str(N); q='q'+str(N)\ncheck('later reversal',solve([[p,1,'a','b',True],[p,2,'b','a',True]]),[[p,'b','a']])\ncheck('later invalid',solve([[p,1,'a','b',True],[p,2,'x','y',False]]),[[p,None,None]])\ncheck('older invalid later valid',solve([[p,1,'x','y',False],[p,2,'a','b',True]]),[[p,'a','b']])\ncheck('empty',solve([]),[])\ncheck('independent pair',solve([[q,2,'c','d',True],[p,1,'a','b',True]]),[[q,'c','d'],[p,'a','b']])\ncheck('revision order arbitrary',solve([[p,3,'c','d',True],[p,1,'a','b',True]]),[[p,'c','d']])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"These are explicit toy record contracts; they do not claim compatibility with a trainer or improved model behavior. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-preference-pair-construction-latest-annotation","generated_at":"2026-09-30T16:48:31.716001+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A small deterministic preference-data transform makes pair semantics inspectable without a training framework.","root_cause":"All annotation revisions are pooled, so an old preference survives a correction.","sha256":"34355a1f9d720cf81a1ab384ab9eeb35c01451d837d58ac490e97e39ae36863f","title":"Preference pairs: stale annotation revisions remain active · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":28.733,"exit_code":1,"observations":[{"actual":[["p1","b","a"]],"check":"later reversal","expected":[["p1","b","a"]],"passed":true},{"actual":[["p1","a","b"]],"check":"later invalid","expected":[["p1",null,null]],"passed":false},{"actual":[["p1","a","b"]],"check":"older invalid later valid","expected":[["p1","a","b"]],"passed":true},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":[["q1","c","d"],["p1","a","b"]],"check":"independent pair","expected":[["q1","c","d"],["p1","a","b"]],"passed":true},{"actual":[["p1","c","d"]],"check":"revision order arbitrary","expected":[["p1","c","d"]],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"later reversal\", \"actual\": [[\"p1\", \"b\", \"a\"]], \"expected\": [[\"p1\", \"b\", \"a\"]], \"passed\": true}, {\"check\": \"later invalid\", \"actual\": [[\"p1\", \"a\", \"b\"]], \"expected\": [[\"p1\", null, null]], \"passed\": false}, {\"check\": \"older invalid later valid\", \"actual\": [[\"p1\", \"a\", \"b\"]], \"expected\": [[\"p1\", \"a\", \"b\"]], \"passed\": true}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"independent pair\", \"actual\": [[\"q1\", \"c\", \"d\"], [\"p1\", \"a\", \"b\"]], \"expected\": [[\"q1\", \"c\", \"d\"], [\"p1\", \"a\", \"b\"]], \"passed\": true}, {\"check\": \"revision order arbitrary\", \"actual\": [[\"p1\", \"c\", \"d\"]], \"expected\": [[\"p1\", \"c\", \"d\"]], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":26.993,"exit_code":1,"observations":[{"actual":[["p1","a","b"]],"check":"later reversal","expected":[["p1","b","a"]],"passed":false},{"actual":[["p1","a","b"]],"check":"later invalid","expected":[["p1",null,null]],"passed":false},{"actual":[["p1",null,null]],"check":"older invalid later valid","expected":[["p1","a","b"]],"passed":false},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":[["q1","c","d"],["p1","a","b"]],"check":"independent pair","expected":[["q1","c","d"],["p1","a","b"]],"passed":true},{"actual":[["p1","c","d"]],"check":"revision order arbitrary","expected":[["p1","c","d"]],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"later reversal\", \"actual\": [[\"p1\", \"a\", \"b\"]], \"expected\": [[\"p1\", \"b\", \"a\"]], \"passed\": false}, {\"check\": \"later invalid\", \"actual\": [[\"p1\", \"a\", \"b\"]], \"expected\": [[\"p1\", null, null]], \"passed\": false}, {\"check\": \"older invalid later valid\", \"actual\": [[\"p1\", null, null]], \"expected\": [[\"p1\", \"a\", \"b\"]], \"passed\": false}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"independent pair\", \"actual\": [[\"q1\", \"c\", \"d\"], [\"p1\", \"a\", \"b\"]], \"expected\": [[\"q1\", \"c\", \"d\"], [\"p1\", \"a\", \"b\"]], \"passed\": true}, {\"check\": \"revision order arbitrary\", \"actual\": [[\"p1\", \"c\", \"d\"]], \"expected\": [[\"p1\", \"c\", \"d\"]], \"passed\": true}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}