FA-100126 / Preference pair construction / Open access

Preference pairs: responses from different sources are paired · case 01

responses from different sources are paired.

Verified by executionVariant 1 · 6 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Source provenance is ignored when candidates are paired for a single comparison.

THE FAILURE

Source provenance is ignored when candidates are paired for a single comparison.

Unsuccessful approach: Grouping by source without an explicit tie breaker makes equal-score endpoints depend on row order.

Case contract

Rows are [source,id,score]. Within each source emit its strict best/worst [source,best_id,worst_id] if at least two candidates; source order is first-seen and score ties break by ID.

Why this case matters

A small deterministic preference-data transform makes pair semantics inspectable without a training framework.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(rows):
    s=sorted(rows,key=lambda r:(r[2],r[1]))
    return [[s[-1][0],s[-1][1],s[0][1]]] if len(s)>1 else []
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
p='p'+str(N); q='q'+str(N)
check('cross-source excluded',solve([[p,'a',N],[q,'b',N+2]]),[])
check('same source pair',solve([[p,'a',N],[p,'b',0]]),[[p,'a','b']])
check('empty',solve([]),[])
check('singleton',solve([[p,'a',N]]),[])
check('multiple groups',solve([[p,'a',3],[p,'b',1],[q,'x',5],[q,'y',0]]),[[p,'a','b'],[q,'x','y']])
check('tie stable IDs',solve([[p,'a',2],[p,'z',2]]),[[p,'z','a']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
cross-source excluded	`[['q1', 'b', 'a']]`	`[]`	Failed
same source pair	`[['p1', 'a', 'b']]`	`[['p1', 'a', 'b']]`	Passed
empty	`[]`	`[]`	Passed
singleton	`[]`	`[]`	Passed
multiple groups	`[['q1', 'x', 'y']]`	`[['p1', 'a', 'b'], ['q1', 'x', 'y']]`	Failed
tie stable IDs	`[['p1', 'z', 'a']]`	`[['p1', 'z', 'a']]`	Passed

SHA-256 / f9814ea108d0102d4d510ed5cef793f2169cf38a8daae59c922cbfc6b0ea304b

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(rows):
    groups={}
    for r in rows: groups.setdefault(r[0],[]).append(r)
    return [[k,max(v,key=lambda x:x[2])[1],min(v,key=lambda x:x[2])[1]] for k,v in groups.items() if len(v)>1]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
p='p'+str(N); q='q'+str(N)
check('cross-source excluded',solve([[p,'a',N],[q,'b',N+2]]),[])
check('same source pair',solve([[p,'a',N],[p,'b',0]]),[[p,'a','b']])
check('empty',solve([]),[])
check('singleton',solve([[p,'a',N]]),[])
check('multiple groups',solve([[p,'a',3],[p,'b',1],[q,'x',5],[q,'y',0]]),[[p,'a','b'],[q,'x','y']])
check('tie stable IDs',solve([[p,'a',2],[p,'z',2]]),[[p,'z','a']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
cross-source excluded	`[]`	`[]`	Passed
same source pair	`[['p1', 'a', 'b']]`	`[['p1', 'a', 'b']]`	Passed
empty	`[]`	`[]`	Passed
singleton	`[]`	`[]`	Passed
multiple groups	`[['p1', 'a', 'b'], ['q1', 'x', 'y']]`	`[['p1', 'a', 'b'], ['q1', 'x', 'y']]`	Passed
tie stable IDs	`[['p1', 'a', 'a']]`	`[['p1', 'z', 'a']]`	Failed

SHA-256 / 8efc87e1791b1adb5a831a7f644be481e2852123f707a4681968d43b5dc1853b

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 6 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Verification & scope

These are explicit toy record contracts; they do not claim compatibility with a trainer or improved model behavior. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-30T16:48:32.171217+00:00.

Case digest / d9b5cdf82fd5c8bfe1b108a9009a185806fbec5e2707f6c07e2d79097f1f5186