FA-74784 / Experiment statistics / Member archive

Team-draft interleaving credit: Clicks credit every ranker containing the document · case 04

Clicks on documents both rankers returned count for both, masking real preferences.

Member previewVariant 4 · 3 implementations · 8 checks per implementation

Case contract

Build an interleaved list of up to length documents. Each round both teams skip documents already placed; when both teams have the same number of picks, the next coin (cycled, consumed only on ties) chooses A on 0 and B on 1; otherwise the team with fewer picks goes. An exhausted team yields to the other. Clicks are positions; each credits the team that placed that document. Return [documents, teams, winner A/B/tie].

Why this case matters

Interleaving compares ranking algorithms with far fewer users than an A/B test, if credit is fair.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
shared top document is placed once	`[["d1", "d3", "d4", "d2"], ["A", "B", "B", "A"], "tie"]`	`[["d1", "d3", "d4", "d2"], ["A", "B", "B", "A"], "B"]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.