FA-74355 / Experiment statistics / Member archive

Sample ratio mismatch check: Zero-weight arms count toward degrees of freedom · case 05

Experiments with a drained arm use a looser critical value and miss real mismatches.

Member previewVariant 5 · 3 implementations · 8 checks per implementation

Case contract

counts and weights are per-arm lists. Expected count = total * w / sum(weights). chi2 sums (observed - expected)^2 / expected over positive-weight arms; any unit in a zero-weight arm is an immediate mismatch [None, True]. df = number of positive-weight arms - 1; mismatch iff chi2 exceeds the alpha = 0.001 critical value (10.828, 13.816, 16.266, 18.467, 20.515 for df 1..5). No units or df < 1 -> [0.0, False]. Return [round(chi2, 6), mismatch].

Why this case matters

SRM checks are the first gate on any experiment readout; a broken check hides assignment bugs.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
drained arm does not loosen the threshold	`[11.56, false]`	`[11.56, true]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.