FA-74392 / Experiment statistics / Member archive

Welch unequal-variance t statistic: Arm variances use the population divisor · case 02

Small arms get understated variance and overconfident t statistics.

Member previewVariant 2 · 3 implementations · 8 checks per implementation

Case contract

a is control, b treatment. With sample variances (divisor n - 1), t = (mean_b - mean_a) / sqrt(va/na + vb/nb) and Welch-Satterthwaite df = (va/na + vb/nb)^2 / ((va/na)^2/(na-1) + (vb/nb)^2/(nb-1)). Fewer than two values in an arm or zero total variance -> None. Return [round(t, 6), round(df, 6)].

Why this case matters

Online experiment readouts drive launch decisions; a silent formula slip flips conclusions.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
two-point arms	`[3.794733, 1.470588]`	`[2.683282, 1.470588]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.