FA-74397 / Experiment statistics / Member archive

Welch unequal-variance t statistic: Welch df divides by arm sizes · case 02

Degrees of freedom are inflated, making small-sample p-values too small.

Member previewVariant 2 · 3 implementations · 8 checks per implementation

Case contract

a is control, b treatment. With sample variances (divisor n - 1), t = (mean_b - mean_a) / sqrt(va/na + vb/nb) and Welch-Satterthwaite df = (va/na + vb/nb)^2 / ((va/na)^2/(na-1) + (vb/nb)^2/(nb-1)). Fewer than two values in an arm or zero total variance -> None. Return [round(t, 6), round(df, 6)].

Why this case matters

Online experiment readouts drive launch decisions; a silent formula slip flips conclusions.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
two-point arms	`[2.683282, 2.941176]`	`[2.683282, 1.470588]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.