FA-74515 / Experiment statistics / Member archive

Per-arm sample size for conversion tests: Only the baseline variance is used · case 05

Plans ignore that the treatment rate has a different variance.

Member previewVariant 5 · 3 implementations · 8 checks per implementation

Case contract

p2 = p_base (1 + mde) when relative else p_base + mde; both rates must lie in (0, 1) and differ (else None). n per arm = ceil((z_{1-alpha/2} + z_power)^2 (p1(1-p1) + p2(1-p2)) / (p2 - p1)^2) using statistics.NormalDist quantiles.

Why this case matters

Underpowered experiments waste traffic; overestimates delay launches.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
ten percent relative lift on five percent	`29826`	`31231`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.