FA-74414 / Experiment statistics / Member archive

Relative lift interval: Control noise is scaled like treatment noise · case 04

Intervals are too narrow whenever the lift is large.

Member previewVariant 4 · 3 implementations · 8 checks per implementation

Case contract

lift = mean_t / mean_c - 1. With variances of the means v_t = var_t/n_t and v_c = var_c/n_c, the delta-method variance is v_t/mean_c^2 + mean_t^2 v_c / mean_c^4 and the interval is lift +/- z * sqrt(variance). mean_c = 0 or nonpositive n -> None. Return [lift, low, high] rounded to 6 places.

Why this case matters

Online experiment readouts drive launch decisions; a silent formula slip flips conclusions.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
ten percent lift	`[0.1, 0.072281, 0.127719]`	`[0.1, 0.070863, 0.129137]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.