FA-74564 / Experiment statistics / Member archive

Revenue outlier capping: The cap is computed from the control arm alone · case 04

A treatment that creates more big spenders gets its gains clipped by a control-based cap.

Member previewVariant 4 · 3 implementations · 8 checks per implementation

Case contract

The cap is the nearest-rank pct-th percentile of the pooled values of both arms: sorted pooled value at 1-based rank ceil(pct/100 * N), clamped to [1, N]. Every value is capped (not dropped) at that threshold and each arm mean is taken over all its users. Empty arm -> None. Return [cap, capped control mean, capped treatment mean].

Why this case matters

Capping whales keeps revenue metrics sensitive; a per-arm cap biases the comparison itself.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
cap comes from pooled data	`[10, 6.25, 7.5]`	`[20, 8.75, 12.5]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.