FA-74707 / Experiment statistics / Member archive

Cluster randomisation design effect: Unequal cluster sizes are ignored · case 02

Design effects are understated when a few accounts are much larger than the rest.

Member previewVariant 2 · 3 implementations · 8 checks per implementation

Case contract

For clusters (e.g. companies) randomised as units: mean size m, population variance of sizes / m^2 = cv^2, design effect = 1 + ((cv^2 + 1) m - 1) icc. Effective sample = total units / design effect; clusters needed = ceil(n_required * design effect / m). No clusters or zero mean -> None. Return [round(deff, 6), round(effective n, 6), clusters needed].

Why this case matters

B2B experiments randomise by account; ignoring clustering makes results look far more certain.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
unequal cluster sizes inflate the design effect	`[1.95, 41.025641, 49]`	`[2.54375, 31.449631, 64]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.