FA-74677 / Experiment statistics / Member archive

Rank-sum test with ties: Variance ignores ties · case 02

z-scores are too small on bucketed metrics with many ties.

Member previewVariant 2 · 3 implementations · 8 checks per implementation

Case contract

Pool a (control) and b (treatment), assign mid-ranks to ties (1-based). U = R_b - n_b(n_b + 1)/2. Tie-corrected variance = n_a n_b / 12 * ((N + 1) - sum(t^3 - t) / (N(N - 1))); z = (U - n_a n_b / 2) / sqrt(variance) without continuity correction; zero variance gives z = 0. Empty arm -> None. Return [U, round(z, 6)].

Why this case matters

Rank tests are used for heavy-tailed metrics such as latency; ties are common in bucketed data.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
two-way tie across arms	`[3.5, 1.161895]`	`[3.5, 1.224745]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.