FA-74484 / Experiment statistics / Member archive

Holm correction across metrics: Every step uses the Bonferroni threshold · case 04

Metrics that Holm would reject after the first step stay unrejected.

Member previewVariant 4 · 3 implementations · 8 checks per implementation

Case contract

Holm step-down: sort p-values ascending (ties by position); reject while p_(k) <= alpha / (m - k) for k = 0, 1, ... and stop at the first failure. Adjusted p_(k) = running max of min(1, (m - k) p_(k)). Return [rejections, adjusted p-values rounded to 6] in the original metric order.

Why this case matters

Experiments track many metrics; multiplicity control keeps secondary wins honest.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
original order is preserved	`[[false, true, false], [0.3, 0.003, 0.04]]`	`[[false, true, true], [0.3, 0.003, 0.04]]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.