FAILURE MAP
← Case archive

FA-74805 / Experiment statistics / Member archive

Always-valid sequential p-value: The ratio uses the latest observation instead of the mean · case 05

One noisy observation swings the evidence regardless of history.

Member previewVariant 5 · 3 implementations · 8 checks per implementation

Case contract

diffs is a stream of paired differences with known variance sigma2; the normal-mixture likelihood ratio after n observations with running mean m is sqrt(sigma2 / (sigma2 + n tau2)) exp(n^2 tau2 m^2 / (2 sigma2 (sigma2 + n tau2))). The always-valid p-value starts at 1 and is the running minimum of 1 / ratio. Return the p-value after each observation rounded to 6.

Why this case matters

Always-valid p-values let teams monitor continuously without inflating false positives.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixtureActualExpectedOutcome
noisy stream[1.0, 1.0, 0.695552, 0.695552][1.0, 1.0, 1.0, 1.0]Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.

Sign in to the archive ↗