FA-74786 / Experiment statistics / Open access
Always-valid sequential p-value: The p-value can rise again · case 01
A result that was significant yesterday becomes insignificant today, breaking always-valid guarantees.
ROOT CAUSE
Each step uses min(1, 1 / ratio) without carrying the previous minimum.
VERIFIED REPAIR
Take the running minimum of 1 / ratio.
Unsuccessful approach: Clipping the ratio at one still lets the p-value rise after a reversal.
Case contract
diffs is a stream of paired differences with known variance sigma2; the normal-mixture likelihood ratio after n observations with running mean m is sqrt(sigma2 / (sigma2 + n tau2)) exp(n^2 tau2 m^2 / (2 sigma2 (sigma2 + n tau2))). The always-valid p-value starts at 1 and is the running minimum of 1 / ratio. Return the p-value after each observation rounded to 6.
Why this case matters
Always-valid p-values let teams monitor continuously without inflating false positives.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import math
N = 1
observations = []
def solve(diffs, sigma2, tau2):
p = 1.0
total = 0.0
out = []
for n, d in enumerate(diffs, 1):
total += d
mean = total / n
lam = math.sqrt(sigma2 / (sigma2 + n * tau2)) * math.exp(n * n * tau2 * mean * mean / (2 * sigma2 * (sigma2 + n * tau2)))
p = min(1.0, 1 / lam)
out.append(round(p, 6))
return out
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 1',
[[0.0, 1.5, 0.0, -1.0, 0.0, 0.5, -1.0], 2.0, 0.5],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 2',
[[0.0, 2.0, -1.0, 1.0, 0.5], 1.0, 1.0],
[1.0, 0.889265, 0.889265, 0.889265, 0.889265]),
('difference stream sample 3',
[[1.0, -1.0, -1.0, 0.5, -1.0, 0.5, 1.5], 1.0, 0.25],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 6', [[0.0, -1.0, 0.0, -1.0, 0.5], 4.0, 0.25], [1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 7', [[2.0, 1.0, 2.0, 0.0], 1.0, 0.5], [0.628805, 0.459128, 0.129788, 0.129788]),
('difference stream sample 28',
[[2.0, 0.0, -1.0, 0.5, 0.0, 1.0, -1.0], 1.0, 1.0],
[0.52026, 0.52026, 0.52026, 0.52026, 0.52026, 0.52026, 0.52026])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 11', [[2.0], 4.0, 1.0], [1.0]),
('difference stream sample 12', [[0.0, 2.0, 0.0], 2.0, 0.25], [1.0, 1.0, 1.0]),
('difference stream sample 59',
[[0.5, 0.5, 1.0, 0.0, 1.0, 0.0], 1.0, 0.5],
[1.0, 1.0, 1.0, 1.0, 0.983659, 0.983659])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 16', [[1.0], 1.0, 0.25], [1.0]),
('difference stream sample 17',
[[1.5, 0.5, 2.0, 0.5, 0.0], 1.0, 0.5],
[0.841754, 0.841754, 0.319226, 0.319226, 0.319226]),
('difference stream sample 18', [[0.0], 4.0, 1.0], [1.0])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 21', [[0.0, -1.0, 1.0, 2.0, 1.5], 2.0, 0.25], [1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 22', [[1.0, 0.5, 1.5], 2.0, 0.25], [1.0, 1.0, 0.955693]),
('difference stream sample 58', [[2.0, 0.0, 2.0], 1.0, 1.0], [0.52026, 0.52026, 0.270671])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| p-value never increases after a reversal | [0.52026, 0.120349, 0.649305, 1.0] | [0.52026, 0.120349, 0.120349, 0.120349] | Failed |
| first observation | [1.0] | [1.0] | Passed |
| null-looking stream stays at one | [1.0, 1.0, 1.0] | [1.0, 1.0, 1.0] | Passed |
| steady effect | [1.0, 1.0, 0.959234, 0.857764, 0.749028] | [1.0, 1.0, 0.959234, 0.857764, 0.749028] | Passed |
| noisy stream | [1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0] | Passed |
| difference stream sample 1 | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | Passed |
| difference stream sample 2 | [1.0, 0.889265, 1.0, 1.0, 1.0] | [1.0, 0.889265, 0.889265, 0.889265, 0.889265] | Failed |
| difference stream sample 3 | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | Passed |
SHA-256 / 7e73b528808012f8d14610715ff50141c88d1a9971f117c90ce7b56fb2e3dd88
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
import math
N = 1
observations = []
def solve(diffs, sigma2, tau2):
p = 1.0
total = 0.0
out = []
for n, d in enumerate(diffs, 1):
total += d
mean = total / n
lam = math.sqrt(sigma2 / (sigma2 + n * tau2)) * math.exp(n * n * tau2 * mean * mean / (2 * sigma2 * (sigma2 + n * tau2)))
p = 1 / max(lam, 1.0)
out.append(round(p, 6))
return out
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 1',
[[0.0, 1.5, 0.0, -1.0, 0.0, 0.5, -1.0], 2.0, 0.5],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 2',
[[0.0, 2.0, -1.0, 1.0, 0.5], 1.0, 1.0],
[1.0, 0.889265, 0.889265, 0.889265, 0.889265]),
('difference stream sample 3',
[[1.0, -1.0, -1.0, 0.5, -1.0, 0.5, 1.5], 1.0, 0.25],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 6', [[0.0, -1.0, 0.0, -1.0, 0.5], 4.0, 0.25], [1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 7', [[2.0, 1.0, 2.0, 0.0], 1.0, 0.5], [0.628805, 0.459128, 0.129788, 0.129788]),
('difference stream sample 28',
[[2.0, 0.0, -1.0, 0.5, 0.0, 1.0, -1.0], 1.0, 1.0],
[0.52026, 0.52026, 0.52026, 0.52026, 0.52026, 0.52026, 0.52026])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 11', [[2.0], 4.0, 1.0], [1.0]),
('difference stream sample 12', [[0.0, 2.0, 0.0], 2.0, 0.25], [1.0, 1.0, 1.0]),
('difference stream sample 59',
[[0.5, 0.5, 1.0, 0.0, 1.0, 0.0], 1.0, 0.5],
[1.0, 1.0, 1.0, 1.0, 0.983659, 0.983659])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 16', [[1.0], 1.0, 0.25], [1.0]),
('difference stream sample 17',
[[1.5, 0.5, 2.0, 0.5, 0.0], 1.0, 0.5],
[0.841754, 0.841754, 0.319226, 0.319226, 0.319226]),
('difference stream sample 18', [[0.0], 4.0, 1.0], [1.0])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 21', [[0.0, -1.0, 1.0, 2.0, 1.5], 2.0, 0.25], [1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 22', [[1.0, 0.5, 1.5], 2.0, 0.25], [1.0, 1.0, 0.955693]),
('difference stream sample 58', [[2.0, 0.0, 2.0], 1.0, 1.0], [0.52026, 0.52026, 0.270671])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| p-value never increases after a reversal | [0.52026, 0.120349, 0.649305, 1.0] | [0.52026, 0.120349, 0.120349, 0.120349] | Failed |
| first observation | [1.0] | [1.0] | Passed |
| null-looking stream stays at one | [1.0, 1.0, 1.0] | [1.0, 1.0, 1.0] | Passed |
| steady effect | [1.0, 1.0, 0.959234, 0.857764, 0.749028] | [1.0, 1.0, 0.959234, 0.857764, 0.749028] | Passed |
| noisy stream | [1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0] | Passed |
| difference stream sample 1 | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | Passed |
| difference stream sample 2 | [1.0, 0.889265, 1.0, 1.0, 1.0] | [1.0, 0.889265, 0.889265, 0.889265, 0.889265] | Failed |
| difference stream sample 3 | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | Passed |
SHA-256 / e64caa3365ec34ccc7a15f2fa445b97195681c10fdc15c410085982df75f953b
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
import math
N = 1
observations = []
def solve(diffs, sigma2, tau2):
p = 1.0
total = 0.0
out = []
for n, d in enumerate(diffs, 1):
total += d
mean = total / n
lam = math.sqrt(sigma2 / (sigma2 + n * tau2)) * math.exp(n * n * tau2 * mean * mean / (2 * sigma2 * (sigma2 + n * tau2)))
p = min(p, 1 / lam)
out.append(round(p, 6))
return out
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 1',
[[0.0, 1.5, 0.0, -1.0, 0.0, 0.5, -1.0], 2.0, 0.5],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 2',
[[0.0, 2.0, -1.0, 1.0, 0.5], 1.0, 1.0],
[1.0, 0.889265, 0.889265, 0.889265, 0.889265]),
('difference stream sample 3',
[[1.0, -1.0, -1.0, 0.5, -1.0, 0.5, 1.5], 1.0, 0.25],
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 6', [[0.0, -1.0, 0.0, -1.0, 0.5], 4.0, 0.25], [1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 7', [[2.0, 1.0, 2.0, 0.0], 1.0, 0.5], [0.628805, 0.459128, 0.129788, 0.129788]),
('difference stream sample 28',
[[2.0, 0.0, -1.0, 0.5, 0.0, 1.0, -1.0], 1.0, 1.0],
[0.52026, 0.52026, 0.52026, 0.52026, 0.52026, 0.52026, 0.52026])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 11', [[2.0], 4.0, 1.0], [1.0]),
('difference stream sample 12', [[0.0, 2.0, 0.0], 2.0, 0.25], [1.0, 1.0, 1.0]),
('difference stream sample 59',
[[0.5, 0.5, 1.0, 0.0, 1.0, 0.0], 1.0, 0.5],
[1.0, 1.0, 1.0, 1.0, 0.983659, 0.983659])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 16', [[1.0], 1.0, 0.25], [1.0]),
('difference stream sample 17',
[[1.5, 0.5, 2.0, 0.5, 0.0], 1.0, 0.5],
[0.841754, 0.841754, 0.319226, 0.319226, 0.319226]),
('difference stream sample 18', [[0.0], 4.0, 1.0], [1.0])],
[('p-value never increases after a reversal',
[[2.0, 2.0, -1.0, -1.0], 1.0, 1.0],
[0.52026, 0.120349, 0.120349, 0.120349]),
('first observation', [[1.0], 1.0, 1.0], [1.0]),
('null-looking stream stays at one', [[0.0, 0.0, 0.0], 1.0, 0.5], [1.0, 1.0, 1.0]),
('steady effect', [[1.0, 1.0, 1.0, 1.0, 1.0], 2.0, 0.5], [1.0, 1.0, 0.959234, 0.857764, 0.749028]),
('noisy stream', [[1.5, -1.0, 2.0, 0.5], 4.0, 1.0], [1.0, 1.0, 1.0, 1.0]),
('difference stream sample 21', [[0.0, -1.0, 1.0, 2.0, 1.5], 2.0, 0.25], [1.0, 1.0, 1.0, 1.0, 1.0]),
('difference stream sample 22', [[1.0, 0.5, 1.5], 2.0, 0.25], [1.0, 1.0, 0.955693]),
('difference stream sample 58', [[2.0, 0.0, 2.0], 1.0, 1.0], [0.52026, 0.52026, 0.270671])]]
for label, args, expected in fixtures[N - 1]:
check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| p-value never increases after a reversal | [0.52026, 0.120349, 0.120349, 0.120349] | [0.52026, 0.120349, 0.120349, 0.120349] | Passed |
| first observation | [1.0] | [1.0] | Passed |
| null-looking stream stays at one | [1.0, 1.0, 1.0] | [1.0, 1.0, 1.0] | Passed |
| steady effect | [1.0, 1.0, 0.959234, 0.857764, 0.749028] | [1.0, 1.0, 0.959234, 0.857764, 0.749028] | Passed |
| noisy stream | [1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0] | Passed |
| difference stream sample 1 | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | Passed |
| difference stream sample 2 | [1.0, 0.889265, 0.889265, 0.889265, 0.889265] | [1.0, 0.889265, 0.889265, 0.889265, 0.889265] | Passed |
| difference stream sample 3 | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0] | Passed |
SHA-256 / cc7052b00953c6f37639b77758a14df183ea4f34f15f9cd2e145d74477687eb7
Verification & scope
A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:48:59.858132+00:00.
Case digest / aca24f132b760744c04058e160a1652a73321e6276093c5b3dd4a345cc65e783