FA-74411 / Experiment statistics / Open access

Relative lift interval: Control noise is scaled like treatment noise · case 01

Intervals are too narrow whenever the lift is large.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The control term is v_c / mean_c^2, dropping the mean_t^2 / mean_c^2 factor.

VERIFIED REPAIR

Use mean_t^2 * v_c / mean_c^4 for the control term.

Unsuccessful approach: Using mean_t * v_c / mean_c^3 applies only one power of the ratio.

Case contract

lift = mean_t / mean_c - 1. With variances of the means v_t = var_t/n_t and v_c = var_c/n_c, the delta-method variance is v_t/mean_c^2 + mean_t^2 v_c / mean_c^4 and the interval is lift +/- z * sqrt(variance). mean_c = 0 or nonpositive n -> None. Return [lift, low, high] rounded to 6 places.

Why this case matters

Online experiment readouts drive launch decisions; a silent formula slip flips conclusions.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import math
N = 1
observations = []
def solve(mean_c, var_c, n_c, mean_t, var_t, n_t, z):
    if mean_c == 0 or n_c <= 0 or n_t <= 0:
        return None
    lift = mean_t / mean_c - 1
    se_t = var_t / n_t
    se_c = var_c / n_c
    v = se_t / mean_c ** 2 + se_c / mean_c ** 2
    half = z * math.sqrt(v)
    return [round(lift, 6), round(lift - half, 6), round(lift + half, 6)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 1', [4.0, 4.0, 1000, 0.45, 16.0, 250, 1.96], [-0.8875, -1.01151, -0.76349]),
  ('summary statistic sample 2', [10.0, 9.0, 400, 3.0, 1.0, 250, 1.645], [-0.7, -0.712769, -0.687231]),
  ('summary statistic sample 3', [2.0, 1.0, 400, 4.0, 1.0, 250, 1.645], [1.0, 0.90268, 1.09732])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 5', [10.0, 4.0, 400, 4.0, 4.0, 1000, 1.645], [-0.6, -0.61231, -0.58769]),
  ('summary statistic sample 6', [10.0, 9.0, 1000, 2.2, 4.0, 1000, 1.645], [-0.78, -0.790956, -0.769044]),
  ('summary statistic sample 7', [2.0, 9.0, 400, 10.5, 1.0, 100, 1.645], [4.25, 3.59708, 4.90292])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 11', [4.0, 4.0, 400, 10.5, 16.0, 1000, 1.96], [1.625, 1.48222, 1.76778]),
  ('summary statistic sample 12', [10.0, 9.0, 400, 0.45, 1.0, 250, 1.96], [-0.955, -0.967467, -0.942533]),
  ('summary statistic sample 13', [0.5, 1.0, 1000, 10.5, 4.0, 100, 1.645], [20.0, 17.718248, 22.281752])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 16', [4.0, 9.0, 100, 4.0, 4.0, 250, 1.645], [0.0, -0.133893, 0.133893]),
  ('summary statistic sample 17', [4.0, 1.0, 100, 4.0, 16.0, 1000, 1.645], [0.0, -0.066312, 0.066312]),
  ('summary statistic sample 22', [0.5, 4.0, 1000, 10.5, 4.0, 1000, 1.96], [20.0, 14.787726, 25.212274])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 21', [0.5, 1.0, 400, 0.45, 1.0, 100, 1.96], [-0.1, -0.529862, 0.329862]),
  ('summary statistic sample 22', [0.5, 4.0, 1000, 10.5, 4.0, 1000, 1.96], [20.0, 14.787726, 25.212274]),
  ('summary statistic sample 31', [0.5, 1.0, 100, 4.0, 16.0, 1000, 1.96], [7.0, 3.825042, 10.174958])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
ten percent lift	`[0.1, 0.072281, 0.127719]`	`[0.1, 0.070863, 0.129137]`	Failed
treatment below control	`[-0.25, -0.388593, -0.111407]`	`[-0.25, -0.3725, -0.1275]`	Failed
equal means still carry control noise	`[0.0, -0.041125, 0.041125]`	`[0.0, -0.041125, 0.041125]`	Passed
small means	`[0.2, 0.016136, 0.383864]`	`[0.2, -0.001413, 0.401413]`	Failed
zero control mean	`None`	`None`	Passed
summary statistic sample 1	`[-0.8875, -1.015276, -0.759724]`	`[-0.8875, -1.01151, -0.76349]`	Failed
summary statistic sample 2	`[-0.7, -0.726779, -0.673221]`	`[-0.7, -0.712769, -0.687231]`	Failed
summary statistic sample 3	`[1.0, 0.933688, 1.066312]`	`[1.0, 0.90268, 1.09732]`	Failed

SHA-256 / 729b876b47db506fbe44efd6263f87cda73feece42d483f86fd64ab05fd3aa29

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import math
N = 1
observations = []
def solve(mean_c, var_c, n_c, mean_t, var_t, n_t, z):
    if mean_c == 0 or n_c <= 0 or n_t <= 0:
        return None
    lift = mean_t / mean_c - 1
    se_t = var_t / n_t
    se_c = var_c / n_c
    v = se_t / mean_c ** 2 + mean_t * se_c / mean_c ** 3
    half = z * math.sqrt(v)
    return [round(lift, 6), round(lift - half, 6), round(lift + half, 6)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 1', [4.0, 4.0, 1000, 0.45, 16.0, 250, 1.96], [-0.8875, -1.01151, -0.76349]),
  ('summary statistic sample 2', [10.0, 9.0, 400, 3.0, 1.0, 250, 1.645], [-0.7, -0.712769, -0.687231]),
  ('summary statistic sample 3', [2.0, 1.0, 400, 4.0, 1.0, 250, 1.645], [1.0, 0.90268, 1.09732])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 5', [10.0, 4.0, 400, 4.0, 4.0, 1000, 1.645], [-0.6, -0.61231, -0.58769]),
  ('summary statistic sample 6', [10.0, 9.0, 1000, 2.2, 4.0, 1000, 1.645], [-0.78, -0.790956, -0.769044]),
  ('summary statistic sample 7', [2.0, 9.0, 400, 10.5, 1.0, 100, 1.645], [4.25, 3.59708, 4.90292])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 11', [4.0, 4.0, 400, 10.5, 16.0, 1000, 1.96], [1.625, 1.48222, 1.76778]),
  ('summary statistic sample 12', [10.0, 9.0, 400, 0.45, 1.0, 250, 1.96], [-0.955, -0.967467, -0.942533]),
  ('summary statistic sample 13', [0.5, 1.0, 1000, 10.5, 4.0, 100, 1.645], [20.0, 17.718248, 22.281752])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 16', [4.0, 9.0, 100, 4.0, 4.0, 250, 1.645], [0.0, -0.133893, 0.133893]),
  ('summary statistic sample 17', [4.0, 1.0, 100, 4.0, 16.0, 1000, 1.645], [0.0, -0.066312, 0.066312]),
  ('summary statistic sample 22', [0.5, 4.0, 1000, 10.5, 4.0, 1000, 1.96], [20.0, 14.787726, 25.212274])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 21', [0.5, 1.0, 400, 0.45, 1.0, 100, 1.96], [-0.1, -0.529862, 0.329862]),
  ('summary statistic sample 22', [0.5, 4.0, 1000, 10.5, 4.0, 1000, 1.96], [20.0, 14.787726, 25.212274]),
  ('summary statistic sample 31', [0.5, 1.0, 100, 4.0, 16.0, 1000, 1.96], [7.0, 3.825042, 10.174958])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
ten percent lift	`[0.1, 0.071597, 0.128403]`	`[0.1, 0.070863, 0.129137]`	Failed
treatment below control	`[-0.25, -0.379642, -0.120358]`	`[-0.25, -0.3725, -0.1275]`	Failed
equal means still carry control noise	`[0.0, -0.041125, 0.041125]`	`[0.0, -0.041125, 0.041125]`	Passed
small means	`[0.2, 0.00796, 0.39204]`	`[0.2, -0.001413, 0.401413]`	Failed
zero control mean	`None`	`None`	Passed
summary statistic sample 1	`[-0.8875, -1.011896, -0.763104]`	`[-0.8875, -1.01151, -0.76349]`	Failed
summary statistic sample 2	`[-0.7, -0.717056, -0.682944]`	`[-0.7, -0.712769, -0.687231]`	Failed
summary statistic sample 3	`[1.0, 0.921971, 1.078029]`	`[1.0, 0.90268, 1.09732]`	Failed

SHA-256 / cc23813fb38d72d902ac9c72ef43ee6dcd2798fdafb58fc78c67165811c248f1

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json
import math
N = 1
observations = []
def solve(mean_c, var_c, n_c, mean_t, var_t, n_t, z):
    if mean_c == 0 or n_c <= 0 or n_t <= 0:
        return None
    lift = mean_t / mean_c - 1
    se_t = var_t / n_t
    se_c = var_c / n_c
    v = se_t / mean_c ** 2 + mean_t ** 2 * se_c / mean_c ** 4
    half = z * math.sqrt(v)
    return [round(lift, 6), round(lift - half, 6), round(lift + half, 6)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 1', [4.0, 4.0, 1000, 0.45, 16.0, 250, 1.96], [-0.8875, -1.01151, -0.76349]),
  ('summary statistic sample 2', [10.0, 9.0, 400, 3.0, 1.0, 250, 1.645], [-0.7, -0.712769, -0.687231]),
  ('summary statistic sample 3', [2.0, 1.0, 400, 4.0, 1.0, 250, 1.645], [1.0, 0.90268, 1.09732])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 5', [10.0, 4.0, 400, 4.0, 4.0, 1000, 1.645], [-0.6, -0.61231, -0.58769]),
  ('summary statistic sample 6', [10.0, 9.0, 1000, 2.2, 4.0, 1000, 1.645], [-0.78, -0.790956, -0.769044]),
  ('summary statistic sample 7', [2.0, 9.0, 400, 10.5, 1.0, 100, 1.645], [4.25, 3.59708, 4.90292])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 11', [4.0, 4.0, 400, 10.5, 16.0, 1000, 1.96], [1.625, 1.48222, 1.76778]),
  ('summary statistic sample 12', [10.0, 9.0, 400, 0.45, 1.0, 250, 1.96], [-0.955, -0.967467, -0.942533]),
  ('summary statistic sample 13', [0.5, 1.0, 1000, 10.5, 4.0, 100, 1.645], [20.0, 17.718248, 22.281752])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 16', [4.0, 9.0, 100, 4.0, 4.0, 250, 1.645], [0.0, -0.133893, 0.133893]),
  ('summary statistic sample 17', [4.0, 1.0, 100, 4.0, 16.0, 1000, 1.645], [0.0, -0.066312, 0.066312]),
  ('summary statistic sample 22', [0.5, 4.0, 1000, 10.5, 4.0, 1000, 1.96], [20.0, 14.787726, 25.212274])],
 [('ten percent lift', [10.0, 4.0, 400, 11.0, 4.0, 400, 1.96], [0.1, 0.070863, 0.129137]),
  ('treatment below control', [2.0, 1.0, 100, 1.5, 1.0, 100, 1.96], [-0.25, -0.3725, -0.1275]),
  ('equal means still carry control noise',
   [4.0, 9.0, 1000, 4.0, 1.0, 1000, 1.645],
   [0.0, -0.041125, 0.041125]),
  ('small means', [0.5, 0.25, 250, 0.6, 0.3, 250, 1.96], [0.2, -0.001413, 0.401413]),
  ('zero control mean', [0.0, 1.0, 10, 1.0, 1.0, 10, 1.96], None),
  ('summary statistic sample 21', [0.5, 1.0, 400, 0.45, 1.0, 100, 1.96], [-0.1, -0.529862, 0.329862]),
  ('summary statistic sample 22', [0.5, 4.0, 1000, 10.5, 4.0, 1000, 1.96], [20.0, 14.787726, 25.212274]),
  ('summary statistic sample 31', [0.5, 1.0, 100, 4.0, 16.0, 1000, 1.96], [7.0, 3.825042, 10.174958])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
ten percent lift	`[0.1, 0.070863, 0.129137]`	`[0.1, 0.070863, 0.129137]`	Passed
treatment below control	`[-0.25, -0.3725, -0.1275]`	`[-0.25, -0.3725, -0.1275]`	Passed
equal means still carry control noise	`[0.0, -0.041125, 0.041125]`	`[0.0, -0.041125, 0.041125]`	Passed
small means	`[0.2, -0.001413, 0.401413]`	`[0.2, -0.001413, 0.401413]`	Passed
zero control mean	`None`	`None`	Passed
summary statistic sample 1	`[-0.8875, -1.01151, -0.76349]`	`[-0.8875, -1.01151, -0.76349]`	Passed
summary statistic sample 2	`[-0.7, -0.712769, -0.687231]`	`[-0.7, -0.712769, -0.687231]`	Passed
summary statistic sample 3	`[1.0, 0.90268, 1.09732]`	`[1.0, 0.90268, 1.09732]`	Passed

SHA-256 / 93edf1568328ee8496d5df5e297007792669b0554c2e83f592814c5b92fe7db4

Verification & scope

A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:56.590416+00:00.

Case digest / 1cd69b91b84d450cb5611385386f4216c39aa136d8d9f49b54bd05a4fbe59e48