FA-74446 / Experiment statistics / Open access

CUPED covariate adjustment: A constant covariate gets a unit coefficient · case 01

Experiments whose covariate never varies report theta = 1.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The zero-variance fallback sets theta to 1.0.

VERIFIED REPAIR

Use theta = 0 when the covariate has no variance.

Unsuccessful approach: Adding one to the variance biases every theta toward zero.

Case contract

theta = cov(y, x) / var(x) over the pooled data of both arms (theta = 0 when x is constant). Each arm's adjusted mean is mean(y_arm) - theta * (mean(x_arm) - pooled mean(x)). Return [theta, adjusted control, adjusted treatment, difference] rounded to 6 places.

Why this case matters

CUPED uses pre-period data to cut variance; mistakes either waste it or bias the estimate.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(y_c, x_c, y_t, x_t):
    ys = y_c + y_t
    xs = x_c + x_t
    n = len(xs)
    mx, my = sum(xs) / n, sum(ys) / n
    vx = sum((x - mx) ** 2 for x in xs)
    cxy = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
    theta = cxy / vx if vx else 1.0
    def adj(ys_, xs_):
        mean_x = sum(xs_) / len(xs_)
        return sum(ys_) / len(ys_) - theta * (mean_x - mx)
    ac, at = adj(y_c, x_c), adj(y_t, x_t)
    return [round(theta, 6), round(ac, 6), round(at, 6), round(at - ac, 6)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 1',
   [[8, 8, 10, 16], [3, 5, 7, 8], [6, 10], [3, 7]],
   [1.319149, 10.170213, 8.659574, -1.510638]),
  ('pre/post sample 2',
   [[9, 8, 18], [8, 8, 8], [0, 3, 11, 1, 2], [0, 2, 8, 0, 1]],
   [1.376593, 6.676516, 6.39409, -0.282426]),
  ('pre/post sample 3',
   [[4, 12, 1], [1, 5, 1], [6, 15, 20, 16], [0, 9, 8, 6]],
   [1.748201, 9.079822, 11.690134, 2.610312])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 4',
   [[9, 8, 8, 3, 8], [6, 4, 7, 1, 5], [10, 20, 12, 6, 6], [3, 8, 4, 0, 0]],
   [1.131285, 6.294972, 11.705028, 5.410056]),
  ('pre/post sample 6',
   [[9, 12, 4, 8, 10], [7, 6, 3, 7, 4], [10, 12], [4, 9]],
   [0.713542, 8.824256, 10.43936, 1.615104]),
  ('pre/post sample 7',
   [[10, 6, 11, 10], [7, 3, 5, 4], [12, 6, 12, 8], [9, 2, 4, 6]],
   [0.666667, 9.416667, 9.333333, -0.083333])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 11', [[10, 16], [8, 7], [7, 19], [6, 9]], [3.0, 13.0, 13.0, 0.0]),
  ('pre/post sample 12', [[10, 10], [4, 7], [10, 9], [6, 3]], [0.2, 9.9, 9.6, -0.3]),
  ('pre/post sample 13', [[8, 5, 9, 7], [6, 5, 7, 5], [16, 11], [5, 8]], [0.375, 7.34375, 13.3125, 5.96875])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 16',
   [[18, 4, 1, 10], [8, 1, 0, 5], [16, 13, 15, 3, 7], [5, 8, 5, 0, 2]],
   [1.789106, 8.746974, 10.402421, 1.655447]),
  ('pre/post sample 17',
   [[10, 10, 3], [7, 9, 3], [5, 12, 7], [3, 9, 3]],
   [1.029412, 6.980392, 8.686275, 1.705882]),
  ('pre/post sample 18',
   [[8, 21], [7, 9], [9, 11, 4], [6, 7, 0]],
   [1.465812, 11.275214, 10.149858, -1.125356])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 21', [[3, 12], [2, 5], [21, 16], [9, 6]], [2.6, 12.7, 13.3, 0.6]),
  ('pre/post sample 22',
   [[12, 5, 5, 2], [9, 2, 4, 1], [9, 8, 10, 9], [3, 5, 8, 4]],
   [0.981481, 6.490741, 8.509259, 2.018519]),
  ('pre/post sample 25', [[2, 7], [1, 3], [19, 5], [7, 2]], [2.831325, 8.039157, 8.460843, 0.421687])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
pre-period imbalance is corrected	`[2.0, 14.0, 14.0, 0.0]`	`[2.0, 14.0, 14.0, 0.0]`	Passed
perfectly correlated covariate	`[2.0, 5.0, 5.0, 0.0]`	`[2.0, 5.0, 5.0, 0.0]`	Passed
constant covariate leaves means unadjusted	`[1.0, 2.0, 3.0, 1.0]`	`[0.0, 2.0, 3.0, 1.0]`	Failed
arms of different size	`[1.851852, 5.851852, 6.574074, 0.722222]`	`[1.851852, 5.851852, 6.574074, 0.722222]`	Passed
covariate balanced across arms	`[1.0, 2.0, 3.0, 1.0]`	`[1.0, 2.0, 3.0, 1.0]`	Passed
pre/post sample 1	`[1.319149, 10.170213, 8.659574, -1.510638]`	`[1.319149, 10.170213, 8.659574, -1.510638]`	Passed
pre/post sample 2	`[1.376593, 6.676516, 6.39409, -0.282426]`	`[1.376593, 6.676516, 6.39409, -0.282426]`	Passed
pre/post sample 3	`[1.748201, 9.079822, 11.690134, 2.610312]`	`[1.748201, 9.079822, 11.690134, 2.610312]`	Passed

SHA-256 / 5424e59b09c42c8a2b5f218bae1af7278613f899fb3eb93854b1b53f57691b90

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(y_c, x_c, y_t, x_t):
    ys = y_c + y_t
    xs = x_c + x_t
    n = len(xs)
    mx, my = sum(xs) / n, sum(ys) / n
    vx = sum((x - mx) ** 2 for x in xs)
    cxy = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
    theta = cxy / (vx + 1)
    def adj(ys_, xs_):
        mean_x = sum(xs_) / len(xs_)
        return sum(ys_) / len(ys_) - theta * (mean_x - mx)
    ac, at = adj(y_c, x_c), adj(y_t, x_t)
    return [round(theta, 6), round(ac, 6), round(at, 6), round(at - ac, 6)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 1',
   [[8, 8, 10, 16], [3, 5, 7, 8], [6, 10], [3, 7]],
   [1.319149, 10.170213, 8.659574, -1.510638]),
  ('pre/post sample 2',
   [[9, 8, 18], [8, 8, 8], [0, 3, 11, 1, 2], [0, 2, 8, 0, 1]],
   [1.376593, 6.676516, 6.39409, -0.282426]),
  ('pre/post sample 3',
   [[4, 12, 1], [1, 5, 1], [6, 15, 20, 16], [0, 9, 8, 6]],
   [1.748201, 9.079822, 11.690134, 2.610312])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 4',
   [[9, 8, 8, 3, 8], [6, 4, 7, 1, 5], [10, 20, 12, 6, 6], [3, 8, 4, 0, 0]],
   [1.131285, 6.294972, 11.705028, 5.410056]),
  ('pre/post sample 6',
   [[9, 12, 4, 8, 10], [7, 6, 3, 7, 4], [10, 12], [4, 9]],
   [0.713542, 8.824256, 10.43936, 1.615104]),
  ('pre/post sample 7',
   [[10, 6, 11, 10], [7, 3, 5, 4], [12, 6, 12, 8], [9, 2, 4, 6]],
   [0.666667, 9.416667, 9.333333, -0.083333])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 11', [[10, 16], [8, 7], [7, 19], [6, 9]], [3.0, 13.0, 13.0, 0.0]),
  ('pre/post sample 12', [[10, 10], [4, 7], [10, 9], [6, 3]], [0.2, 9.9, 9.6, -0.3]),
  ('pre/post sample 13', [[8, 5, 9, 7], [6, 5, 7, 5], [16, 11], [5, 8]], [0.375, 7.34375, 13.3125, 5.96875])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 16',
   [[18, 4, 1, 10], [8, 1, 0, 5], [16, 13, 15, 3, 7], [5, 8, 5, 0, 2]],
   [1.789106, 8.746974, 10.402421, 1.655447]),
  ('pre/post sample 17',
   [[10, 10, 3], [7, 9, 3], [5, 12, 7], [3, 9, 3]],
   [1.029412, 6.980392, 8.686275, 1.705882]),
  ('pre/post sample 18',
   [[8, 21], [7, 9], [9, 11, 4], [6, 7, 0]],
   [1.465812, 11.275214, 10.149858, -1.125356])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 21', [[3, 12], [2, 5], [21, 16], [9, 6]], [2.6, 12.7, 13.3, 0.6]),
  ('pre/post sample 22',
   [[12, 5, 5, 2], [9, 2, 4, 1], [9, 8, 10, 9], [3, 5, 8, 4]],
   [0.981481, 6.490741, 8.509259, 2.018519]),
  ('pre/post sample 25', [[2, 7], [1, 3], [19, 5], [7, 2]], [2.831325, 8.039157, 8.460843, 0.421687])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
pre-period imbalance is corrected	`[1.818182, 13.818182, 14.181818, 0.363636]`	`[2.0, 14.0, 14.0, 0.0]`	Failed
perfectly correlated covariate	`[1.692308, 4.846154, 5.153846, 0.307692]`	`[2.0, 5.0, 5.0, 0.0]`	Failed
constant covariate leaves means unadjusted	`[0.0, 2.0, 3.0, 1.0]`	`[0.0, 2.0, 3.0, 1.0]`	Passed
arms of different size	`[1.724138, 5.724138, 6.637931, 0.913793]`	`[1.851852, 5.851852, 6.574074, 0.722222]`	Failed
covariate balanced across arms	`[0.8, 2.0, 3.0, 1.0]`	`[1.0, 2.0, 3.0, 1.0]`	Failed
pre/post sample 1	`[1.265306, 10.183673, 8.632653, -1.55102]`	`[1.319149, 10.170213, 8.659574, -1.510638]`	Failed
pre/post sample 2	`[1.363949, 6.72235, 6.36659, -0.35576]`	`[1.376593, 6.676516, 6.39409, -0.282426]`	Failed
pre/post sample 3	`[1.726465, 9.037385, 11.721961, 2.684577]`	`[1.748201, 9.079822, 11.690134, 2.610312]`	Failed

SHA-256 / 06c2fc172c31c91f50e55e62df8926bbe861fb1db4489da86eb4d083fcd62d28

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(y_c, x_c, y_t, x_t):
    ys = y_c + y_t
    xs = x_c + x_t
    n = len(xs)
    mx, my = sum(xs) / n, sum(ys) / n
    vx = sum((x - mx) ** 2 for x in xs)
    cxy = sum((x - mx) * (y - my) for x, y in zip(xs, ys))
    theta = cxy / vx if vx else 0.0
    def adj(ys_, xs_):
        mean_x = sum(xs_) / len(xs_)
        return sum(ys_) / len(ys_) - theta * (mean_x - mx)
    ac, at = adj(y_c, x_c), adj(y_t, x_t)
    return [round(theta, 6), round(ac, 6), round(at, 6), round(at - ac, 6)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 1',
   [[8, 8, 10, 16], [3, 5, 7, 8], [6, 10], [3, 7]],
   [1.319149, 10.170213, 8.659574, -1.510638]),
  ('pre/post sample 2',
   [[9, 8, 18], [8, 8, 8], [0, 3, 11, 1, 2], [0, 2, 8, 0, 1]],
   [1.376593, 6.676516, 6.39409, -0.282426]),
  ('pre/post sample 3',
   [[4, 12, 1], [1, 5, 1], [6, 15, 20, 16], [0, 9, 8, 6]],
   [1.748201, 9.079822, 11.690134, 2.610312])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 4',
   [[9, 8, 8, 3, 8], [6, 4, 7, 1, 5], [10, 20, 12, 6, 6], [3, 8, 4, 0, 0]],
   [1.131285, 6.294972, 11.705028, 5.410056]),
  ('pre/post sample 6',
   [[9, 12, 4, 8, 10], [7, 6, 3, 7, 4], [10, 12], [4, 9]],
   [0.713542, 8.824256, 10.43936, 1.615104]),
  ('pre/post sample 7',
   [[10, 6, 11, 10], [7, 3, 5, 4], [12, 6, 12, 8], [9, 2, 4, 6]],
   [0.666667, 9.416667, 9.333333, -0.083333])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 11', [[10, 16], [8, 7], [7, 19], [6, 9]], [3.0, 13.0, 13.0, 0.0]),
  ('pre/post sample 12', [[10, 10], [4, 7], [10, 9], [6, 3]], [0.2, 9.9, 9.6, -0.3]),
  ('pre/post sample 13', [[8, 5, 9, 7], [6, 5, 7, 5], [16, 11], [5, 8]], [0.375, 7.34375, 13.3125, 5.96875])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 16',
   [[18, 4, 1, 10], [8, 1, 0, 5], [16, 13, 15, 3, 7], [5, 8, 5, 0, 2]],
   [1.789106, 8.746974, 10.402421, 1.655447]),
  ('pre/post sample 17',
   [[10, 10, 3], [7, 9, 3], [5, 12, 7], [3, 9, 3]],
   [1.029412, 6.980392, 8.686275, 1.705882]),
  ('pre/post sample 18',
   [[8, 21], [7, 9], [9, 11, 4], [6, 7, 0]],
   [1.465812, 11.275214, 10.149858, -1.125356])],
 [('pre-period imbalance is corrected',
   [[10, 12, 14], [5, 6, 7], [14, 16, 18], [7, 8, 9]],
   [2.0, 14.0, 14.0, 0.0]),
  ('perfectly correlated covariate', [[2, 4, 6], [1, 2, 3], [4, 6, 8], [2, 3, 4]], [2.0, 5.0, 5.0, 0.0]),
  ('constant covariate leaves means unadjusted',
   [[1, 2, 3], [5, 5, 5], [2, 3, 4], [5, 5, 5]],
   [0.0, 2.0, 3.0, 1.0]),
  ('arms of different size',
   [[3, 5], [1, 2], [4, 6, 9, 11], [1, 2, 4, 5]],
   [1.851852, 5.851852, 6.574074, 0.722222]),
  ('covariate balanced across arms', [[1, 3], [1, 3], [2, 4], [1, 3]], [1.0, 2.0, 3.0, 1.0]),
  ('pre/post sample 21', [[3, 12], [2, 5], [21, 16], [9, 6]], [2.6, 12.7, 13.3, 0.6]),
  ('pre/post sample 22',
   [[12, 5, 5, 2], [9, 2, 4, 1], [9, 8, 10, 9], [3, 5, 8, 4]],
   [0.981481, 6.490741, 8.509259, 2.018519]),
  ('pre/post sample 25', [[2, 7], [1, 3], [19, 5], [7, 2]], [2.831325, 8.039157, 8.460843, 0.421687])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
pre-period imbalance is corrected	`[2.0, 14.0, 14.0, 0.0]`	`[2.0, 14.0, 14.0, 0.0]`	Passed
perfectly correlated covariate	`[2.0, 5.0, 5.0, 0.0]`	`[2.0, 5.0, 5.0, 0.0]`	Passed
constant covariate leaves means unadjusted	`[0.0, 2.0, 3.0, 1.0]`	`[0.0, 2.0, 3.0, 1.0]`	Passed
arms of different size	`[1.851852, 5.851852, 6.574074, 0.722222]`	`[1.851852, 5.851852, 6.574074, 0.722222]`	Passed
covariate balanced across arms	`[1.0, 2.0, 3.0, 1.0]`	`[1.0, 2.0, 3.0, 1.0]`	Passed
pre/post sample 1	`[1.319149, 10.170213, 8.659574, -1.510638]`	`[1.319149, 10.170213, 8.659574, -1.510638]`	Passed
pre/post sample 2	`[1.376593, 6.676516, 6.39409, -0.282426]`	`[1.376593, 6.676516, 6.39409, -0.282426]`	Passed
pre/post sample 3	`[1.748201, 9.079822, 11.690134, 2.610312]`	`[1.748201, 9.079822, 11.690134, 2.610312]`	Passed

SHA-256 / 83995744da68935f0d596e2314393315a2905f47ac608b3a6087c2fa3857e739

Verification & scope

A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:56.882592+00:00.

Case digest / 5685c9bee5e8ce3dcec698217efca9ee8e38e0de1f233230fb5514a29c5e6053