FA-74546 / Experiment statistics / Open access

Post-exposure metric window: The analysis window includes one extra day · case 01

A 7-day metric actually sums 8 days of activity.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The end check uses day <= start + window_days.

VERIFIED REPAIR

Use a half-open window ending before start + window_days.

Unsuccessful approach: Shortening the window by one drops the legitimate last day.

Case contract

events are [user, day, value]; exposures maps user -> first exposure day. A user's metric sums values of their events with exposure_day <= day < exposure_day + window_days. Every exposed user appears, even with no qualifying events; unexposed users are dropped. Return sorted [user, total].

Why this case matters

Counting pre-exposure activity dilutes effects; dropping silent users biases means upward.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(events, exposures, window_days):
    totals = {u: 0 for u in exposures}
    for user, day, value in events:
        if user not in exposures:
            continue
        start = exposures[user]
        if start <= day <= start + window_days:
            totals[user] += value
    return sorted([u, t] for u, t in totals.items())
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 1',
   [[['a', 7, 1], ['b', 7, 1], ['d', 10, 1], ['e', 10, 1], ['b', 5, 1], ['c', 11, 1]],
    {'c': 5, 'a': 0, 'b': 4},
    1],
   [['a', 0], ['b', 0], ['c', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]])],
 [('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 6',
   [[['d', 1, 1], ['e', 8, 5], ['c', 5, 2], ['e', 1, 2], ['c', 9, 2], ['a', 1, 2]], {'d': 0}, 3],
   [['d', 1]]),
  ('event log sample 43',
   [[['e', 7, 2], ['e', 1, 2], ['a', 3, 2], ['e', 3, 1]], {'b': 4, 'a': 3, 'd': 3}, 1],
   [['a', 2], ['b', 0], ['d', 0]]),
  ('event log sample 54',
   [[['e', 4, 1], ['d', 6, 5], ['d', 3, 2], ['e', 5, 1], ['c', 0, 1], ['b', 11, 2], ['a', 6, 5], ['d', 3, 5]],
    {'a': 0, 'c': 5, 'b': 4},
    7],
   [['a', 5], ['b', 0], ['c', 0]])],
 [('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 11', [[['c', 7, 2], ['b', 3, 1]], {'a': 4, 'd': 2}, 1], [['a', 0], ['d', 0]]),
  ('event log sample 18',
   [[['e', 3, 1], ['c', 7, 5], ['a', 5, 5], ['a', 4, 5], ['a', 2, 5], ['d', 1, 5], ['e', 3, 5], ['e', 2, 2]],
    {'a': 2},
    1],
   [['a', 5]]),
  ('event log sample 36',
   [[['a', 5, 5], ['a', 5, 5], ['b', 2, 5], ['e', 8, 1], ['d', 5, 1], ['e', 7, 5]], {'c': 0, 'a': 2}, 3],
   [['a', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 16',
   [[['d', 0, 5], ['b', 3, 2], ['d', 1, 1], ['d', 2, 1]], {'c': 1, 'b': 5}, 1],
   [['b', 0], ['c', 0]]),
  ('event log sample 30',
   [[['e', 7, 5],
     ['a', 6, 5],
     ['e', 3, 5],
     ['a', 3, 5],
     ['a', 3, 1],
     ['c', 11, 2],
     ['d', 8, 2],
     ['b', 10, 1]],
    {'d': 5},
    3],
   [['d', 0]]),
  ('event log sample 59',
   [[['a', 2, 2],
     ['c', 5, 1],
     ['d', 0, 2],
     ['d', 11, 5],
     ['e', 10, 1],
     ['e', 1, 5],
     ['c', 9, 5],
     ['b', 4, 5]],
    {'a': 1, 'b': 4, 'c': 3},
    7],
   [['a', 2], ['b', 5], ['c', 6]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 21', [[], {'d': 5, 'c': 2}, 7], [['c', 0], ['d', 0]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]]),
  ('event log sample 40',
   [[['a', 8, 2], ['c', 8, 1], ['b', 8, 1], ['d', 1, 2], ['a', 2, 5]], {'d': 1, 'b': 2}, 7],
   [['b', 1], ['d', 2]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
pre-exposure activity is excluded	`[['a', 2]]`	`[['a', 2]]`	Passed
exposure day itself counts	`[['a', 4]]`	`[['a', 4]]`	Passed
window end is exclusive	`[['a', 10]]`	`[['a', 1]]`	Failed
last day inside the window counts	`[['b', 2]]`	`[['b', 2]]`	Passed
exposed user with no events reports zero	`[['a', 1], ['b', 0]]`	`[['a', 1], ['b', 0]]`	Passed
each user has their own window	`[['a', 1], ['b', 1]]`	`[['a', 1], ['b', 1]]`	Passed
event log sample 1	`[['a', 0], ['b', 1], ['c', 0]]`	`[['a', 0], ['b', 0], ['c', 0]]`	Failed
event log sample 2	`[['c', 0], ['d', 0]]`	`[['c', 0], ['d', 0]]`	Passed

SHA-256 / 21f4fc8a95878180acbefa508a6d54f7a110df9ef7612c7fd47b9d05fb9dd2dc

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(events, exposures, window_days):
    totals = {u: 0 for u in exposures}
    for user, day, value in events:
        if user not in exposures:
            continue
        start = exposures[user]
        if start <= day < start + window_days - 1:
            totals[user] += value
    return sorted([u, t] for u, t in totals.items())
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 1',
   [[['a', 7, 1], ['b', 7, 1], ['d', 10, 1], ['e', 10, 1], ['b', 5, 1], ['c', 11, 1]],
    {'c': 5, 'a': 0, 'b': 4},
    1],
   [['a', 0], ['b', 0], ['c', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]])],
 [('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 6',
   [[['d', 1, 1], ['e', 8, 5], ['c', 5, 2], ['e', 1, 2], ['c', 9, 2], ['a', 1, 2]], {'d': 0}, 3],
   [['d', 1]]),
  ('event log sample 43',
   [[['e', 7, 2], ['e', 1, 2], ['a', 3, 2], ['e', 3, 1]], {'b': 4, 'a': 3, 'd': 3}, 1],
   [['a', 2], ['b', 0], ['d', 0]]),
  ('event log sample 54',
   [[['e', 4, 1], ['d', 6, 5], ['d', 3, 2], ['e', 5, 1], ['c', 0, 1], ['b', 11, 2], ['a', 6, 5], ['d', 3, 5]],
    {'a': 0, 'c': 5, 'b': 4},
    7],
   [['a', 5], ['b', 0], ['c', 0]])],
 [('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 11', [[['c', 7, 2], ['b', 3, 1]], {'a': 4, 'd': 2}, 1], [['a', 0], ['d', 0]]),
  ('event log sample 18',
   [[['e', 3, 1], ['c', 7, 5], ['a', 5, 5], ['a', 4, 5], ['a', 2, 5], ['d', 1, 5], ['e', 3, 5], ['e', 2, 2]],
    {'a': 2},
    1],
   [['a', 5]]),
  ('event log sample 36',
   [[['a', 5, 5], ['a', 5, 5], ['b', 2, 5], ['e', 8, 1], ['d', 5, 1], ['e', 7, 5]], {'c': 0, 'a': 2}, 3],
   [['a', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 16',
   [[['d', 0, 5], ['b', 3, 2], ['d', 1, 1], ['d', 2, 1]], {'c': 1, 'b': 5}, 1],
   [['b', 0], ['c', 0]]),
  ('event log sample 30',
   [[['e', 7, 5],
     ['a', 6, 5],
     ['e', 3, 5],
     ['a', 3, 5],
     ['a', 3, 1],
     ['c', 11, 2],
     ['d', 8, 2],
     ['b', 10, 1]],
    {'d': 5},
    3],
   [['d', 0]]),
  ('event log sample 59',
   [[['a', 2, 2],
     ['c', 5, 1],
     ['d', 0, 2],
     ['d', 11, 5],
     ['e', 10, 1],
     ['e', 1, 5],
     ['c', 9, 5],
     ['b', 4, 5]],
    {'a': 1, 'b': 4, 'c': 3},
    7],
   [['a', 2], ['b', 5], ['c', 6]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 21', [[], {'d': 5, 'c': 2}, 7], [['c', 0], ['d', 0]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]]),
  ('event log sample 40',
   [[['a', 8, 2], ['c', 8, 1], ['b', 8, 1], ['d', 1, 2], ['a', 2, 5]], {'d': 1, 'b': 2}, 7],
   [['b', 1], ['d', 2]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
pre-exposure activity is excluded	`[['a', 2]]`	`[['a', 2]]`	Passed
exposure day itself counts	`[['a', 4]]`	`[['a', 4]]`	Passed
window end is exclusive	`[['a', 1]]`	`[['a', 1]]`	Passed
last day inside the window counts	`[['b', 0]]`	`[['b', 2]]`	Failed
exposed user with no events reports zero	`[['a', 1], ['b', 0]]`	`[['a', 1], ['b', 0]]`	Passed
each user has their own window	`[['a', 0], ['b', 0]]`	`[['a', 1], ['b', 1]]`	Failed
event log sample 1	`[['a', 0], ['b', 0], ['c', 0]]`	`[['a', 0], ['b', 0], ['c', 0]]`	Passed
event log sample 2	`[['c', 0], ['d', 0]]`	`[['c', 0], ['d', 0]]`	Passed

SHA-256 / aa95945c361c32b3c70219b03c700d48e8218626765d419ab7fe1e40c7cee314

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(events, exposures, window_days):
    totals = {u: 0 for u in exposures}
    for user, day, value in events:
        if user not in exposures:
            continue
        start = exposures[user]
        if start <= day < start + window_days:
            totals[user] += value
    return sorted([u, t] for u, t in totals.items())
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 1',
   [[['a', 7, 1], ['b', 7, 1], ['d', 10, 1], ['e', 10, 1], ['b', 5, 1], ['c', 11, 1]],
    {'c': 5, 'a': 0, 'b': 4},
    1],
   [['a', 0], ['b', 0], ['c', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]])],
 [('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 6',
   [[['d', 1, 1], ['e', 8, 5], ['c', 5, 2], ['e', 1, 2], ['c', 9, 2], ['a', 1, 2]], {'d': 0}, 3],
   [['d', 1]]),
  ('event log sample 43',
   [[['e', 7, 2], ['e', 1, 2], ['a', 3, 2], ['e', 3, 1]], {'b': 4, 'a': 3, 'd': 3}, 1],
   [['a', 2], ['b', 0], ['d', 0]]),
  ('event log sample 54',
   [[['e', 4, 1], ['d', 6, 5], ['d', 3, 2], ['e', 5, 1], ['c', 0, 1], ['b', 11, 2], ['a', 6, 5], ['d', 3, 5]],
    {'a': 0, 'c': 5, 'b': 4},
    7],
   [['a', 5], ['b', 0], ['c', 0]])],
 [('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 11', [[['c', 7, 2], ['b', 3, 1]], {'a': 4, 'd': 2}, 1], [['a', 0], ['d', 0]]),
  ('event log sample 18',
   [[['e', 3, 1], ['c', 7, 5], ['a', 5, 5], ['a', 4, 5], ['a', 2, 5], ['d', 1, 5], ['e', 3, 5], ['e', 2, 2]],
    {'a': 2},
    1],
   [['a', 5]]),
  ('event log sample 36',
   [[['a', 5, 5], ['a', 5, 5], ['b', 2, 5], ['e', 8, 1], ['d', 5, 1], ['e', 7, 5]], {'c': 0, 'a': 2}, 3],
   [['a', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 16',
   [[['d', 0, 5], ['b', 3, 2], ['d', 1, 1], ['d', 2, 1]], {'c': 1, 'b': 5}, 1],
   [['b', 0], ['c', 0]]),
  ('event log sample 30',
   [[['e', 7, 5],
     ['a', 6, 5],
     ['e', 3, 5],
     ['a', 3, 5],
     ['a', 3, 1],
     ['c', 11, 2],
     ['d', 8, 2],
     ['b', 10, 1]],
    {'d': 5},
    3],
   [['d', 0]]),
  ('event log sample 59',
   [[['a', 2, 2],
     ['c', 5, 1],
     ['d', 0, 2],
     ['d', 11, 5],
     ['e', 10, 1],
     ['e', 1, 5],
     ['c', 9, 5],
     ['b', 4, 5]],
    {'a': 1, 'b': 4, 'c': 3},
    7],
   [['a', 2], ['b', 5], ['c', 6]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 21', [[], {'d': 5, 'c': 2}, 7], [['c', 0], ['d', 0]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]]),
  ('event log sample 40',
   [[['a', 8, 2], ['c', 8, 1], ['b', 8, 1], ['d', 1, 2], ['a', 2, 5]], {'d': 1, 'b': 2}, 7],
   [['b', 1], ['d', 2]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
pre-exposure activity is excluded	`[['a', 2]]`	`[['a', 2]]`	Passed
exposure day itself counts	`[['a', 4]]`	`[['a', 4]]`	Passed
window end is exclusive	`[['a', 1]]`	`[['a', 1]]`	Passed
last day inside the window counts	`[['b', 2]]`	`[['b', 2]]`	Passed
exposed user with no events reports zero	`[['a', 1], ['b', 0]]`	`[['a', 1], ['b', 0]]`	Passed
each user has their own window	`[['a', 1], ['b', 1]]`	`[['a', 1], ['b', 1]]`	Passed
event log sample 1	`[['a', 0], ['b', 0], ['c', 0]]`	`[['a', 0], ['b', 0], ['c', 0]]`	Passed
event log sample 2	`[['c', 0], ['d', 0]]`	`[['c', 0], ['d', 0]]`	Passed

SHA-256 / 9718b6d34df5332b55d2ed8d9cf74218b9ad0e851587a48a8a15808b27a7bcab

Verification & scope

A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:57.819975+00:00.

Case digest / 54f61997947e9710cd55d98e6dbffc82c9c8c5431ed53087fd21ac642716e852