FAILURE MAP
← Case archive

FA-74541 / Experiment statistics / Open access

Post-exposure metric window: Activity before exposure is counted · case 01

Treatment effects are diluted by behaviour that happened before users saw the change.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The window check only bounds the end, not the exposure start.

VERIFIED REPAIR

Require day >= exposure day.

Unsuccessful approach: Excluding the exposure day itself drops the first, often strongest, day of impact.

Case contract

events are [user, day, value]; exposures maps user -> first exposure day. A user's metric sums values of their events with exposure_day <= day < exposure_day + window_days. Every exposed user appears, even with no qualifying events; unexposed users are dropped. Return sorted [user, total].

Why this case matters

Counting pre-exposure activity dilutes effects; dropping silent users biases means upward.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(events, exposures, window_days):
    totals = {u: 0 for u in exposures}
    for user, day, value in events:
        if user not in exposures:
            continue
        start = exposures[user]
        if day < start + window_days:
            totals[user] += value
    return sorted([u, t] for u, t in totals.items())
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('event log sample 1',
   [[['a', 7, 1], ['b', 7, 1], ['d', 10, 1], ['e', 10, 1], ['b', 5, 1], ['c', 11, 1]],
    {'c': 5, 'a': 0, 'b': 4},
    1],
   [['a', 0], ['b', 0], ['c', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]]),
  ('event log sample 3',
   [[['b', 11, 5], ['e', 0, 5], ['e', 7, 5], ['c', 6, 5]], {'b': 2, 'c': 2, 'a': 2}, 1],
   [['a', 0], ['b', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]]),
  ('event log sample 44',
   [[['b', 9, 5], ['b', 6, 2], ['e', 8, 1], ['b', 3, 2], ['a', 3, 5], ['a', 8, 1], ['e', 8, 1]],
    {'b': 4, 'c': 0},
    7],
   [['b', 7], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 11', [[['c', 7, 2], ['b', 3, 1]], {'a': 4, 'd': 2}, 1], [['a', 0], ['d', 0]]),
  ('event log sample 26',
   [[['a', 2, 5], ['d', 4, 5], ['c', 4, 1], ['c', 1, 2], ['e', 3, 5], ['e', 9, 1], ['e', 9, 1]],
    {'c': 5, 'a': 5, 'b': 3},
    3],
   [['a', 0], ['b', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 7',
   [[['d', 4, 1], ['a', 5, 2], ['a', 10, 5], ['b', 3, 2], ['a', 3, 2], ['c', 0, 1]], {'b': 5, 'a': 5}, 3],
   [['a', 2], ['b', 0]]),
  ('event log sample 16',
   [[['d', 0, 5], ['b', 3, 2], ['d', 1, 1], ['d', 2, 1]], {'c': 1, 'b': 5}, 1],
   [['b', 0], ['c', 0]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]]),
  ('event log sample 21', [[], {'d': 5, 'c': 2}, 7], [['c', 0], ['d', 0]]),
  ('event log sample 22', [[['e', 3, 2]], {'c': 1, 'b': 4}, 1], [['b', 0], ['c', 0]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
pre-exposure activity is excluded[['a', 7]][['a', 2]]Failed
exposure day itself counts[['a', 4]][['a', 4]]Passed
window end is exclusive[['a', 1]][['a', 1]]Passed
last day inside the window counts[['b', 2]][['b', 2]]Passed
exposed user with no events reports zero[['a', 1], ['b', 0]][['a', 1], ['b', 0]]Passed
event log sample 1[['a', 0], ['b', 0], ['c', 0]][['a', 0], ['b', 0], ['c', 0]]Passed
event log sample 2[['c', 1], ['d', 0]][['c', 0], ['d', 0]]Failed
event log sample 3[['a', 0], ['b', 0], ['c', 0]][['a', 0], ['b', 0], ['c', 0]]Passed

SHA-256 / 6be0ab246064b77b00bab99f598f83b238d6f3c878a754255663a65a75cead12

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(events, exposures, window_days):
    totals = {u: 0 for u in exposures}
    for user, day, value in events:
        if user not in exposures:
            continue
        start = exposures[user]
        if start < day < start + window_days:
            totals[user] += value
    return sorted([u, t] for u, t in totals.items())
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('event log sample 1',
   [[['a', 7, 1], ['b', 7, 1], ['d', 10, 1], ['e', 10, 1], ['b', 5, 1], ['c', 11, 1]],
    {'c': 5, 'a': 0, 'b': 4},
    1],
   [['a', 0], ['b', 0], ['c', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]]),
  ('event log sample 3',
   [[['b', 11, 5], ['e', 0, 5], ['e', 7, 5], ['c', 6, 5]], {'b': 2, 'c': 2, 'a': 2}, 1],
   [['a', 0], ['b', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]]),
  ('event log sample 44',
   [[['b', 9, 5], ['b', 6, 2], ['e', 8, 1], ['b', 3, 2], ['a', 3, 5], ['a', 8, 1], ['e', 8, 1]],
    {'b': 4, 'c': 0},
    7],
   [['b', 7], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 11', [[['c', 7, 2], ['b', 3, 1]], {'a': 4, 'd': 2}, 1], [['a', 0], ['d', 0]]),
  ('event log sample 26',
   [[['a', 2, 5], ['d', 4, 5], ['c', 4, 1], ['c', 1, 2], ['e', 3, 5], ['e', 9, 1], ['e', 9, 1]],
    {'c': 5, 'a': 5, 'b': 3},
    3],
   [['a', 0], ['b', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 7',
   [[['d', 4, 1], ['a', 5, 2], ['a', 10, 5], ['b', 3, 2], ['a', 3, 2], ['c', 0, 1]], {'b': 5, 'a': 5}, 3],
   [['a', 2], ['b', 0]]),
  ('event log sample 16',
   [[['d', 0, 5], ['b', 3, 2], ['d', 1, 1], ['d', 2, 1]], {'c': 1, 'b': 5}, 1],
   [['b', 0], ['c', 0]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]]),
  ('event log sample 21', [[], {'d': 5, 'c': 2}, 7], [['c', 0], ['d', 0]]),
  ('event log sample 22', [[['e', 3, 2]], {'c': 1, 'b': 4}, 1], [['b', 0], ['c', 0]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
pre-exposure activity is excluded[['a', 0]][['a', 2]]Failed
exposure day itself counts[['a', 0]][['a', 4]]Failed
window end is exclusive[['a', 0]][['a', 1]]Failed
last day inside the window counts[['b', 2]][['b', 2]]Passed
exposed user with no events reports zero[['a', 0], ['b', 0]][['a', 1], ['b', 0]]Failed
event log sample 1[['a', 0], ['b', 0], ['c', 0]][['a', 0], ['b', 0], ['c', 0]]Passed
event log sample 2[['c', 0], ['d', 0]][['c', 0], ['d', 0]]Passed
event log sample 3[['a', 0], ['b', 0], ['c', 0]][['a', 0], ['b', 0], ['c', 0]]Passed

SHA-256 / c97fda9386f6e30a6d99c8a7c280b93009afb90698c9f53ee99483e2fa03f9d6

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(events, exposures, window_days):
    totals = {u: 0 for u in exposures}
    for user, day, value in events:
        if user not in exposures:
            continue
        start = exposures[user]
        if start <= day < start + window_days:
            totals[user] += value
    return sorted([u, t] for u, t in totals.items())
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('event log sample 1',
   [[['a', 7, 1], ['b', 7, 1], ['d', 10, 1], ['e', 10, 1], ['b', 5, 1], ['c', 11, 1]],
    {'c': 5, 'a': 0, 'b': 4},
    1],
   [['a', 0], ['b', 0], ['c', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]]),
  ('event log sample 3',
   [[['b', 11, 5], ['e', 0, 5], ['e', 7, 5], ['c', 6, 5]], {'b': 2, 'c': 2, 'a': 2}, 1],
   [['a', 0], ['b', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]]),
  ('event log sample 44',
   [[['b', 9, 5], ['b', 6, 2], ['e', 8, 1], ['b', 3, 2], ['a', 3, 5], ['a', 8, 1], ['e', 8, 1]],
    {'b': 4, 'c': 0},
    7],
   [['b', 7], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('window end is exclusive', [[['a', 3, 1], ['a', 10, 9]], {'a': 3}, 7], [['a', 1]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 11', [[['c', 7, 2], ['b', 3, 1]], {'a': 4, 'd': 2}, 1], [['a', 0], ['d', 0]]),
  ('event log sample 26',
   [[['a', 2, 5], ['d', 4, 5], ['c', 4, 1], ['c', 1, 2], ['e', 3, 5], ['e', 9, 1], ['e', 9, 1]],
    {'c': 5, 'a': 5, 'b': 3},
    3],
   [['a', 0], ['b', 0], ['c', 0]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('last day inside the window counts', [[['b', 9, 2]], {'b': 3}, 7], [['b', 2]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 7',
   [[['d', 4, 1], ['a', 5, 2], ['a', 10, 5], ['b', 3, 2], ['a', 3, 2], ['c', 0, 1]], {'b': 5, 'a': 5}, 3],
   [['a', 2], ['b', 0]]),
  ('event log sample 16',
   [[['d', 0, 5], ['b', 3, 2], ['d', 1, 1], ['d', 2, 1]], {'c': 1, 'b': 5}, 1],
   [['b', 0], ['c', 0]]),
  ('event log sample 24',
   [[['c', 5, 5], ['c', 9, 2], ['c', 6, 5], ['c', 6, 5], ['b', 11, 1], ['e', 6, 5], ['c', 2, 2]],
    {'c': 2, 'b': 3},
    7],
   [['b', 0], ['c', 17]])],
 [('pre-exposure activity is excluded', [[['a', 1, 5], ['a', 3, 2]], {'a': 3}, 7], [['a', 2]]),
  ('exposure day itself counts', [[['a', 3, 4]], {'a': 3}, 7], [['a', 4]]),
  ('exposed user with no events reports zero', [[['a', 3, 1]], {'a': 3, 'b': 4}, 7], [['a', 1], ['b', 0]]),
  ('each user has their own window', [[['a', 2, 1], ['b', 8, 1]], {'a': 0, 'b': 6}, 3], [['a', 1], ['b', 1]]),
  ('unexposed users are dropped', [[['z', 1, 9]], {'a': 0}, 3], [['a', 0]]),
  ('event log sample 2',
   [[['c', 1, 1], ['a', 6, 2], ['a', 2, 5]], {'c': 3, 'd': 3}, 7],
   [['c', 0], ['d', 0]]),
  ('event log sample 21', [[], {'d': 5, 'c': 2}, 7], [['c', 0], ['d', 0]]),
  ('event log sample 22', [[['e', 3, 2]], {'c': 1, 'b': 4}, 1], [['b', 0], ['c', 0]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
pre-exposure activity is excluded[['a', 2]][['a', 2]]Passed
exposure day itself counts[['a', 4]][['a', 4]]Passed
window end is exclusive[['a', 1]][['a', 1]]Passed
last day inside the window counts[['b', 2]][['b', 2]]Passed
exposed user with no events reports zero[['a', 1], ['b', 0]][['a', 1], ['b', 0]]Passed
event log sample 1[['a', 0], ['b', 0], ['c', 0]][['a', 0], ['b', 0], ['c', 0]]Passed
event log sample 2[['c', 0], ['d', 0]][['c', 0], ['d', 0]]Passed
event log sample 3[['a', 0], ['b', 0], ['c', 0]][['a', 0], ['b', 0], ['c', 0]]Passed

SHA-256 / b77bebfd1da0adca5225d12afcaf151c89045c098c2ac04e215ca438d28d428d

Verification & scope

A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:57.814810+00:00.

Case digest / f2ce8cb92fe0074b60c8a8f17d781b497e14daede09b51a52e1c2ec2e7f44f1c