FA-12646 / Causal analysis data contracts / Open access
An ATT request silently averages untreated potential outcomes · case 01
An ATT request silently averages untreated potential outcomes.
ROOT CAUSE
A complete synthetic potential-outcome table is averaged across the wrong target population.
VERIFIED REPAIR
Select treated rows for ATT and all rows for ATE.
Unsuccessful approach: Selecting by positive individual effect substitutes outcome response for target membership.
Case contract
Rows [treated,y0,y1] contain both synthetic potential outcomes. target is ATT or ATE; return the target average y1-y0, None for an empty target.
Why this case matters
A deterministic synthetic study model isolates this data-contract defect; outputs alone establish no real-world causal identification.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(rows, target):
return sum(r[2]-r[1] for r in rows)/len(rows) if rows else None
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('ATT membership', solve([[1,0,2*N],[0,0,8*N]], 'ATT'), 2*N)
check('ATE', solve([[1,0,2*N],[0,0,8*N]], 'ATE'), 5*N)
check('negative treated', solve([[1,3*N,N],[0,0,N]], 'ATT'), -2*N)
check('no treated', solve([[0,0,N]], 'ATT'), None)
check('empty', solve([], 'ATE'), None)
check('null effect treated', solve([[1,N,N]], 'ATT'), 0)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| ATT membership | 5.0 | 2 | Failed |
| ATE | 5.0 | 5 | Passed |
| negative treated | -0.5 | -2 | Failed |
| no treated | 1.0 | None | Failed |
| empty | None | None | Passed |
| null effect treated | 0.0 | 0 | Passed |
SHA-256 / f45c629b3a6f208fabf2c13e6122cd75a332f79842f95f9e517a8d84ee4faeee
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(rows, target):
chosen = [r for r in rows if target == 'ATE' or r[2] > r[1]]
return sum(r[2]-r[1] for r in chosen)/len(chosen) if chosen else None
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('ATT membership', solve([[1,0,2*N],[0,0,8*N]], 'ATT'), 2*N)
check('ATE', solve([[1,0,2*N],[0,0,8*N]], 'ATE'), 5*N)
check('negative treated', solve([[1,3*N,N],[0,0,N]], 'ATT'), -2*N)
check('no treated', solve([[0,0,N]], 'ATT'), None)
check('empty', solve([], 'ATE'), None)
check('null effect treated', solve([[1,N,N]], 'ATT'), 0)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| ATT membership | 5.0 | 2 | Failed |
| ATE | 5.0 | 5 | Passed |
| negative treated | 1.0 | -2 | Failed |
| no treated | 1.0 | None | Failed |
| empty | None | None | Passed |
| null effect treated | None | 0 | Failed |
SHA-256 / 7c03f25e16ea4e0346fe6b768637de0f83aab0a3158b7f90f2ad25e6ee22baf8
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(rows, target):
chosen = [r for r in rows if target == 'ATE' or r[0] == 1]
return sum(r[2]-r[1] for r in chosen)/len(chosen) if chosen else None
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('ATT membership', solve([[1,0,2*N],[0,0,8*N]], 'ATT'), 2*N)
check('ATE', solve([[1,0,2*N],[0,0,8*N]], 'ATE'), 5*N)
check('negative treated', solve([[1,3*N,N],[0,0,N]], 'ATT'), -2*N)
check('no treated', solve([[0,0,N]], 'ATT'), None)
check('empty', solve([], 'ATE'), None)
check('null effect treated', solve([[1,N,N]], 'ATT'), 0)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| ATT membership | 2.0 | 2 | Passed |
| ATE | 5.0 | 5 | Passed |
| negative treated | -2.0 | -2 | Passed |
| no treated | None | None | Passed |
| empty | None | None | Passed |
| null effect treated | 0.0 | 0 | Passed |
SHA-256 / 1daa1169f581048a28978c8c984ba41a939c3b08fc925cfad9ba92860db4e758
Verification & scope
Finite, fully specified synthetic data only; identification assumptions are supplied by the fixture design, not inferred from observations. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:38:58.793562+00:00.
Case digest / c9b7df43521233a454d972db12fc39fe7eb2a40f08179d292bdef14430eb6fd8