{"abstract":"CACE is understated when control users can reach the feature through another path.","category":"Experiment statistics","checks":8,"contract":"Outcome sums cover every assigned user. ITT = outcome_sum_t / assigned_t - outcome_sum_c / assigned_c. Compliance difference = took_t / assigned_t - took_c / assigned_c (control users can cross over). CACE = ITT / compliance difference when that difference is positive, else None. Nonpositive assignment counts -> None. Return [ITT, compliance difference, CACE] rounded to 6.","contract_signature":"assigned_c, took_c, outcome_sum_c, assigned_t, took_t, outcome_sum_t","evaluation_group":"w2-experiment-statistics-complier-effect","failed_approach":"Subtracting raw counts over the treatment size ignores unequal arm sizes.","family":"w2-experiment-statistics-complier-effect-control-crossover","id":"FA-74696","implementations":{"attempt":{"sha256":"e6673d13f4063be5b27b8db4aaffd53d951149388f3258bda408cc710b10b75e","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(assigned_c, took_c, outcome_sum_c, assigned_t, took_t, outcome_sum_t):\n    if assigned_c <= 0 or assigned_t <= 0:\n        return None\n    itt = outcome_sum_t / assigned_t - outcome_sum_c / assigned_c\n    comp = (took_t - took_c) / assigned_t\n    cace = round(itt / comp, 6) if comp > 0 else None\n    return [round(itt, 6), round(comp, 6), cace]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 1', [200, 22, 428, 400, 23, 389], [-1.1675, -0.0525, None]),\n  ('uptake sample 2', [100, 22, 279, 400, 165, 505], [-1.5275, 0.1925, -7.935065]),\n  ('uptake sample 3', [100, 11, 241, 400, 240, 17], [-2.3675, 0.49, -4.831633])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 5', [200, 62, 357, 300, 290, 266], [-0.898333, 0.656667, -1.36802]),\n  ('uptake sample 6', [400, 10, 585, 300, 223, 613], [0.580833, 0.718333, 0.808585]),\n  ('uptake sample 12', [100, 3, 98, 400, 319, 1150], [1.895, 0.7675, 2.469055])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 11', [400, 10, 635, 100, 93, 173], [0.1425, 0.905, 0.157459]),\n  ('uptake sample 12', [100, 3, 98, 400, 319, 1150], [1.895, 0.7675, 2.469055]),\n  ('uptake sample 19', [200, 30, 146, 300, 293, 69], [-0.5, 0.826667, -0.604839])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 16', [100, 6, 298, 300, 104, 511], [-1.276667, 0.286667, -4.453488]),\n  ('uptake sample 19', [200, 30, 146, 300, 293, 69], [-0.5, 0.826667, -0.604839]),\n  ('uptake sample 29', [400, 11, 731, 100, 70, 53], [-1.2975, 0.6725, -1.929368])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 21', [100, 18, 78, 100, 98, 114], [0.36, 0.8, 0.45]),\n  ('uptake sample 26', [200, 20, 354, 300, 180, 2], [-1.763333, 0.5, -3.526667]),\n  ('uptake sample 39', [200, 59, 406, 400, 358, 158], [-1.635, 0.6, -2.725])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"4d4328a897fa475ea8b8fde21b5acf24404b9656c86eb6287a0d4abae558f08a","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(assigned_c, took_c, outcome_sum_c, assigned_t, took_t, outcome_sum_t):\n    if assigned_c <= 0 or assigned_t <= 0:\n        return None\n    itt = outcome_sum_t / assigned_t - outcome_sum_c / assigned_c\n    comp = took_t / assigned_t\n    cace = round(itt / comp, 6) if comp > 0 else None\n    return [round(itt, 6), round(comp, 6), cace]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 1', [200, 22, 428, 400, 23, 389], [-1.1675, -0.0525, None]),\n  ('uptake sample 2', [100, 22, 279, 400, 165, 505], [-1.5275, 0.1925, -7.935065]),\n  ('uptake sample 3', [100, 11, 241, 400, 240, 17], [-2.3675, 0.49, -4.831633])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 5', [200, 62, 357, 300, 290, 266], [-0.898333, 0.656667, -1.36802]),\n  ('uptake sample 6', [400, 10, 585, 300, 223, 613], [0.580833, 0.718333, 0.808585]),\n  ('uptake sample 12', [100, 3, 98, 400, 319, 1150], [1.895, 0.7675, 2.469055])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 11', [400, 10, 635, 100, 93, 173], [0.1425, 0.905, 0.157459]),\n  ('uptake sample 12', [100, 3, 98, 400, 319, 1150], [1.895, 0.7675, 2.469055]),\n  ('uptake sample 19', [200, 30, 146, 300, 293, 69], [-0.5, 0.826667, -0.604839])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 16', [100, 6, 298, 300, 104, 511], [-1.276667, 0.286667, -4.453488]),\n  ('uptake sample 19', [200, 30, 146, 300, 293, 69], [-0.5, 0.826667, -0.604839]),\n  ('uptake sample 29', [400, 11, 731, 100, 70, 53], [-1.2975, 0.6725, -1.929368])],\n [('crossover in control reduces compliance difference', [200, 20, 300, 200, 120, 420], [0.6, 0.5, 1.2]),\n  ('unequal arm sizes', [100, 0, 150, 400, 200, 800], [0.5, 0.5, 1.0]),\n  ('no uptake difference has no CACE', [100, 50, 100, 100, 50, 120], [0.2, 0.0, None]),\n  ('low but positive compliance still yields CACE', [100, 0, 100, 100, 30, 130], [0.3, 0.3, 1.0]),\n  ('negative compliance difference', [100, 60, 100, 100, 40, 90], [-0.1, -0.2, None]),\n  ('uptake sample 21', [100, 18, 78, 100, 98, 114], [0.36, 0.8, 0.45]),\n  ('uptake sample 26', [200, 20, 354, 300, 180, 2], [-1.763333, 0.5, -3.526667]),\n  ('uptake sample 39', [200, 59, 406, 400, 358, 158], [-1.635, 0.6, -2.725])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w2-experiment-statistics-complier-effect-control-crossover","generated_at":"2026-09-29T14:48:59.157804+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"Opt-in features have partial uptake; the effect on adopters needs an instrumental-variable estimate.","root_cause":"The compliance difference is just the treatment uptake rate.","sha256":"60859a83e4ad9602fc870a73476b3c3415badf3f250e62c926e4423c608a87a0","title":"Complier average causal effect: Control crossover is ignored in compliance · case 01","variant":1,"variant_policy":"Five numbered records share a model and may reuse boundary fixtures.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":38.925,"exit_code":1,"observations":[{"actual":[0.6,0.5,1.2],"check":"crossover in control reduces compliance difference","expected":[0.6,0.5,1.2],"passed":true},{"actual":[0.5,0.5,1.0],"check":"unequal arm sizes","expected":[0.5,0.5,1.0],"passed":true},{"actual":[0.2,0.0,null],"check":"no uptake difference has no CACE","expected":[0.2,0.0,null],"passed":true},{"actual":[0.3,0.3,1.0],"check":"low but positive compliance still yields CACE","expected":[0.3,0.3,1.0],"passed":true},{"actual":[-0.1,-0.2,null],"check":"negative compliance difference","expected":[-0.1,-0.2,null],"passed":true},{"actual":[-1.1675,0.0025,-467.0],"check":"uptake sample 1","expected":[-1.1675,-0.0525,null],"passed":false},{"actual":[-1.5275,0.3575,-4.272727],"check":"uptake sample 2","expected":[-1.5275,0.1925,-7.935065],"passed":false},{"actual":[-2.3675,0.5725,-4.135371],"check":"uptake sample 3","expected":[-2.3675,0.49,-4.831633],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"crossover in control reduces compliance difference\", \"actual\": [0.6, 0.5, 1.2], \"expected\": [0.6, 0.5, 1.2], \"passed\": true}, {\"check\": \"unequal arm sizes\", \"actual\": [0.5, 0.5, 1.0], \"expected\": [0.5, 0.5, 1.0], \"passed\": true}, {\"check\": \"no uptake difference has no CACE\", \"actual\": [0.2, 0.0, null], \"expected\": [0.2, 0.0, null], \"passed\": true}, {\"check\": \"low but positive compliance still yields CACE\", \"actual\": [0.3, 0.3, 1.0], \"expected\": [0.3, 0.3, 1.0], \"passed\": true}, {\"check\": \"negative compliance difference\", \"actual\": [-0.1, -0.2, null], \"expected\": [-0.1, -0.2, null], \"passed\": true}, {\"check\": \"uptake sample 1\", \"actual\": [-1.1675, 0.0025, -467.0], \"expected\": [-1.1675, -0.0525, null], \"passed\": false}, {\"check\": \"uptake sample 2\", \"actual\": [-1.5275, 0.3575, -4.272727], \"expected\": [-1.5275, 0.1925, -7.935065], \"passed\": false}, {\"check\": \"uptake sample 3\", \"actual\": [-2.3675, 0.5725, -4.135371], \"expected\": [-2.3675, 0.49, -4.831633], \"passed\": false}], \"passed\": false}\n"},"broken":{"elapsed_ms":38.558,"exit_code":1,"observations":[{"actual":[0.6,0.6,1.0],"check":"crossover in control reduces compliance difference","expected":[0.6,0.5,1.2],"passed":false},{"actual":[0.5,0.5,1.0],"check":"unequal arm sizes","expected":[0.5,0.5,1.0],"passed":true},{"actual":[0.2,0.5,0.4],"check":"no uptake difference has no CACE","expected":[0.2,0.0,null],"passed":false},{"actual":[0.3,0.3,1.0],"check":"low but positive compliance still yields CACE","expected":[0.3,0.3,1.0],"passed":true},{"actual":[-0.1,0.4,-0.25],"check":"negative compliance difference","expected":[-0.1,-0.2,null],"passed":false},{"actual":[-1.1675,0.0575,-20.304348],"check":"uptake sample 1","expected":[-1.1675,-0.0525,null],"passed":false},{"actual":[-1.5275,0.4125,-3.70303],"check":"uptake sample 2","expected":[-1.5275,0.1925,-7.935065],"passed":false},{"actual":[-2.3675,0.6,-3.945833],"check":"uptake sample 3","expected":[-2.3675,0.49,-4.831633],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"crossover in control reduces compliance difference\", \"actual\": [0.6, 0.6, 1.0], \"expected\": [0.6, 0.5, 1.2], \"passed\": false}, {\"check\": \"unequal arm sizes\", \"actual\": [0.5, 0.5, 1.0], \"expected\": [0.5, 0.5, 1.0], \"passed\": true}, {\"check\": \"no uptake difference has no CACE\", \"actual\": [0.2, 0.5, 0.4], \"expected\": [0.2, 0.0, null], \"passed\": false}, {\"check\": \"low but positive compliance still yields CACE\", \"actual\": [0.3, 0.3, 1.0], \"expected\": [0.3, 0.3, 1.0], \"passed\": true}, {\"check\": \"negative compliance difference\", \"actual\": [-0.1, 0.4, -0.25], \"expected\": [-0.1, -0.2, null], \"passed\": false}, {\"check\": \"uptake sample 1\", \"actual\": [-1.1675, 0.0575, -20.304348], \"expected\": [-1.1675, -0.0525, null], \"passed\": false}, {\"check\": \"uptake sample 2\", \"actual\": [-1.5275, 0.4125, -3.70303], \"expected\": [-1.5275, 0.1925, -7.935065], \"passed\": false}, {\"check\": \"uptake sample 3\", \"actual\": [-2.3675, 0.6, -3.945833], \"expected\": [-2.3675, 0.49, -4.831633], \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}