{"abstract":"A treatment that changes who triggers distorts the diluted estimate.","category":"Experiment statistics","checks":8,"contract":"delta_trig is the absolute effect among triggered users. The trigger rate is pooled across arms: (trig_c + trig_t) / (n_c + n_t). The diluted site-wide effect is delta_trig * rate and the relative effect divides that by the overall control mean. No users or zero overall mean -> None. Return [rate, diluted absolute, diluted relative] rounded to 6.","evaluation_group":"w2-experiment-statistics-trigger-dilution","failed_approach":"Averaging the two arm rates weights a small arm as heavily as a large one.","family":"w2-experiment-statistics-trigger-dilution-pooled-trigger-rate","id":"FA-74606","implementations":{"attempt":{"sha256":"23ee91362027e2f0675519c1fa02ea89d239357656b6ada91805ef6a977a9260","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(delta_trig, trig_c, n_c, trig_t, n_t, overall_mean_c):\n    if n_c + n_t == 0 or overall_mean_c == 0:\n        return None\n    rate = (trig_c / n_c + trig_t / n_t) / 2\n    absolute = delta_trig * rate\n    return [round(rate, 6), round(absolute, 6), round(absolute / overall_mean_c, 6)]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 1', [0.5, 4, 400, 61, 100, 4.0], [0.13, 0.065, 0.01625]),\n  ('trigger sample 2', [0.1, 25, 100, 28, 300, 1.0], [0.1325, 0.01325, 0.01325]),\n  ('trigger sample 3', [0.5, 640, 1000, 682, 1000, 10.0], [0.661, 0.3305, 0.03305])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 6', [0.5, 791, 1000, 75, 100, 4.0], [0.787273, 0.393636, 0.098409]),\n  ('trigger sample 7', [2.0, 92, 100, 4, 100, 1.0], [0.48, 0.96, 0.96]),\n  ('trigger sample 9', [2.0, 873, 1000, 77, 100, 1.0], [0.863636, 1.727273, 1.727273])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 11', [0.1, 40, 100, 14, 100, 4.0], [0.27, 0.027, 0.00675]),\n  ('trigger sample 13', [2.0, 183, 400, 86, 300, 10.0], [0.384286, 0.768571, 0.076857]),\n  ('trigger sample 19', [-1.0, 294, 1000, 72, 100, 4.0], [0.332727, -0.332727, -0.083182])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 16', [2.0, 50, 100, 332, 1000, 10.0], [0.347273, 0.694545, 0.069455]),\n  ('trigger sample 20', [0.1, 357, 400, 89, 300, 4.0], [0.637143, 0.063714, 0.015929]),\n  ('trigger sample 27', [2.0, 77, 100, 143, 1000, 4.0], [0.2, 0.4, 0.1])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 21', [-1.0, 26, 100, 60, 100, 4.0], [0.43, -0.43, -0.1075]),\n  ('trigger sample 27', [2.0, 77, 100, 143, 1000, 4.0], [0.2, 0.4, 0.1]),\n  ('trigger sample 36', [-1.0, 456, 1000, 75, 100, 4.0], [0.482727, -0.482727, -0.120682])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"f4cda56d9a534aa6cdc6d07397c36d557a891c43aa2871e147a631a863f115df","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(delta_trig, trig_c, n_c, trig_t, n_t, overall_mean_c):\n    if n_c + n_t == 0 or overall_mean_c == 0:\n        return None\n    rate = trig_t / n_t\n    absolute = delta_trig * rate\n    return [round(rate, 6), round(absolute, 6), round(absolute / overall_mean_c, 6)]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 1', [0.5, 4, 400, 61, 100, 4.0], [0.13, 0.065, 0.01625]),\n  ('trigger sample 2', [0.1, 25, 100, 28, 300, 1.0], [0.1325, 0.01325, 0.01325]),\n  ('trigger sample 3', [0.5, 640, 1000, 682, 1000, 10.0], [0.661, 0.3305, 0.03305])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 6', [0.5, 791, 1000, 75, 100, 4.0], [0.787273, 0.393636, 0.098409]),\n  ('trigger sample 7', [2.0, 92, 100, 4, 100, 1.0], [0.48, 0.96, 0.96]),\n  ('trigger sample 9', [2.0, 873, 1000, 77, 100, 1.0], [0.863636, 1.727273, 1.727273])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 11', [0.1, 40, 100, 14, 100, 4.0], [0.27, 0.027, 0.00675]),\n  ('trigger sample 13', [2.0, 183, 400, 86, 300, 10.0], [0.384286, 0.768571, 0.076857]),\n  ('trigger sample 19', [-1.0, 294, 1000, 72, 100, 4.0], [0.332727, -0.332727, -0.083182])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 16', [2.0, 50, 100, 332, 1000, 10.0], [0.347273, 0.694545, 0.069455]),\n  ('trigger sample 20', [0.1, 357, 400, 89, 300, 4.0], [0.637143, 0.063714, 0.015929]),\n  ('trigger sample 27', [2.0, 77, 100, 143, 1000, 4.0], [0.2, 0.4, 0.1])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 21', [-1.0, 26, 100, 60, 100, 4.0], [0.43, -0.43, -0.1075]),\n  ('trigger sample 27', [2.0, 77, 100, 143, 1000, 4.0], [0.2, 0.4, 0.1]),\n  ('trigger sample 36', [-1.0, 456, 1000, 75, 100, 4.0], [0.482727, -0.482727, -0.120682])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"fixed":{"sha256":"dc17d7a03d701660529ff71c09c3e0d12a9e75cbea8173f724c03e468c6a573f","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(delta_trig, trig_c, n_c, trig_t, n_t, overall_mean_c):\n    if n_c + n_t == 0 or overall_mean_c == 0:\n        return None\n    rate = (trig_c + trig_t) / (n_c + n_t)\n    absolute = delta_trig * rate\n    return [round(rate, 6), round(absolute, 6), round(absolute / overall_mean_c, 6)]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 1', [0.5, 4, 400, 61, 100, 4.0], [0.13, 0.065, 0.01625]),\n  ('trigger sample 2', [0.1, 25, 100, 28, 300, 1.0], [0.1325, 0.01325, 0.01325]),\n  ('trigger sample 3', [0.5, 640, 1000, 682, 1000, 10.0], [0.661, 0.3305, 0.03305])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 6', [0.5, 791, 1000, 75, 100, 4.0], [0.787273, 0.393636, 0.098409]),\n  ('trigger sample 7', [2.0, 92, 100, 4, 100, 1.0], [0.48, 0.96, 0.96]),\n  ('trigger sample 9', [2.0, 873, 1000, 77, 100, 1.0], [0.863636, 1.727273, 1.727273])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 11', [0.1, 40, 100, 14, 100, 4.0], [0.27, 0.027, 0.00675]),\n  ('trigger sample 13', [2.0, 183, 400, 86, 300, 10.0], [0.384286, 0.768571, 0.076857]),\n  ('trigger sample 19', [-1.0, 294, 1000, 72, 100, 4.0], [0.332727, -0.332727, -0.083182])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 16', [2.0, 50, 100, 332, 1000, 10.0], [0.347273, 0.694545, 0.069455]),\n  ('trigger sample 20', [0.1, 357, 400, 89, 300, 4.0], [0.637143, 0.063714, 0.015929]),\n  ('trigger sample 27', [2.0, 77, 100, 143, 1000, 4.0], [0.2, 0.4, 0.1])],\n [('unequal arms pool the trigger rate', [2.0, 100, 1000, 30, 100, 10.0], [0.118182, 0.236364, 0.023636]),\n  ('relative effect uses overall mean', [0.5, 50, 100, 50, 100, 5.0], [0.5, 0.25, 0.05]),\n  ('negative effect dilutes too', [-1.0, 10, 400, 30, 300, 4.0], [0.057143, -0.057143, -0.014286]),\n  ('nobody triggered', [3.0, 0, 100, 0, 100, 2.0], [0.0, 0.0, 0.0]),\n  ('zero overall mean', [1.0, 5, 10, 5, 10, 0.0], None),\n  ('trigger sample 21', [-1.0, 26, 100, 60, 100, 4.0], [0.43, -0.43, -0.1075]),\n  ('trigger sample 27', [2.0, 77, 100, 143, 1000, 4.0], [0.2, 0.4, 0.1]),\n  ('trigger sample 36', [-1.0, 456, 1000, 75, 100, 4.0], [0.482727, -0.482727, -0.120682])]]\nfor label, args, expected in fixtures[N - 1]:\n    check(label, solve(*args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"A deterministic toy experiment-analysis model with a stipulated contract; results are rounded and are not a substitute for a validated statistics package. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w2-experiment-statistics-trigger-dilution-pooled-trigger-rate","generated_at":"2026-09-29T14:48:58.409194+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"Triggered analyses are sensitive but must be diluted before claiming site-wide impact.","repair":"Pool triggered and total users across both arms.","root_cause":"rate = trig_t / n_t.","sha256":"20e785fc31c1cb15d12164258c166bdfacdc1b3cbee07705064f7ee089fa5f87","title":"Triggered effect dilution: The trigger rate comes from the treatment arm · case 01","variant":1,"variant_policy":"Five numbered records share a model and may reuse boundary fixtures.","verification":{"attempt":{"elapsed_ms":40.271,"exit_code":1,"observations":[{"actual":[0.2,0.4,0.04],"check":"unequal arms pool the trigger rate","expected":[0.118182,0.236364,0.023636],"passed":false},{"actual":[0.5,0.25,0.05],"check":"relative effect uses overall mean","expected":[0.5,0.25,0.05],"passed":true},{"actual":[0.0625,-0.0625,-0.015625],"check":"negative effect dilutes too","expected":[0.057143,-0.057143,-0.014286],"passed":false},{"actual":[0.0,0.0,0.0],"check":"nobody triggered","expected":[0.0,0.0,0.0],"passed":true},{"actual":null,"check":"zero overall mean","expected":null,"passed":true},{"actual":[0.31,0.155,0.03875],"check":"trigger sample 1","expected":[0.13,0.065,0.01625],"passed":false},{"actual":[0.171667,0.017167,0.017167],"check":"trigger sample 2","expected":[0.1325,0.01325,0.01325],"passed":false},{"actual":[0.661,0.3305,0.03305],"check":"trigger sample 3","expected":[0.661,0.3305,0.03305],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"unequal arms pool the trigger rate\", \"actual\": [0.2, 0.4, 0.04], \"expected\": [0.118182, 0.236364, 0.023636], \"passed\": false}, {\"check\": \"relative effect uses overall mean\", \"actual\": [0.5, 0.25, 0.05], \"expected\": [0.5, 0.25, 0.05], \"passed\": true}, {\"check\": \"negative effect dilutes too\", \"actual\": [0.0625, -0.0625, -0.015625], \"expected\": [0.057143, -0.057143, -0.014286], \"passed\": false}, {\"check\": \"nobody triggered\", \"actual\": [0.0, 0.0, 0.0], \"expected\": [0.0, 0.0, 0.0], \"passed\": true}, {\"check\": \"zero overall mean\", \"actual\": null, \"expected\": null, \"passed\": true}, {\"check\": \"trigger sample 1\", \"actual\": [0.31, 0.155, 0.03875], \"expected\": [0.13, 0.065, 0.01625], \"passed\": false}, {\"check\": \"trigger sample 2\", \"actual\": [0.171667, 0.017167, 0.017167], \"expected\": [0.1325, 0.01325, 0.01325], \"passed\": false}, {\"check\": \"trigger sample 3\", \"actual\": [0.661, 0.3305, 0.03305], \"expected\": [0.661, 0.3305, 0.03305], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":39.639,"exit_code":1,"observations":[{"actual":[0.3,0.6,0.06],"check":"unequal arms pool the trigger rate","expected":[0.118182,0.236364,0.023636],"passed":false},{"actual":[0.5,0.25,0.05],"check":"relative effect uses overall mean","expected":[0.5,0.25,0.05],"passed":true},{"actual":[0.1,-0.1,-0.025],"check":"negative effect dilutes too","expected":[0.057143,-0.057143,-0.014286],"passed":false},{"actual":[0.0,0.0,0.0],"check":"nobody triggered","expected":[0.0,0.0,0.0],"passed":true},{"actual":null,"check":"zero overall mean","expected":null,"passed":true},{"actual":[0.61,0.305,0.07625],"check":"trigger sample 1","expected":[0.13,0.065,0.01625],"passed":false},{"actual":[0.093333,0.009333,0.009333],"check":"trigger sample 2","expected":[0.1325,0.01325,0.01325],"passed":false},{"actual":[0.682,0.341,0.0341],"check":"trigger sample 3","expected":[0.661,0.3305,0.03305],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"unequal arms pool the trigger rate\", \"actual\": [0.3, 0.6, 0.06], \"expected\": [0.118182, 0.236364, 0.023636], \"passed\": false}, {\"check\": \"relative effect uses overall mean\", \"actual\": [0.5, 0.25, 0.05], \"expected\": [0.5, 0.25, 0.05], \"passed\": true}, {\"check\": \"negative effect dilutes too\", \"actual\": [0.1, -0.1, -0.025], \"expected\": [0.057143, -0.057143, -0.014286], \"passed\": false}, {\"check\": \"nobody triggered\", \"actual\": [0.0, 0.0, 0.0], \"expected\": [0.0, 0.0, 0.0], \"passed\": true}, {\"check\": \"zero overall mean\", \"actual\": null, \"expected\": null, \"passed\": true}, {\"check\": \"trigger sample 1\", \"actual\": [0.61, 0.305, 0.07625], \"expected\": [0.13, 0.065, 0.01625], \"passed\": false}, {\"check\": \"trigger sample 2\", \"actual\": [0.093333, 0.009333, 0.009333], \"expected\": [0.1325, 0.01325, 0.01325], \"passed\": false}, {\"check\": \"trigger sample 3\", \"actual\": [0.682, 0.341, 0.0341], \"expected\": [0.661, 0.3305, 0.03305], \"passed\": false}], \"passed\": false}\n"},"fixed":{"elapsed_ms":38.969,"exit_code":0,"observations":[{"actual":[0.118182,0.236364,0.023636],"check":"unequal arms pool the trigger rate","expected":[0.118182,0.236364,0.023636],"passed":true},{"actual":[0.5,0.25,0.05],"check":"relative effect uses overall mean","expected":[0.5,0.25,0.05],"passed":true},{"actual":[0.057143,-0.057143,-0.014286],"check":"negative effect dilutes too","expected":[0.057143,-0.057143,-0.014286],"passed":true},{"actual":[0.0,0.0,0.0],"check":"nobody triggered","expected":[0.0,0.0,0.0],"passed":true},{"actual":null,"check":"zero overall mean","expected":null,"passed":true},{"actual":[0.13,0.065,0.01625],"check":"trigger sample 1","expected":[0.13,0.065,0.01625],"passed":true},{"actual":[0.1325,0.01325,0.01325],"check":"trigger sample 2","expected":[0.1325,0.01325,0.01325],"passed":true},{"actual":[0.661,0.3305,0.03305],"check":"trigger sample 3","expected":[0.661,0.3305,0.03305],"passed":true}],"passed":true,"stderr":"","stdout":"{\"observations\": [{\"check\": \"unequal arms pool the trigger rate\", \"actual\": [0.118182, 0.236364, 0.023636], \"expected\": [0.118182, 0.236364, 0.023636], \"passed\": true}, {\"check\": \"relative effect uses overall mean\", \"actual\": [0.5, 0.25, 0.05], \"expected\": [0.5, 0.25, 0.05], \"passed\": true}, {\"check\": \"negative effect dilutes too\", \"actual\": [0.057143, -0.057143, -0.014286], \"expected\": [0.057143, -0.057143, -0.014286], \"passed\": true}, {\"check\": \"nobody triggered\", \"actual\": [0.0, 0.0, 0.0], \"expected\": [0.0, 0.0, 0.0], \"passed\": true}, {\"check\": \"zero overall mean\", \"actual\": null, \"expected\": null, \"passed\": true}, {\"check\": \"trigger sample 1\", \"actual\": [0.13, 0.065, 0.01625], \"expected\": [0.13, 0.065, 0.01625], \"passed\": true}, {\"check\": \"trigger sample 2\", \"actual\": [0.1325, 0.01325, 0.01325], \"expected\": [0.1325, 0.01325, 0.01325], \"passed\": true}, {\"check\": \"trigger sample 3\", \"actual\": [0.661, 0.3305, 0.03305], \"expected\": [0.661, 0.3305, 0.03305], \"passed\": true}], \"passed\": true}\n"}},"verified":true,"visibility":"public"}