{"abstract":"Evaluation errors are mistaken for healthy results.","category":"Observability","checks":8,"contract":"Disabled rules become disabled; absent observations become no-data; explicit evaluation error becomes error; values above threshold become firing after required consecutive count, otherwise pending; healthy observations reset count to zero.","evaluation_group":"xt-alert-state","failed_approach":"The attempted repair substitutes if error: return ('no-data', count). Fixture 2 still yields ('no-data', 2) instead of ('error', 2).","family":"xt-alert-state-error-state","id":"FA-10311","implementations":{"attempt":{"sha256":"a8fe982e39eab087dc8cd207c2b0acaba1fd184234c5aa96096273e3a4a183dd","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(enabled, value, error, threshold, count, required):\n    if not enabled: return ('disabled', 0)\n    if error: return ('no-data', count)\n    if value is None: return ('no-data', count)\n    if value <= threshold: return ('ok', 0)\n    count += 1\n    return ('firing' if count >= required else 'pending', count)\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('fixture 1', solve(False, 9, False, 5, 2, 3), ('disabled', 0))\ncheck('fixture 2', solve(True, 9, True, 5, 2, 3), ('error', 2))\ncheck('fixture 3', solve(True, None, False, 5, 2, 3), ('no-data', 2))\ncheck('fixture 4', solve(True, 5, False, 5, 2, 3), ('ok', 0))\ncheck('fixture 5', solve(True, 4, False, 5, 2, 3), ('ok', 0))\ncheck('fixture 6', solve(True, 6, False, 5, 1, 3), ('pending', 2))\ncheck('fixture 7', solve(True, 6, False, 5, 2, 3), ('firing', 3))\ncheck('fixture 8', solve(True, 6, False, 5, 3, 3), ('firing', 4))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"52758850c83105a1df0c1a98cf7e7d277214b7727da1c139e6c58765d84c1f00","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(enabled, value, error, threshold, count, required):\n    if not enabled: return ('disabled', 0)\n    if error: return ('ok', 0)\n    if value is None: return ('no-data', count)\n    if value <= threshold: return ('ok', 0)\n    count += 1\n    return ('firing' if count >= required else 'pending', count)\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('fixture 1', solve(False, 9, False, 5, 2, 3), ('disabled', 0))\ncheck('fixture 2', solve(True, 9, True, 5, 2, 3), ('error', 2))\ncheck('fixture 3', solve(True, None, False, 5, 2, 3), ('no-data', 2))\ncheck('fixture 4', solve(True, 5, False, 5, 2, 3), ('ok', 0))\ncheck('fixture 5', solve(True, 4, False, 5, 2, 3), ('ok', 0))\ncheck('fixture 6', solve(True, 6, False, 5, 1, 3), ('pending', 2))\ncheck('fixture 7', solve(True, 6, False, 5, 2, 3), ('firing', 3))\ncheck('fixture 8', solve(True, 6, False, 5, 3, 3), ('firing', 4))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"fixed":{"sha256":"35c4b559190c1ca2e050e8d7eb5ff1ba4510b8a78cb28644c58185ff701d959c","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(enabled, value, error, threshold, count, required):\n    if not enabled: return ('disabled', 0)\n    if error: return ('error', count)\n    if value is None: return ('no-data', count)\n    if value <= threshold: return ('ok', 0)\n    count += 1\n    return ('firing' if count >= required else 'pending', count)\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('fixture 1', solve(False, 9, False, 5, 2, 3), ('disabled', 0))\ncheck('fixture 2', solve(True, 9, True, 5, 2, 3), ('error', 2))\ncheck('fixture 3', solve(True, None, False, 5, 2, 3), ('no-data', 2))\ncheck('fixture 4', solve(True, 5, False, 5, 2, 3), ('ok', 0))\ncheck('fixture 5', solve(True, 4, False, 5, 2, 3), ('ok', 0))\ncheck('fixture 6', solve(True, 6, False, 5, 1, 3), ('pending', 2))\ncheck('fixture 7', solve(True, 6, False, 5, 2, 3), ('firing', 3))\ncheck('fixture 8', solve(True, 6, False, 5, 3, 3), ('firing', 4))\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":" This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"xt-alert-state-error-state","generated_at":"2026-09-29T14:38:37.877042+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"An offline model of alert evaluation state transition, suitable for testing build and release tooling without external services.","repair":"Preserve evaluation error state and prior count.","root_cause":"The implementation substitutes if error: return ('ok', 0) for if error: return ('error', count), so evaluation errors are mistaken for healthy results.","sha256":"4208421580b7537f4a2b26bd4f607de16892e565f29a946ca3b503c4c4983bd7","title":"Alert evaluation state transition: Evaluation errors are mistaken for healthy results · case 01","variant":1,"variant_policy":"Five numbered records share a model and may reuse boundary fixtures.","verification":{"attempt":{"elapsed_ms":40.015,"exit_code":1,"observations":[{"actual":["disabled",0],"check":"fixture 1","expected":["disabled",0],"passed":true},{"actual":["no-data",2],"check":"fixture 2","expected":["error",2],"passed":false},{"actual":["no-data",2],"check":"fixture 3","expected":["no-data",2],"passed":true},{"actual":["ok",0],"check":"fixture 4","expected":["ok",0],"passed":true},{"actual":["ok",0],"check":"fixture 5","expected":["ok",0],"passed":true},{"actual":["pending",2],"check":"fixture 6","expected":["pending",2],"passed":true},{"actual":["firing",3],"check":"fixture 7","expected":["firing",3],"passed":true},{"actual":["firing",4],"check":"fixture 8","expected":["firing",4],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"fixture 1\", \"actual\": [\"disabled\", 0], \"expected\": [\"disabled\", 0], \"passed\": true}, {\"check\": \"fixture 2\", \"actual\": [\"no-data\", 2], \"expected\": [\"error\", 2], \"passed\": false}, {\"check\": \"fixture 3\", \"actual\": [\"no-data\", 2], \"expected\": [\"no-data\", 2], \"passed\": true}, {\"check\": \"fixture 4\", \"actual\": [\"ok\", 0], \"expected\": [\"ok\", 0], \"passed\": true}, {\"check\": \"fixture 5\", \"actual\": [\"ok\", 0], \"expected\": [\"ok\", 0], \"passed\": true}, {\"check\": \"fixture 6\", \"actual\": [\"pending\", 2], \"expected\": [\"pending\", 2], \"passed\": true}, {\"check\": \"fixture 7\", \"actual\": [\"firing\", 3], \"expected\": [\"firing\", 3], \"passed\": true}, {\"check\": \"fixture 8\", \"actual\": [\"firing\", 4], \"expected\": [\"firing\", 4], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":41.253,"exit_code":1,"observations":[{"actual":["disabled",0],"check":"fixture 1","expected":["disabled",0],"passed":true},{"actual":["ok",0],"check":"fixture 2","expected":["error",2],"passed":false},{"actual":["no-data",2],"check":"fixture 3","expected":["no-data",2],"passed":true},{"actual":["ok",0],"check":"fixture 4","expected":["ok",0],"passed":true},{"actual":["ok",0],"check":"fixture 5","expected":["ok",0],"passed":true},{"actual":["pending",2],"check":"fixture 6","expected":["pending",2],"passed":true},{"actual":["firing",3],"check":"fixture 7","expected":["firing",3],"passed":true},{"actual":["firing",4],"check":"fixture 8","expected":["firing",4],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"fixture 1\", \"actual\": [\"disabled\", 0], \"expected\": [\"disabled\", 0], \"passed\": true}, {\"check\": \"fixture 2\", \"actual\": [\"ok\", 0], \"expected\": [\"error\", 2], \"passed\": false}, {\"check\": \"fixture 3\", \"actual\": [\"no-data\", 2], \"expected\": [\"no-data\", 2], \"passed\": true}, {\"check\": \"fixture 4\", \"actual\": [\"ok\", 0], \"expected\": [\"ok\", 0], \"passed\": true}, {\"check\": \"fixture 5\", \"actual\": [\"ok\", 0], \"expected\": [\"ok\", 0], \"passed\": true}, {\"check\": \"fixture 6\", \"actual\": [\"pending\", 2], \"expected\": [\"pending\", 2], \"passed\": true}, {\"check\": \"fixture 7\", \"actual\": [\"firing\", 3], \"expected\": [\"firing\", 3], \"passed\": true}, {\"check\": \"fixture 8\", \"actual\": [\"firing\", 4], \"expected\": [\"firing\", 4], \"passed\": true}], \"passed\": false}\n"},"fixed":{"elapsed_ms":41.589,"exit_code":0,"observations":[{"actual":["disabled",0],"check":"fixture 1","expected":["disabled",0],"passed":true},{"actual":["error",2],"check":"fixture 2","expected":["error",2],"passed":true},{"actual":["no-data",2],"check":"fixture 3","expected":["no-data",2],"passed":true},{"actual":["ok",0],"check":"fixture 4","expected":["ok",0],"passed":true},{"actual":["ok",0],"check":"fixture 5","expected":["ok",0],"passed":true},{"actual":["pending",2],"check":"fixture 6","expected":["pending",2],"passed":true},{"actual":["firing",3],"check":"fixture 7","expected":["firing",3],"passed":true},{"actual":["firing",4],"check":"fixture 8","expected":["firing",4],"passed":true}],"passed":true,"stderr":"","stdout":"{\"observations\": [{\"check\": \"fixture 1\", \"actual\": [\"disabled\", 0], \"expected\": [\"disabled\", 0], \"passed\": true}, {\"check\": \"fixture 2\", \"actual\": [\"error\", 2], \"expected\": [\"error\", 2], \"passed\": true}, {\"check\": \"fixture 3\", \"actual\": [\"no-data\", 2], \"expected\": [\"no-data\", 2], \"passed\": true}, {\"check\": \"fixture 4\", \"actual\": [\"ok\", 0], \"expected\": [\"ok\", 0], \"passed\": true}, {\"check\": \"fixture 5\", \"actual\": [\"ok\", 0], \"expected\": [\"ok\", 0], \"passed\": true}, {\"check\": \"fixture 6\", \"actual\": [\"pending\", 2], \"expected\": [\"pending\", 2], \"passed\": true}, {\"check\": \"fixture 7\", \"actual\": [\"firing\", 3], \"expected\": [\"firing\", 3], \"passed\": true}, {\"check\": \"fixture 8\", \"actual\": [\"firing\", 4], \"expected\": [\"firing\", 4], \"passed\": true}], \"passed\": true}\n"}},"verified":true,"visibility":"public"}