{"abstract":"An automatic signal stays at danger for good after the first train.","category":"Railway interlocking logic","checks":8,"contract":"Each step gives [section beyond occupied, route set, approach occupied]. An auto signal behaves as permanently routed. A signal shows danger whenever the section beyond is occupied; a signal that was clear when a train occupies that section latches a stick that keeps it at danger until the route is set afresh (off then on); auto signals drop the stick every step. Otherwise it clears when routed, not stuck, and for approach-released signals only with the approach occupied. Output the state per step.","contract_signature":"x","evaluation_group":"w2-railway_interlocking_logic-signal-replacement-stick","failed_approach":"Resetting only when a train approaches keeps an automatic signal red with an empty approach.","family":"w2-railway_interlocking_logic-signal-replacement-stick-automatic-stick-reset","id":"FA-67191","implementations":{"attempt":{"sha256":"a8752d244c321685ae582954701b14771b4d05386dd3cdc57a972c1d95355bb8","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(x):\n    mode = x['mode']\n    state = 'danger'\n    stick = False\n    prev_route = False\n    out = []\n    for occ, route, appr in x['steps']:\n        if mode == 'auto':\n            route = True\n        if (route and not prev_route) or (mode == 'auto' and appr):\n            stick = False\n        if occ:\n            if state == 'clear':\n                stick = True\n            state = 'danger'\n        elif route and not stick and (mode != 'approach' or appr):\n            state = 'clear'\n        else:\n            state = 'danger'\n        prev_route = route\n        out.append(state)\n    return out\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 22', {'mode': 'auto', 'steps': [[True, False, True], [False, False, True], [False, True, False], [False, True, True], [False, False, False], [True, False, True], [False, True, True], [False, False, False], [True, False, False]]}, ['danger', 'clear', 'clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger']), ('sampled regression 11', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, False, False], [True, False, True], [False, False, False], [False, False, False], [True, False, True], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear']), ('boundary: second train behind on the same route', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('boundary: route re-set after a train', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, False, True], [False, True, True]]}, ['clear', 'danger', 'danger', 'clear']), ('control 1', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, True, False], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'clear', 'clear']), ('control 4', {'mode': 'controlled', 'steps': [[False, False, False], [False, True, True], [True, True, True], [False, True, True], [False, False, False]]}, ['danger', 'clear', 'danger', 'danger', 'danger']), ('control 7', {'mode': 'approach', 'steps': [[False, False, False], [False, False, False], [False, False, True], [False, False, True], [False, True, False], [False, False, True], [False, True, True], [False, True, True], [False, True, True], [True, True, True]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear', 'clear', 'clear', 'danger'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 60', {'mode': 'auto', 'steps': [[False, False, False], [True, True, False], [False, False, False], [False, False, False], [False, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'clear']), ('boundary: approach-released signal with empty approach', {'mode': 'approach', 'steps': [[False, True, False], [False, True, True]]}, ['danger', 'clear']), ('boundary: approach-released signal replaced by train', {'mode': 'approach', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('boundary: occupied before clearing', {'mode': 'auto', 'steps': [[True, False, False], [True, False, False], [False, False, False]]}, ['danger', 'danger', 'clear']), ('control 12', {'mode': 'controlled', 'steps': [[False, False, True], [False, True, True], [True, False, True], [False, False, False], [False, False, True], [True, True, True], [True, False, False]]}, ['danger', 'clear', 'danger', 'danger', 'danger', 'danger', 'danger']), ('control 15', {'mode': 'approach', 'steps': [[False, True, True], [False, True, False], [False, True, True]]}, ['clear', 'danger', 'clear']), ('control 18', {'mode': 'approach', 'steps': [[False, True, False], [False, False, False], [False, False, False], [True, True, False], [False, True, False], [True, True, True], [False, True, True]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 11', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, False, False], [True, False, True], [False, False, False], [False, False, False], [True, False, True], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear']), ('boundary: occupied before clearing', {'mode': 'auto', 'steps': [[True, False, False], [True, False, False], [False, False, False]]}, ['danger', 'danger', 'clear']), ('boundary: controlled signal with a standing train', {'mode': 'controlled', 'steps': [[True, True, True], [False, True, True]]}, ['danger', 'clear']), ('boundary: second train behind on the same route', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('control 23', {'mode': 'controlled', 'steps': [[False, True, True], [False, True, False], [False, True, True], [True, True, False], [False, True, False]]}, ['clear', 'clear', 'clear', 'danger', 'danger']), ('control 26', {'mode': 'approach', 'steps': [[False, False, False], [False, True, True], [True, True, False], [True, False, True], [True, True, False], [True, False, True], [False, False, False], [False, True, True], [False, False, True], [False, True, False]]}, ['danger', 'clear', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear', 'danger', 'danger']), ('sampled regression 29', {'mode': 'auto', 'steps': [[True, False, False], [False, False, False], [True, False, False], [False, False, True], [False, False, False], [False, True, True], [False, True, True], [False, False, False], [True, False, True], [False, True, False]]}, ['danger', 'clear', 'danger', 'clear', 'clear', 'clear', 'clear', 'clear', 'danger', 'clear'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 47', {'mode': 'auto', 'steps': [[True, True, True], [False, False, True], [True, False, False], [False, False, True], [False, False, True], [True, True, False], [False, True, True], [True, True, False], [False, False, False], [False, False, False]]}, ['danger', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'danger', 'clear', 'clear']), ('sampled regression 60', {'mode': 'auto', 'steps': [[False, False, False], [True, True, False], [False, False, False], [False, False, False], [False, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'clear']), ('boundary: controlled signal with a standing train', {'mode': 'controlled', 'steps': [[True, True, True], [False, True, True]]}, ['danger', 'clear']), ('boundary: second train behind on the same route', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('control 34', {'mode': 'approach', 'steps': [[False, True, True], [False, False, False], [False, True, True], [False, False, False], [True, False, True], [False, False, True], [True, False, False], [True, False, True]]}, ['clear', 'danger', 'clear', 'danger', 'danger', 'danger', 'danger', 'danger']), ('control 37', {'mode': 'approach', 'steps': [[False, False, True], [True, False, False], [False, False, True], [False, False, True], [True, False, True], [False, False, False], [False, False, True], [False, False, True], [False, False, False]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger']), ('sampled regression 40', {'mode': 'auto', 'steps': [[False, True, True], [True, True, True], [False, True, True], [False, True, False], [True, False, True], [True, False, True], [True, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'danger', 'danger', 'danger'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 70', {'mode': 'auto', 'steps': [[False, False, True], [True, False, False], [False, False, True], [False, False, True], [True, False, False], [False, False, True], [False, False, False], [False, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear', 'clear']), ('sampled regression 11', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, False, False], [True, False, True], [False, False, False], [False, False, False], [True, False, True], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear']), ('boundary: route re-set after a train', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, False, True], [False, True, True]]}, ['clear', 'danger', 'danger', 'clear']), ('boundary: approach-released signal with empty approach', {'mode': 'approach', 'steps': [[False, True, False], [False, True, True]]}, ['danger', 'clear']), ('control 45', {'mode': 'approach', 'steps': [[False, False, False], [False, False, False], [False, True, False], [False, True, False], [True, True, False], [False, True, False], [False, True, False], [False, True, True]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear']), ('control 48', {'mode': 'approach', 'steps': [[True, True, False], [False, True, True], [False, False, False], [False, False, True], [False, True, True], [True, True, False]]}, ['danger', 'clear', 'danger', 'danger', 'clear', 'danger']), ('control 51', {'mode': 'controlled', 'steps': [[False, False, True], [False, False, False], [False, True, False], [False, True, True], [True, False, True], [False, False, True], [True, True, False]]}, ['danger', 'danger', 'clear', 'clear', 'danger', 'danger', 'danger'])]]\nfor label, args, expected in fixtures[N-1]:\n    check(label, solve(args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"f96a14fab01aa7d5d918ac7fe7e10833dce26813cd1452ac5a7a8ae76ff9db9d","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(x):\n    mode = x['mode']\n    state = 'danger'\n    stick = False\n    prev_route = False\n    out = []\n    for occ, route, appr in x['steps']:\n        if mode == 'auto':\n            route = True\n        if (route and not prev_route):\n            stick = False\n        if occ:\n            if state == 'clear':\n                stick = True\n            state = 'danger'\n        elif route and not stick and (mode != 'approach' or appr):\n            state = 'clear'\n        else:\n            state = 'danger'\n        prev_route = route\n        out.append(state)\n    return out\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\nfixtures = [[('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 22', {'mode': 'auto', 'steps': [[True, False, True], [False, False, True], [False, True, False], [False, True, True], [False, False, False], [True, False, True], [False, True, True], [False, False, False], [True, False, False]]}, ['danger', 'clear', 'clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger']), ('sampled regression 11', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, False, False], [True, False, True], [False, False, False], [False, False, False], [True, False, True], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear']), ('boundary: second train behind on the same route', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('boundary: route re-set after a train', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, False, True], [False, True, True]]}, ['clear', 'danger', 'danger', 'clear']), ('control 1', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, True, False], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'clear', 'clear']), ('control 4', {'mode': 'controlled', 'steps': [[False, False, False], [False, True, True], [True, True, True], [False, True, True], [False, False, False]]}, ['danger', 'clear', 'danger', 'danger', 'danger']), ('control 7', {'mode': 'approach', 'steps': [[False, False, False], [False, False, False], [False, False, True], [False, False, True], [False, True, False], [False, False, True], [False, True, True], [False, True, True], [False, True, True], [True, True, True]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear', 'clear', 'clear', 'danger'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 60', {'mode': 'auto', 'steps': [[False, False, False], [True, True, False], [False, False, False], [False, False, False], [False, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'clear']), ('boundary: approach-released signal with empty approach', {'mode': 'approach', 'steps': [[False, True, False], [False, True, True]]}, ['danger', 'clear']), ('boundary: approach-released signal replaced by train', {'mode': 'approach', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('boundary: occupied before clearing', {'mode': 'auto', 'steps': [[True, False, False], [True, False, False], [False, False, False]]}, ['danger', 'danger', 'clear']), ('control 12', {'mode': 'controlled', 'steps': [[False, False, True], [False, True, True], [True, False, True], [False, False, False], [False, False, True], [True, True, True], [True, False, False]]}, ['danger', 'clear', 'danger', 'danger', 'danger', 'danger', 'danger']), ('control 15', {'mode': 'approach', 'steps': [[False, True, True], [False, True, False], [False, True, True]]}, ['clear', 'danger', 'clear']), ('control 18', {'mode': 'approach', 'steps': [[False, True, False], [False, False, False], [False, False, False], [True, True, False], [False, True, False], [True, True, True], [False, True, True]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 11', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, False, False], [True, False, True], [False, False, False], [False, False, False], [True, False, True], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear']), ('boundary: occupied before clearing', {'mode': 'auto', 'steps': [[True, False, False], [True, False, False], [False, False, False]]}, ['danger', 'danger', 'clear']), ('boundary: controlled signal with a standing train', {'mode': 'controlled', 'steps': [[True, True, True], [False, True, True]]}, ['danger', 'clear']), ('boundary: second train behind on the same route', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('control 23', {'mode': 'controlled', 'steps': [[False, True, True], [False, True, False], [False, True, True], [True, True, False], [False, True, False]]}, ['clear', 'clear', 'clear', 'danger', 'danger']), ('control 26', {'mode': 'approach', 'steps': [[False, False, False], [False, True, True], [True, True, False], [True, False, True], [True, True, False], [True, False, True], [False, False, False], [False, True, True], [False, False, True], [False, True, False]]}, ['danger', 'clear', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear', 'danger', 'danger']), ('sampled regression 29', {'mode': 'auto', 'steps': [[True, False, False], [False, False, False], [True, False, False], [False, False, True], [False, False, False], [False, True, True], [False, True, True], [False, False, False], [True, False, True], [False, True, False]]}, ['danger', 'clear', 'danger', 'clear', 'clear', 'clear', 'clear', 'clear', 'danger', 'clear'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 47', {'mode': 'auto', 'steps': [[True, True, True], [False, False, True], [True, False, False], [False, False, True], [False, False, True], [True, True, False], [False, True, True], [True, True, False], [False, False, False], [False, False, False]]}, ['danger', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'danger', 'clear', 'clear']), ('sampled regression 60', {'mode': 'auto', 'steps': [[False, False, False], [True, True, False], [False, False, False], [False, False, False], [False, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'clear']), ('boundary: controlled signal with a standing train', {'mode': 'controlled', 'steps': [[True, True, True], [False, True, True]]}, ['danger', 'clear']), ('boundary: second train behind on the same route', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, True, True]]}, ['clear', 'danger', 'danger']), ('control 34', {'mode': 'approach', 'steps': [[False, True, True], [False, False, False], [False, True, True], [False, False, False], [True, False, True], [False, False, True], [True, False, False], [True, False, True]]}, ['clear', 'danger', 'clear', 'danger', 'danger', 'danger', 'danger', 'danger']), ('control 37', {'mode': 'approach', 'steps': [[False, False, True], [True, False, False], [False, False, True], [False, False, True], [True, False, True], [False, False, False], [False, False, True], [False, False, True], [False, False, False]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger']), ('sampled regression 40', {'mode': 'auto', 'steps': [[False, True, True], [True, True, True], [False, True, True], [False, True, False], [True, False, True], [True, False, True], [True, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'danger', 'danger', 'danger'])], [('regression: automatic signal re-clears', {'mode': 'auto', 'steps': [[False, False, False], [True, False, True], [False, False, False]]}, ['clear', 'danger', 'clear']), ('sampled regression 70', {'mode': 'auto', 'steps': [[False, False, True], [True, False, False], [False, False, True], [False, False, True], [True, False, False], [False, False, True], [False, False, False], [False, False, True]]}, ['clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear', 'clear']), ('sampled regression 11', {'mode': 'auto', 'steps': [[False, True, False], [False, False, True], [False, False, False], [True, False, True], [False, False, False], [False, False, False], [True, False, True], [False, False, True], [False, False, False]]}, ['clear', 'clear', 'clear', 'danger', 'clear', 'clear', 'danger', 'clear', 'clear']), ('boundary: route re-set after a train', {'mode': 'controlled', 'steps': [[False, True, True], [True, True, True], [False, False, True], [False, True, True]]}, ['clear', 'danger', 'danger', 'clear']), ('boundary: approach-released signal with empty approach', {'mode': 'approach', 'steps': [[False, True, False], [False, True, True]]}, ['danger', 'clear']), ('control 45', {'mode': 'approach', 'steps': [[False, False, False], [False, False, False], [False, True, False], [False, True, False], [True, True, False], [False, True, False], [False, True, False], [False, True, True]]}, ['danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'danger', 'clear']), ('control 48', {'mode': 'approach', 'steps': [[True, True, False], [False, True, True], [False, False, False], [False, False, True], [False, True, True], [True, True, False]]}, ['danger', 'clear', 'danger', 'danger', 'clear', 'danger']), ('control 51', {'mode': 'controlled', 'steps': [[False, False, True], [False, False, False], [False, True, False], [False, True, True], [True, False, True], [False, False, True], [True, True, False]]}, ['danger', 'danger', 'clear', 'clear', 'danger', 'danger', 'danger'])]]\nfor label, args, expected in fixtures[N-1]:\n    check(label, solve(args), expected)\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"Stipulated toy interlocking contract for a bounded teaching model; it makes no claim of conformance to any railway signalling standard and omits real safety cases. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w2-railway_interlocking_logic-signal-replacement-stick-automatic-stick-reset","generated_at":"2026-09-29T14:47:50.624942+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"Interlocking logic decides whether trains may be given authority; a wrong decision at this point either grants unsafe movements or strands traffic.","root_cause":"Automatic signals keep the stick because their route never re-sets.","sha256":"9c73d3797a1b3301f97317b5436719256651183e3ffc66197fbbaa7fa57da426","title":"Signal replacement and stick control: automatic stick reset · case 01","variant":1,"variant_policy":"Five numbered records share a model and may reuse boundary fixtures.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":40.496,"exit_code":1,"observations":[{"actual":["clear","danger","danger"],"check":"regression: automatic signal re-clears","expected":["clear","danger","clear"],"passed":false},{"actual":["danger","clear","clear","clear","clear","danger","clear","clear","danger"],"check":"sampled regression 22","expected":["danger","clear","clear","clear","clear","danger","clear","clear","danger"],"passed":true},{"actual":["clear","clear","clear","danger","danger","danger","danger","clear","clear"],"check":"sampled regression 11","expected":["clear","clear","clear","danger","clear","clear","danger","clear","clear"],"passed":false},{"actual":["clear","danger","danger"],"check":"boundary: second train behind on the same route","expected":["clear","danger","danger"],"passed":true},{"actual":["clear","danger","danger","clear"],"check":"boundary: route re-set after a train","expected":["clear","danger","danger","clear"],"passed":true},{"actual":["clear","clear","clear","clear","clear"],"check":"control 1","expected":["clear","clear","clear","clear","clear"],"passed":true},{"actual":["danger","clear","danger","danger","danger"],"check":"control 4","expected":["danger","clear","danger","danger","danger"],"passed":true},{"actual":["danger","danger","danger","danger","danger","danger","clear","clear","clear","danger"],"check":"control 7","expected":["danger","danger","danger","danger","danger","danger","clear","clear","clear","danger"],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"regression: automatic signal re-clears\", \"actual\": [\"clear\", \"danger\", \"danger\"], \"expected\": [\"clear\", \"danger\", \"clear\"], \"passed\": false}, {\"check\": \"sampled regression 22\", \"actual\": [\"danger\", \"clear\", \"clear\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\", \"danger\"], \"expected\": [\"danger\", \"clear\", \"clear\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\", \"danger\"], \"passed\": true}, {\"check\": \"sampled regression 11\", \"actual\": [\"clear\", \"clear\", \"clear\", \"danger\", \"danger\", \"danger\", \"danger\", \"clear\", \"clear\"], \"expected\": [\"clear\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\"], \"passed\": false}, {\"check\": \"boundary: second train behind on the same route\", \"actual\": [\"clear\", \"danger\", \"danger\"], \"expected\": [\"clear\", \"danger\", \"danger\"], \"passed\": true}, {\"check\": \"boundary: route re-set after a train\", \"actual\": [\"clear\", \"danger\", \"danger\", \"clear\"], \"expected\": [\"clear\", \"danger\", \"danger\", \"clear\"], \"passed\": true}, {\"check\": \"control 1\", \"actual\": [\"clear\", \"clear\", \"clear\", \"clear\", \"clear\"], \"expected\": [\"clear\", \"clear\", \"clear\", \"clear\", \"clear\"], \"passed\": true}, {\"check\": \"control 4\", \"actual\": [\"danger\", \"clear\", \"danger\", \"danger\", \"danger\"], \"expected\": [\"danger\", \"clear\", \"danger\", \"danger\", \"danger\"], \"passed\": true}, {\"check\": \"control 7\", \"actual\": [\"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"clear\", \"clear\", \"clear\", \"danger\"], \"expected\": [\"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"clear\", \"clear\", \"clear\", \"danger\"], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":39.705,"exit_code":1,"observations":[{"actual":["clear","danger","danger"],"check":"regression: automatic signal re-clears","expected":["clear","danger","clear"],"passed":false},{"actual":["danger","clear","clear","clear","clear","danger","danger","danger","danger"],"check":"sampled regression 22","expected":["danger","clear","clear","clear","clear","danger","clear","clear","danger"],"passed":false},{"actual":["clear","clear","clear","danger","danger","danger","danger","danger","danger"],"check":"sampled regression 11","expected":["clear","clear","clear","danger","clear","clear","danger","clear","clear"],"passed":false},{"actual":["clear","danger","danger"],"check":"boundary: second train behind on the same route","expected":["clear","danger","danger"],"passed":true},{"actual":["clear","danger","danger","clear"],"check":"boundary: route re-set after a train","expected":["clear","danger","danger","clear"],"passed":true},{"actual":["clear","clear","clear","clear","clear"],"check":"control 1","expected":["clear","clear","clear","clear","clear"],"passed":true},{"actual":["danger","clear","danger","danger","danger"],"check":"control 4","expected":["danger","clear","danger","danger","danger"],"passed":true},{"actual":["danger","danger","danger","danger","danger","danger","clear","clear","clear","danger"],"check":"control 7","expected":["danger","danger","danger","danger","danger","danger","clear","clear","clear","danger"],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"regression: automatic signal re-clears\", \"actual\": [\"clear\", \"danger\", \"danger\"], \"expected\": [\"clear\", \"danger\", \"clear\"], \"passed\": false}, {\"check\": \"sampled regression 22\", \"actual\": [\"danger\", \"clear\", \"clear\", \"clear\", \"clear\", \"danger\", \"danger\", \"danger\", \"danger\"], \"expected\": [\"danger\", \"clear\", \"clear\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\", \"danger\"], \"passed\": false}, {\"check\": \"sampled regression 11\", \"actual\": [\"clear\", \"clear\", \"clear\", \"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"danger\"], \"expected\": [\"clear\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\", \"danger\", \"clear\", \"clear\"], \"passed\": false}, {\"check\": \"boundary: second train behind on the same route\", \"actual\": [\"clear\", \"danger\", \"danger\"], \"expected\": [\"clear\", \"danger\", \"danger\"], \"passed\": true}, {\"check\": \"boundary: route re-set after a train\", \"actual\": [\"clear\", \"danger\", \"danger\", \"clear\"], \"expected\": [\"clear\", \"danger\", \"danger\", \"clear\"], \"passed\": true}, {\"check\": \"control 1\", \"actual\": [\"clear\", \"clear\", \"clear\", \"clear\", \"clear\"], \"expected\": [\"clear\", \"clear\", \"clear\", \"clear\", \"clear\"], \"passed\": true}, {\"check\": \"control 4\", \"actual\": [\"danger\", \"clear\", \"danger\", \"danger\", \"danger\"], \"expected\": [\"danger\", \"clear\", \"danger\", \"danger\", \"danger\"], \"passed\": true}, {\"check\": \"control 7\", \"actual\": [\"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"clear\", \"clear\", \"clear\", \"danger\"], \"expected\": [\"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"danger\", \"clear\", \"clear\", \"clear\", \"danger\"], \"passed\": true}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}