{"abstract":"Multi-label annotations are returned without vocabulary encoding.","category":"Training label alignment","checks":8,"contract":"Encode each set of labels as a binary vector in vocabulary order; unknown labels are ignored.","contract_signature":"annotations, vocabulary","evaluation_group":"training-label-alignment-multi-label-vocabulary-projection","failed_approach":"Dropping absent classes to compact the vector loses the required zero positions.","family":"w3-training-label-alignment-multi-label-set-order","id":"FA-97976","implementations":{"attempt":{"sha256":"a7d753400ea87b49b4f15610357bad4f231363e85799ecfcba0d92e59355f222","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(annotations, vocabulary):\n    return [[int(c in row) for c in vocabulary if c in row] for row in annotations]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('annotation order irrelevant',solve([['b','a']],['b','a']),[[1,1]])\ncheck('absent class zero',solve([['c']],['a','b','c']),[[0,0,1]])\ncheck('empty annotation',solve([[]],['x']),[[0]])\ncheck('empty batch',solve([],['x']),[])\ncheck('unknown ignored',solve([['x','q']],['q','x']),[[1,1]])\ncheck('vocabulary order',solve([['a']],['z','a']),[[0,1]])\ncheck('duplicate annotation',solve([['x','x']],['x']),[[1]])\ncheck('variant label with absent class',solve([['class'+str(N)]],['other','class'+str(N)]),[[0,1]])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"755c25d8ec3d7dc1ab3832dc25390b6cb25b44fe82043ebeb3109dd1ed52e9ed","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(annotations, vocabulary):\n    return [list(row) for row in annotations]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('annotation order irrelevant',solve([['b','a']],['b','a']),[[1,1]])\ncheck('absent class zero',solve([['c']],['a','b','c']),[[0,0,1]])\ncheck('empty annotation',solve([[]],['x']),[[0]])\ncheck('empty batch',solve([],['x']),[])\ncheck('unknown ignored',solve([['x','q']],['q','x']),[[1,1]])\ncheck('vocabulary order',solve([['a']],['z','a']),[[0,1]])\ncheck('duplicate annotation',solve([['x','x']],['x']),[[1]])\ncheck('variant label with absent class',solve([['class'+str(N)]],['other','class'+str(N)]),[[0,1]])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"Annotations are collections of hashable class names; vocabulary names are unique. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-training-label-alignment-multi-label-set-order","generated_at":"2026-09-30T16:48:07.978098+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"A small deterministic training-data operation makes label ownership and ordering explicit.","root_cause":"The encoder returns annotation lists verbatim instead of fixed-width vocabulary vectors.","sha256":"b28a7a2d9dbf40ff12bf91d597c765eb95fcdba6b884e80ebd14587d0ce55ec3","title":"Multi-label annotations are returned without vocabulary encoding · case 01","variant":1,"variant_policy":"The fixture code references N, supplied as 1 through 5. Fixed boundary checks may repeat. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":28.288,"exit_code":1,"observations":[{"actual":[[1,1]],"check":"annotation order irrelevant","expected":[[1,1]],"passed":true},{"actual":[[1]],"check":"absent class zero","expected":[[0,0,1]],"passed":false},{"actual":[[]],"check":"empty annotation","expected":[[0]],"passed":false},{"actual":[],"check":"empty batch","expected":[],"passed":true},{"actual":[[1,1]],"check":"unknown ignored","expected":[[1,1]],"passed":true},{"actual":[[1]],"check":"vocabulary order","expected":[[0,1]],"passed":false},{"actual":[[1]],"check":"duplicate annotation","expected":[[1]],"passed":true},{"actual":[[1]],"check":"variant label with absent class","expected":[[0,1]],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"annotation order irrelevant\", \"actual\": [[1, 1]], \"expected\": [[1, 1]], \"passed\": true}, {\"check\": \"absent class zero\", \"actual\": [[1]], \"expected\": [[0, 0, 1]], \"passed\": false}, {\"check\": \"empty annotation\", \"actual\": [[]], \"expected\": [[0]], \"passed\": false}, {\"check\": \"empty batch\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"unknown ignored\", \"actual\": [[1, 1]], \"expected\": [[1, 1]], \"passed\": true}, {\"check\": \"vocabulary order\", \"actual\": [[1]], \"expected\": [[0, 1]], \"passed\": false}, {\"check\": \"duplicate annotation\", \"actual\": [[1]], \"expected\": [[1]], \"passed\": true}, {\"check\": \"variant label with absent class\", \"actual\": [[1]], \"expected\": [[0, 1]], \"passed\": false}], \"passed\": false}\n"},"broken":{"elapsed_ms":25.226,"exit_code":1,"observations":[{"actual":[["b","a"]],"check":"annotation order irrelevant","expected":[[1,1]],"passed":false},{"actual":[["c"]],"check":"absent class zero","expected":[[0,0,1]],"passed":false},{"actual":[[]],"check":"empty annotation","expected":[[0]],"passed":false},{"actual":[],"check":"empty batch","expected":[],"passed":true},{"actual":[["x","q"]],"check":"unknown ignored","expected":[[1,1]],"passed":false},{"actual":[["a"]],"check":"vocabulary order","expected":[[0,1]],"passed":false},{"actual":[["x","x"]],"check":"duplicate annotation","expected":[[1]],"passed":false},{"actual":[["class1"]],"check":"variant label with absent class","expected":[[0,1]],"passed":false}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"annotation order irrelevant\", \"actual\": [[\"b\", \"a\"]], \"expected\": [[1, 1]], \"passed\": false}, {\"check\": \"absent class zero\", \"actual\": [[\"c\"]], \"expected\": [[0, 0, 1]], \"passed\": false}, {\"check\": \"empty annotation\", \"actual\": [[]], \"expected\": [[0]], \"passed\": false}, {\"check\": \"empty batch\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"unknown ignored\", \"actual\": [[\"x\", \"q\"]], \"expected\": [[1, 1]], \"passed\": false}, {\"check\": \"vocabulary order\", \"actual\": [[\"a\"]], \"expected\": [[0, 1]], \"passed\": false}, {\"check\": \"duplicate annotation\", \"actual\": [[\"x\", \"x\"]], \"expected\": [[1]], \"passed\": false}, {\"check\": \"variant label with absent class\", \"actual\": [[\"class1\"]], \"expected\": [[0, 1]], \"passed\": false}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}