{"abstract":"entity holdout selects a record instead of the whole entity.","category":"Dataset split isolation","checks":6,"contract":"Rows [id,entity]; hold out the lexically greatest distinct entity and return its IDs sorted.","contract_signature":"rows","evaluation_group":"ds-entity-holdout","failed_approach":"Selecting only one record from the chosen entity leaves its other records on the training side.","family":"w3-dataset-split-isolation-entity-holdout","id":"FA-97876","implementations":{"attempt":{"sha256":"33b8502cec4c78a69ac197f6f234ce804b6a0a6e4768a533cee91b580b1fe04b","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    held=max((e for _,e in rows),default='')\n    return sorted(i for i,e in rows if e==held)[:1]\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('entity rows scattered',solve([['a','x'],['b','y'],['c','x']]),['b'])\ncheck('empty',solve([]),[])\ncheck('single entity',solve([['a','x'],['b','x']]),['a','b'])\ncheck('lexical entity',solve([['z','z'],['a','a']]),['z'])\ncheck('one row per entity',solve([['a','a'],['b','b']]),['b'])\ncheck('duplicate IDs by entity',solve([['a','z'],['a','x']]),['a'])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"},"broken":{"sha256":"7055319ddb3cf55c1fa567159ff9fe6aca45384f60bee859a8e682a0a83104ed","source":"\"\"\"Failure Map reference implementation. Python standard library only.\"\"\"\nimport json\n\nN = 1\nobservations = []\ndef solve(rows):\n    return sorted(i for i,e in rows[-1:])\ndef check(label, actual, expected):\n    observations.append({\"check\": label, \"actual\": actual, \"expected\": expected, \"passed\": actual == expected})\ncheck('entity rows scattered',solve([['a','x'],['b','y'],['c','x']]),['b'])\ncheck('empty',solve([]),[])\ncheck('single entity',solve([['a','x'],['b','x']]),['a','b'])\ncheck('lexical entity',solve([['z','z'],['a','a']]),['z'])\ncheck('one row per entity',solve([['a','a'],['b','b']]),['b'])\ncheck('duplicate IDs by entity',solve([['a','z'],['a','x']]),['a'])\nprint(json.dumps({\"observations\": observations, \"passed\": all(x[\"passed\"] for x in observations)}, ensure_ascii=False))\nraise SystemExit(0 if all(x[\"passed\"] for x in observations) else 1)\n"}},"limitations":"Inputs are small in-memory records with self-contained contracts; no claim is made about production sampling quality. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.","method":"Deterministic executable model with adversarial boundary fixtures.","provenance":{"created_by":"Failure Map","dependencies":"Python standard library","family":"w3-dataset-split-isolation-entity-holdout","generated_at":"2026-09-30T16:48:06.688110+00:00","license":"CC0-1.0","python":"3.12.14","seed":1,"split":"open-access"},"relevance":"This toy split allocator makes membership rules explicit for reproducible evaluation fixtures.","root_cause":"The implementation takes one positional row as the holdout, ignoring entity membership and deterministic entity ordering.","sha256":"1a6585e7e2d11a1689304a33d8d7b3bcff085f705bc1f236953b95a4664567c3","title":"Dataset split: entity holdout selects a record instead of the whole entity · case 01","variant":1,"variant_policy":"The same explicitly authored fixtures repeat across all five numbered records; runtime N is unused. These records share one mechanism and must remain together in evaluation splits.","verified":true,"visibility":"public","verification":{"attempt":{"elapsed_ms":24.011,"exit_code":1,"observations":[{"actual":["b"],"check":"entity rows scattered","expected":["b"],"passed":true},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":["a"],"check":"single entity","expected":["a","b"],"passed":false},{"actual":["z"],"check":"lexical entity","expected":["z"],"passed":true},{"actual":["b"],"check":"one row per entity","expected":["b"],"passed":true},{"actual":["a"],"check":"duplicate IDs by entity","expected":["a"],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"entity rows scattered\", \"actual\": [\"b\"], \"expected\": [\"b\"], \"passed\": true}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"single entity\", \"actual\": [\"a\"], \"expected\": [\"a\", \"b\"], \"passed\": false}, {\"check\": \"lexical entity\", \"actual\": [\"z\"], \"expected\": [\"z\"], \"passed\": true}, {\"check\": \"one row per entity\", \"actual\": [\"b\"], \"expected\": [\"b\"], \"passed\": true}, {\"check\": \"duplicate IDs by entity\", \"actual\": [\"a\"], \"expected\": [\"a\"], \"passed\": true}], \"passed\": false}\n"},"broken":{"elapsed_ms":33.273,"exit_code":1,"observations":[{"actual":["c"],"check":"entity rows scattered","expected":["b"],"passed":false},{"actual":[],"check":"empty","expected":[],"passed":true},{"actual":["b"],"check":"single entity","expected":["a","b"],"passed":false},{"actual":["a"],"check":"lexical entity","expected":["z"],"passed":false},{"actual":["b"],"check":"one row per entity","expected":["b"],"passed":true},{"actual":["a"],"check":"duplicate IDs by entity","expected":["a"],"passed":true}],"passed":false,"stderr":"","stdout":"{\"observations\": [{\"check\": \"entity rows scattered\", \"actual\": [\"c\"], \"expected\": [\"b\"], \"passed\": false}, {\"check\": \"empty\", \"actual\": [], \"expected\": [], \"passed\": true}, {\"check\": \"single entity\", \"actual\": [\"b\"], \"expected\": [\"a\", \"b\"], \"passed\": false}, {\"check\": \"lexical entity\", \"actual\": [\"a\"], \"expected\": [\"z\"], \"passed\": false}, {\"check\": \"one row per entity\", \"actual\": [\"b\"], \"expected\": [\"b\"], \"passed\": true}, {\"check\": \"duplicate IDs by entity\", \"actual\": [\"a\"], \"expected\": [\"a\"], \"passed\": true}], \"passed\": false}\n"}},"member_only":{"stages":["fixed"],"fields":["implementations.fixed","verification.fixed","harness","repair"],"note":"The verified repair, its recorded checks, the repair description, and the scoring harness are available to members."}}