FA-97911 / Dataset split isolation / Open access
Dataset split: per-class allocation rounds each quota down independently · case 01
per-class allocation rounds each quota down independently.
ROOT CAUSE
Independent floors systematically underfill small classes.
THE FAILURE
Independent floors systematically underfill small classes.
Unsuccessful approach: Rounding each class to nearest can over-allocate ties.
Case contract
Class counts are nonnegative integers and total is between zero and their sum; return class->train count via largest remainders, ties lexical.
Why this case matters
This toy split allocator makes membership rules explicit for reproducible evaluation fixtures.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(counts, total):
return {k:int(v*total/sum(counts.values())) for k,v in counts.items()}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('preserve target total',solve({'a':1,'b':1,'c':1},2),{'a':1,'b':1,'c':0})
check('proportional exact',solve({'a':3,'b':1},2),{'a':2,'b':0})
check('zero total',solve({'a':2,'b':1},0),{'a':0,'b':0})
check('all rows',solve({'a':2,'b':3},5),{'a':2,'b':3})
check('empty classes',solve({},0),{})
check('remainder lexical tie',solve({'b':1,'a':1},1),{'a':1,'b':0})
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| preserve target total | {'a': 0, 'b': 0, 'c': 0} | {'a': 1, 'b': 1, 'c': 0} | Failed |
| proportional exact | {'a': 1, 'b': 0} | {'a': 2, 'b': 0} | Failed |
| zero total | {'a': 0, 'b': 0} | {'a': 0, 'b': 0} | Passed |
| all rows | {'a': 2, 'b': 3} | {'a': 2, 'b': 3} | Passed |
| empty classes | {} | {} | Passed |
| remainder lexical tie | {'a': 0, 'b': 0} | {'a': 1, 'b': 0} | Failed |
SHA-256 / 909b71ea14e5113d40685529f6a32673ec9baa70d1bde1a2fb8f65f6d3bf3f28
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(counts, total):
return {k:round(v*total/sum(counts.values())) for k,v in counts.items()}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('preserve target total',solve({'a':1,'b':1,'c':1},2),{'a':1,'b':1,'c':0})
check('proportional exact',solve({'a':3,'b':1},2),{'a':2,'b':0})
check('zero total',solve({'a':2,'b':1},0),{'a':0,'b':0})
check('all rows',solve({'a':2,'b':3},5),{'a':2,'b':3})
check('empty classes',solve({},0),{})
check('remainder lexical tie',solve({'b':1,'a':1},1),{'a':1,'b':0})
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| preserve target total | {'a': 1, 'b': 1, 'c': 1} | {'a': 1, 'b': 1, 'c': 0} | Failed |
| proportional exact | {'a': 2, 'b': 0} | {'a': 2, 'b': 0} | Passed |
| zero total | {'a': 0, 'b': 0} | {'a': 0, 'b': 0} | Passed |
| all rows | {'a': 2, 'b': 3} | {'a': 2, 'b': 3} | Passed |
| empty classes | {} | {} | Passed |
| remainder lexical tie | {'a': 0, 'b': 0} | {'a': 1, 'b': 0} | Failed |
SHA-256 / 962b6b8b790c4bdd8a0bc83da618f2e27c20c0ee05991011fc9ee95eebd62705
HELD IN THE MEMBER ARCHIVE
The verified repair and its recorded checks are member-only.
This mechanism has 6 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.
Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.
Member access is invitation-based. Sign in with your invited account to inspect the repair.
Sign in to the archive ↗Verification & scope
Inputs are small in-memory records with self-contained contracts; no claim is made about production sampling quality. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-30T16:48:07.117722+00:00.
Case digest / a2ce7e68b9daf332b45993a518723abfd11170f3676523afc79704c3531fa6be