FAILURE MAP
← Case archive

FA-97921 / Dataset split isolation / Open access

Dataset split: impossible fold requests silently produce an incomplete fold map · case 01

impossible fold requests silently produce an incomplete fold map.

Verified by executionVariant 1 · 6 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The requested fold count is not checked against the number of distinct groups, so some requested folds receive no group.

THE FAILURE

The requested fold count is not checked against the number of distinct groups, so some requested folds receive no group.

Unsuccessful approach: Clamping to the available group count still hides that the requested number of folds cannot be filled.

Case contract

For a positive integer k, return sorted groups assigned round-robin to folds; return empty mapping when k exceeds the number of distinct groups.

Why this case matters

This toy split allocator makes membership rules explicit for reproducible evaluation fixtures.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(groups, k):
    return {g:j%max(1,k) for j,g in enumerate(sorted(set(groups)))}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('balanced map',solve(['c','a','b'],2),{'a':0,'b':1,'c':0})
check('duplicates count once',solve(['a','a','b'],2),{'a':0,'b':1})
check('empty rejected',solve([],1),{})
check('single group',solve(['x'],1),{'x':0})
check('invalid zero',solve(['x'],0),{})
check('too many folds',solve(['x'],2),{})
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
balanced map{'a': 0, 'b': 1, 'c': 0}{'a': 0, 'b': 1, 'c': 0}Passed
duplicates count once{'a': 0, 'b': 1}{'a': 0, 'b': 1}Passed
empty rejected{}{}Passed
single group{'x': 0}{'x': 0}Passed
invalid zero{'x': 0}{}Failed
too many folds{'x': 0}{}Failed

SHA-256 / b354bb870e608b98f8cfcea05ed1816502c293e91b68b6fe4f2b767e60426f1e

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(groups, k):
    if k<1: return {}
    uniq=sorted(set(groups)); k=min(k,len(uniq))
    return {g:j%k for j,g in enumerate(uniq)}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('balanced map',solve(['c','a','b'],2),{'a':0,'b':1,'c':0})
check('duplicates count once',solve(['a','a','b'],2),{'a':0,'b':1})
check('empty rejected',solve([],1),{})
check('single group',solve(['x'],1),{'x':0})
check('invalid zero',solve(['x'],0),{})
check('too many folds',solve(['x'],2),{})
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
balanced map{'a': 0, 'b': 1, 'c': 0}{'a': 0, 'b': 1, 'c': 0}Passed
duplicates count once{'a': 0, 'b': 1}{'a': 0, 'b': 1}Passed
empty rejected{}{}Passed
single group{'x': 0}{'x': 0}Passed
invalid zero{}{}Passed
too many folds{'x': 0}{}Failed

SHA-256 / a24712bd295ebb9237340f821af2332f7119cafb8a998a5b3dfb3ec2d9066f4f

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 6 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Sign in to the archive ↗

Verification & scope

Inputs are small in-memory records with self-contained contracts; no claim is made about production sampling quality. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-30T16:48:07.548420+00:00.

Case digest / 1809193a418bd46c5f71fc77f3b6b5be461b111a07b5b4948ef83de818b19721