FA-44746 / Data systems / Open access
Schema union applies defaults to explicit null cells · case 01
Schema union applies defaults to explicit null cells.
ROOT CAUSE
union-by-field-id: Schema union applies defaults to explicit null cells.
VERIFIED REPAIR
Preserve the stated physical representation and operation order: Union batches using stable numeric field identities. The output schema is an ordered list [id,current_name,default]; batch schemas map old names to IDs. Project each batch row into output order, use defaults only for absent columns, and preserve explicit nulls and all rows.
Unsuccessful approach: Checking only nullness still treats explicit null as an absent field.
Case contract
Union batches using stable numeric field identities. The output schema is an ordered list [id,current_name,default]; batch schemas map old names to IDs. Project each batch row into output order, use defaults only for absent columns, and preserve explicit nulls and all rows.
Why this case matters
A bounded deterministic data engine model makes representation and changelog faults reproducible.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(d):
try:
output,batches=d
result=[]
for schema,rows in batches:
columns={field:index for index,(field,name) in enumerate(schema)}
for row in rows:
record=[]
for field,name,default in output:
value=(row[columns[field]] if field in columns else None) or default
record.append(value)
result.append(record)
return result
except (IndexError, KeyError, ValueError, StopIteration) as exc:
return {"representation_error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
if N == 1:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[1, 2]]]]]), [[2, 1]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[1, 2]]], [[[2, 'b'], [1, 'a']], [[3, 4]]]]]), [[1, 2], [4, 3]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 1]], [[[[1, 'a']], [[2]]]]]), [[2, 1]])
check('explicit null', solve([[[1, 'a', 1]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[1], [1]]]]]), [[1], [1]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 1]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 2:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[2, 3]]]]]), [[3, 2]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[2, 3]]], [[[2, 'b'], [1, 'a']], [[4, 5]]]]]), [[2, 3], [5, 4]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 2]], [[[[1, 'a']], [[3]]]]]), [[3, 2]])
check('explicit null', solve([[[1, 'a', 2]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[2], [2]]]]]), [[2], [2]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 2]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 3:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[3, 4]]]]]), [[4, 3]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[3, 4]]], [[[2, 'b'], [1, 'a']], [[5, 6]]]]]), [[3, 4], [6, 5]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 3]], [[[[1, 'a']], [[4]]]]]), [[4, 3]])
check('explicit null', solve([[[1, 'a', 3]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[3], [3]]]]]), [[3], [3]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 3]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 4:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[4, 5]]]]]), [[5, 4]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[4, 5]]], [[[2, 'b'], [1, 'a']], [[6, 7]]]]]), [[4, 5], [7, 6]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 4]], [[[[1, 'a']], [[5]]]]]), [[5, 4]])
check('explicit null', solve([[[1, 'a', 4]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[4], [4]]]]]), [[4], [4]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 4]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 5:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[5, 6]]]]]), [[6, 5]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[5, 6]]], [[[2, 'b'], [1, 'a']], [[7, 8]]]]]), [[5, 6], [8, 7]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 5]], [[[[1, 'a']], [[6]]]]]), [[6, 5]])
check('explicit null', solve([[[1, 'a', 5]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[5], [5]]]]]), [[5], [5]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 5]], [[[[1, 'a']], [[0]]]]]), [[0]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| renamed field | [[2, 1]] | [[2, 1]] | Passed |
| reordered batch | [[1, 2], [4, 3]] | [[1, 2], [4, 3]] | Passed |
| absent default | [[2, 1]] | [[2, 1]] | Passed |
| explicit null | [[1]] | [[None]] | Failed |
| duplicate rows | [[1], [1]] | [[1], [1]] | Passed |
| no batches | [] | [] | Passed |
| zero preserved | [[1]] | [[0]] | Failed |
SHA-256 / d68e5884c86f633631fee2aa4fe77d81ae524d0a976b02ecabfe403441b074d8
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(d):
try:
output,batches=d
result=[]
for schema,rows in batches:
columns={field:index for index,(field,name) in enumerate(schema)}
for row in rows:
record=[]
for field,name,default in output:
value=default if field not in columns or row[columns[field]] is None else row[columns[field]]
record.append(value)
result.append(record)
return result
except (IndexError, KeyError, ValueError, StopIteration) as exc:
return {"representation_error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
if N == 1:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[1, 2]]]]]), [[2, 1]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[1, 2]]], [[[2, 'b'], [1, 'a']], [[3, 4]]]]]), [[1, 2], [4, 3]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 1]], [[[[1, 'a']], [[2]]]]]), [[2, 1]])
check('explicit null', solve([[[1, 'a', 1]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[1], [1]]]]]), [[1], [1]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 1]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 2:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[2, 3]]]]]), [[3, 2]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[2, 3]]], [[[2, 'b'], [1, 'a']], [[4, 5]]]]]), [[2, 3], [5, 4]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 2]], [[[[1, 'a']], [[3]]]]]), [[3, 2]])
check('explicit null', solve([[[1, 'a', 2]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[2], [2]]]]]), [[2], [2]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 2]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 3:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[3, 4]]]]]), [[4, 3]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[3, 4]]], [[[2, 'b'], [1, 'a']], [[5, 6]]]]]), [[3, 4], [6, 5]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 3]], [[[[1, 'a']], [[4]]]]]), [[4, 3]])
check('explicit null', solve([[[1, 'a', 3]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[3], [3]]]]]), [[3], [3]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 3]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 4:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[4, 5]]]]]), [[5, 4]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[4, 5]]], [[[2, 'b'], [1, 'a']], [[6, 7]]]]]), [[4, 5], [7, 6]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 4]], [[[[1, 'a']], [[5]]]]]), [[5, 4]])
check('explicit null', solve([[[1, 'a', 4]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[4], [4]]]]]), [[4], [4]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 4]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 5:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[5, 6]]]]]), [[6, 5]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[5, 6]]], [[[2, 'b'], [1, 'a']], [[7, 8]]]]]), [[5, 6], [8, 7]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 5]], [[[[1, 'a']], [[6]]]]]), [[6, 5]])
check('explicit null', solve([[[1, 'a', 5]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[5], [5]]]]]), [[5], [5]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 5]], [[[[1, 'a']], [[0]]]]]), [[0]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| renamed field | [[2, 1]] | [[2, 1]] | Passed |
| reordered batch | [[1, 2], [4, 3]] | [[1, 2], [4, 3]] | Passed |
| absent default | [[2, 1]] | [[2, 1]] | Passed |
| explicit null | [[1]] | [[None]] | Failed |
| duplicate rows | [[1], [1]] | [[1], [1]] | Passed |
| no batches | [] | [] | Passed |
| zero preserved | [[0]] | [[0]] | Passed |
SHA-256 / f5a000cbd3270b5965b250eb29363f8abf84b46f1b8218f468b46ff21f465d73
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(d):
try:
output,batches=d
result=[]
for schema,rows in batches:
columns={field:index for index,(field,name) in enumerate(schema)}
for row in rows:
record=[]
for field,name,default in output:
value=row[columns[field]] if field in columns else default
record.append(value)
result.append(record)
return result
except (IndexError, KeyError, ValueError, StopIteration) as exc:
return {"representation_error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
if N == 1:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[1, 2]]]]]), [[2, 1]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[1, 2]]], [[[2, 'b'], [1, 'a']], [[3, 4]]]]]), [[1, 2], [4, 3]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 1]], [[[[1, 'a']], [[2]]]]]), [[2, 1]])
check('explicit null', solve([[[1, 'a', 1]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[1], [1]]]]]), [[1], [1]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 1]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 2:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[2, 3]]]]]), [[3, 2]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[2, 3]]], [[[2, 'b'], [1, 'a']], [[4, 5]]]]]), [[2, 3], [5, 4]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 2]], [[[[1, 'a']], [[3]]]]]), [[3, 2]])
check('explicit null', solve([[[1, 'a', 2]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[2], [2]]]]]), [[2], [2]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 2]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 3:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[3, 4]]]]]), [[4, 3]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[3, 4]]], [[[2, 'b'], [1, 'a']], [[5, 6]]]]]), [[3, 4], [6, 5]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 3]], [[[[1, 'a']], [[4]]]]]), [[4, 3]])
check('explicit null', solve([[[1, 'a', 3]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[3], [3]]]]]), [[3], [3]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 3]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 4:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[4, 5]]]]]), [[5, 4]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[4, 5]]], [[[2, 'b'], [1, 'a']], [[6, 7]]]]]), [[4, 5], [7, 6]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 4]], [[[[1, 'a']], [[5]]]]]), [[5, 4]])
check('explicit null', solve([[[1, 'a', 4]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[4], [4]]]]]), [[4], [4]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 4]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 5:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[5, 6]]]]]), [[6, 5]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[5, 6]]], [[[2, 'b'], [1, 'a']], [[7, 8]]]]]), [[5, 6], [8, 7]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 5]], [[[[1, 'a']], [[6]]]]]), [[6, 5]])
check('explicit null', solve([[[1, 'a', 5]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[5], [5]]]]]), [[5], [5]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 5]], [[[[1, 'a']], [[0]]]]]), [[0]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| renamed field | [[2, 1]] | [[2, 1]] | Passed |
| reordered batch | [[1, 2], [4, 3]] | [[1, 2], [4, 3]] | Passed |
| absent default | [[2, 1]] | [[2, 1]] | Passed |
| explicit null | [[None]] | [[None]] | Passed |
| duplicate rows | [[1], [1]] | [[1], [1]] | Passed |
| no batches | [] | [] | Passed |
| zero preserved | [[0]] | [[0]] | Passed |
SHA-256 / 1d8e9d04b1bd1fea76fe15a46c6c0751a4827174f5723c72cc97117615813379
Verification & scope
Offline stipulated semantics over valid small inputs; no performance, concurrency, or production-engine conformance claim. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:44:15.218890+00:00.
Case digest / b9e5c577fe3ab59e823df91ab6a4359ca7381a5e4f10d9ee5d928790e84dc02c