FA-44751 / Data systems / Open access
Schema union ignores non-null defaults for missing columns · case 01
Schema union ignores non-null defaults for missing columns.
ROOT CAUSE
union-by-field-id: Schema union ignores non-null defaults for missing columns.
VERIFIED REPAIR
Preserve the stated physical representation and operation order: Union batches using stable numeric field identities. The output schema is an ordered list [id,current_name,default]; batch schemas map old names to IDs. Project each batch row into output order, use defaults only for absent columns, and preserve explicit nulls and all rows.
Unsuccessful approach: Zero is not the declared field default.
Case contract
Union batches using stable numeric field identities. The output schema is an ordered list [id,current_name,default]; batch schemas map old names to IDs. Project each batch row into output order, use defaults only for absent columns, and preserve explicit nulls and all rows.
Why this case matters
A bounded deterministic data engine model makes representation and changelog faults reproducible.
1 / The failure
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(d):
try:
output,batches=d
result=[]
for schema,rows in batches:
columns={field:index for index,(field,name) in enumerate(schema)}
for row in rows:
record=[]
for field,name,default in output:
value=row[columns[field]] if field in columns else None
record.append(value)
result.append(record)
return result
except (IndexError, KeyError, ValueError, StopIteration) as exc:
return {"representation_error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
if N == 1:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[1, 2]]]]]), [[2, 1]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[1, 2]]], [[[2, 'b'], [1, 'a']], [[3, 4]]]]]), [[1, 2], [4, 3]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 1]], [[[[1, 'a']], [[2]]]]]), [[2, 1]])
check('explicit null', solve([[[1, 'a', 1]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[1], [1]]]]]), [[1], [1]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 1]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 2:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[2, 3]]]]]), [[3, 2]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[2, 3]]], [[[2, 'b'], [1, 'a']], [[4, 5]]]]]), [[2, 3], [5, 4]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 2]], [[[[1, 'a']], [[3]]]]]), [[3, 2]])
check('explicit null', solve([[[1, 'a', 2]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[2], [2]]]]]), [[2], [2]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 2]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 3:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[3, 4]]]]]), [[4, 3]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[3, 4]]], [[[2, 'b'], [1, 'a']], [[5, 6]]]]]), [[3, 4], [6, 5]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 3]], [[[[1, 'a']], [[4]]]]]), [[4, 3]])
check('explicit null', solve([[[1, 'a', 3]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[3], [3]]]]]), [[3], [3]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 3]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 4:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[4, 5]]]]]), [[5, 4]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[4, 5]]], [[[2, 'b'], [1, 'a']], [[6, 7]]]]]), [[4, 5], [7, 6]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 4]], [[[[1, 'a']], [[5]]]]]), [[5, 4]])
check('explicit null', solve([[[1, 'a', 4]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[4], [4]]]]]), [[4], [4]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 4]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 5:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[5, 6]]]]]), [[6, 5]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[5, 6]]], [[[2, 'b'], [1, 'a']], [[7, 8]]]]]), [[5, 6], [8, 7]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 5]], [[[[1, 'a']], [[6]]]]]), [[6, 5]])
check('explicit null', solve([[[1, 'a', 5]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[5], [5]]]]]), [[5], [5]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 5]], [[[[1, 'a']], [[0]]]]]), [[0]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| renamed field | [[2, 1]] | [[2, 1]] | Passed |
| reordered batch | [[1, 2], [4, 3]] | [[1, 2], [4, 3]] | Passed |
| absent default | [[2, None]] | [[2, 1]] | Failed |
| explicit null | [[None]] | [[None]] | Passed |
| duplicate rows | [[1], [1]] | [[1], [1]] | Passed |
| no batches | [] | [] | Passed |
| zero preserved | [[0]] | [[0]] | Passed |
SHA-256 / 46d05aa55b39d61dc82bc9cc8510d00b2ae4aa0e556d5a62a1c8a79ca8837c3c
2 / The unsuccessful fix
Exit 1"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(d):
try:
output,batches=d
result=[]
for schema,rows in batches:
columns={field:index for index,(field,name) in enumerate(schema)}
for row in rows:
record=[]
for field,name,default in output:
value=row[columns[field]] if field in columns else 0
record.append(value)
result.append(record)
return result
except (IndexError, KeyError, ValueError, StopIteration) as exc:
return {"representation_error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
if N == 1:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[1, 2]]]]]), [[2, 1]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[1, 2]]], [[[2, 'b'], [1, 'a']], [[3, 4]]]]]), [[1, 2], [4, 3]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 1]], [[[[1, 'a']], [[2]]]]]), [[2, 1]])
check('explicit null', solve([[[1, 'a', 1]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[1], [1]]]]]), [[1], [1]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 1]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 2:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[2, 3]]]]]), [[3, 2]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[2, 3]]], [[[2, 'b'], [1, 'a']], [[4, 5]]]]]), [[2, 3], [5, 4]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 2]], [[[[1, 'a']], [[3]]]]]), [[3, 2]])
check('explicit null', solve([[[1, 'a', 2]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[2], [2]]]]]), [[2], [2]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 2]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 3:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[3, 4]]]]]), [[4, 3]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[3, 4]]], [[[2, 'b'], [1, 'a']], [[5, 6]]]]]), [[3, 4], [6, 5]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 3]], [[[[1, 'a']], [[4]]]]]), [[4, 3]])
check('explicit null', solve([[[1, 'a', 3]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[3], [3]]]]]), [[3], [3]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 3]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 4:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[4, 5]]]]]), [[5, 4]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[4, 5]]], [[[2, 'b'], [1, 'a']], [[6, 7]]]]]), [[4, 5], [7, 6]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 4]], [[[[1, 'a']], [[5]]]]]), [[5, 4]])
check('explicit null', solve([[[1, 'a', 4]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[4], [4]]]]]), [[4], [4]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 4]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 5:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[5, 6]]]]]), [[6, 5]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[5, 6]]], [[[2, 'b'], [1, 'a']], [[7, 8]]]]]), [[5, 6], [8, 7]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 5]], [[[[1, 'a']], [[6]]]]]), [[6, 5]])
check('explicit null', solve([[[1, 'a', 5]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[5], [5]]]]]), [[5], [5]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 5]], [[[[1, 'a']], [[0]]]]]), [[0]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| renamed field | [[2, 1]] | [[2, 1]] | Passed |
| reordered batch | [[1, 2], [4, 3]] | [[1, 2], [4, 3]] | Passed |
| absent default | [[2, 0]] | [[2, 1]] | Failed |
| explicit null | [[None]] | [[None]] | Passed |
| duplicate rows | [[1], [1]] | [[1], [1]] | Passed |
| no batches | [] | [] | Passed |
| zero preserved | [[0]] | [[0]] | Passed |
SHA-256 / 7dd24a676f7aaa579e91cc8fe4e204e23412ad71fb5b747e9dd943cda395b680
3 / The verified repair
Exit 0"""Failure Map reference implementation. Python standard library only."""
import json
N = 1
observations = []
def solve(d):
try:
output,batches=d
result=[]
for schema,rows in batches:
columns={field:index for index,(field,name) in enumerate(schema)}
for row in rows:
record=[]
for field,name,default in output:
value=row[columns[field]] if field in columns else default
record.append(value)
result.append(record)
return result
except (IndexError, KeyError, ValueError, StopIteration) as exc:
return {"representation_error": type(exc).__name__}
def check(label, actual, expected):
observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
if N == 1:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[1, 2]]]]]), [[2, 1]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[1, 2]]], [[[2, 'b'], [1, 'a']], [[3, 4]]]]]), [[1, 2], [4, 3]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 1]], [[[[1, 'a']], [[2]]]]]), [[2, 1]])
check('explicit null', solve([[[1, 'a', 1]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[1], [1]]]]]), [[1], [1]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 1]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 2:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[2, 3]]]]]), [[3, 2]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[2, 3]]], [[[2, 'b'], [1, 'a']], [[4, 5]]]]]), [[2, 3], [5, 4]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 2]], [[[[1, 'a']], [[3]]]]]), [[3, 2]])
check('explicit null', solve([[[1, 'a', 2]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[2], [2]]]]]), [[2], [2]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 2]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 3:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[3, 4]]]]]), [[4, 3]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[3, 4]]], [[[2, 'b'], [1, 'a']], [[5, 6]]]]]), [[3, 4], [6, 5]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 3]], [[[[1, 'a']], [[4]]]]]), [[4, 3]])
check('explicit null', solve([[[1, 'a', 3]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[3], [3]]]]]), [[3], [3]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 3]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 4:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[4, 5]]]]]), [[5, 4]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[4, 5]]], [[[2, 'b'], [1, 'a']], [[6, 7]]]]]), [[4, 5], [7, 6]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 4]], [[[[1, 'a']], [[5]]]]]), [[5, 4]])
check('explicit null', solve([[[1, 'a', 4]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[4], [4]]]]]), [[4], [4]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 4]], [[[[1, 'a']], [[0]]]]]), [[0]])
elif N == 5:
check('renamed field', solve([[[2, 'new', 99], [1, 'a', 88]], [[[[1, 'a'], [2, 'old']], [[5, 6]]]]]), [[6, 5]])
check('reordered batch', solve([[[1, 'a', 0], [2, 'b', 0]], [[[[1, 'a'], [2, 'b']], [[5, 6]]], [[[2, 'b'], [1, 'a']], [[7, 8]]]]]), [[5, 6], [8, 7]])
check('absent default', solve([[[1, 'a', 7], [2, 'b', 5]], [[[[1, 'a']], [[6]]]]]), [[6, 5]])
check('explicit null', solve([[[1, 'a', 5]], [[[[1, 'a']], [[None]]]]]), [[None]])
check('duplicate rows', solve([[[1, 'a', 0]], [[[[1, 'a']], [[5], [5]]]]]), [[5], [5]])
check('no batches', solve([[[1, 'a', 0]], []]), [])
check('zero preserved', solve([[[1, 'a', 5]], [[[[1, 'a']], [[0]]]]]), [[0]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
| Boundary fixture | Actual | Expected | Outcome |
|---|---|---|---|
| renamed field | [[2, 1]] | [[2, 1]] | Passed |
| reordered batch | [[1, 2], [4, 3]] | [[1, 2], [4, 3]] | Passed |
| absent default | [[2, 1]] | [[2, 1]] | Passed |
| explicit null | [[None]] | [[None]] | Passed |
| duplicate rows | [[1], [1]] | [[1], [1]] | Passed |
| no batches | [] | [] | Passed |
| zero preserved | [[0]] | [[0]] | Passed |
SHA-256 / 1d8e9d04b1bd1fea76fe15a46c6c0751a4827174f5723c72cc97117615813379
Verification & scope
Offline stipulated semantics over valid small inputs; no performance, concurrency, or production-engine conformance claim. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.
Observations recorded using Python 3.12.14 at 2026-09-29T14:44:15.260990+00:00.
Case digest / 8dc3b210499a45b021e7bec036c082304c3222d0468a6d8bd363bf92553da406