FA-186 / Encodings / Open access

A network chunk boundary corrupts a Unicode character · case 01

A valid character split across chunks becomes replacement characters or vanishes.

Verified by executionVariant 1 · 7 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Each transport chunk is decoded as if it were an independently complete UTF-8 string.

VERIFIED REPAIR

Carry decoder state across chunks and finalize it to detect incomplete trailing sequences.

Unsuccessful approach: Ignoring decode errors hides corrupt data and drops fragments of valid split characters.

Case contract

Decode a list of byte chunks as one strict UTF-8 stream. Return the decoded string, or None for malformed or incomplete input. Empty chunks have no semantic effect.

Why this case matters

HTTP, socket and stream-reader chunk boundaries are independent of character boundaries. Finalization matters even when every earlier chunk was accepted.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import codecs
N = 1
observations = []
def solve(chunks):
    return ''.join(chunk.decode('utf-8', errors='replace') for chunk in chunks)
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
prefix = 'x' * N
check('two-byte sequence split', solve([prefix.encode() + b'\xc3', b'\xa9']), prefix + 'é')
check('four-byte sequence split into single bytes', solve([b'\xf0', b'\x9f', b'\x99', b'\x82'] * N), '🙂' * N)
check('ASCII stream', solve([b'a' * N, b'b']), 'a' * N + 'b')
check('empty chunks within a sequence', solve([b'\xc3', b'', b'\xa9', b'']), 'é')
check('incomplete final sequence rejected', solve([prefix.encode(), b'\xe7\x95']), None)
check('invalid continuation rejected', solve([b'\xc3', b'x']), None)
check('no chunks', solve([]), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
two-byte sequence split	`x��`	`xé`	Failed
four-byte sequence split into single bytes	`��`	`🙂`	Failed
ASCII stream	`ab`	`ab`	Passed
empty chunks within a sequence	`��`	`é`	Failed
incomplete final sequence rejected	`x�`	`None`	Failed
invalid continuation rejected	`�x`	`None`	Failed
no chunks			Passed

SHA-256 / cf57df10c71594ac78d8b49f9767aaa3f7e672bb42ed269d5a9af908676efad3

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import codecs
N = 1
observations = []
def solve(chunks):
    return ''.join(chunk.decode('utf-8', errors='ignore') for chunk in chunks)
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
prefix = 'x' * N
check('two-byte sequence split', solve([prefix.encode() + b'\xc3', b'\xa9']), prefix + 'é')
check('four-byte sequence split into single bytes', solve([b'\xf0', b'\x9f', b'\x99', b'\x82'] * N), '🙂' * N)
check('ASCII stream', solve([b'a' * N, b'b']), 'a' * N + 'b')
check('empty chunks within a sequence', solve([b'\xc3', b'', b'\xa9', b'']), 'é')
check('incomplete final sequence rejected', solve([prefix.encode(), b'\xe7\x95']), None)
check('invalid continuation rejected', solve([b'\xc3', b'x']), None)
check('no chunks', solve([]), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
two-byte sequence split	`x`	`xé`	Failed
four-byte sequence split into single bytes		`🙂`	Failed
ASCII stream	`ab`	`ab`	Passed
empty chunks within a sequence		`é`	Failed
incomplete final sequence rejected	`x`	`None`	Failed
invalid continuation rejected	`x`	`None`	Failed
no chunks			Passed

SHA-256 / 497f09ffc5ff84fb9f457c6d95b7e2af543fa132cd26981df54212d12e2a3b3b

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json
import codecs
N = 1
observations = []
def solve(chunks):
    decoder = codecs.getincrementaldecoder('utf-8')(errors='strict')
    try:
        parts = [decoder.decode(chunk, final=False) for chunk in chunks]
        parts.append(decoder.decode(b'', final=True))
        return ''.join(parts)
    except UnicodeDecodeError:
        return None
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
prefix = 'x' * N
check('two-byte sequence split', solve([prefix.encode() + b'\xc3', b'\xa9']), prefix + 'é')
check('four-byte sequence split into single bytes', solve([b'\xf0', b'\x9f', b'\x99', b'\x82'] * N), '🙂' * N)
check('ASCII stream', solve([b'a' * N, b'b']), 'a' * N + 'b')
check('empty chunks within a sequence', solve([b'\xc3', b'', b'\xa9', b'']), 'é')
check('incomplete final sequence rejected', solve([prefix.encode(), b'\xe7\x95']), None)
check('invalid continuation rejected', solve([b'\xc3', b'x']), None)
check('no chunks', solve([]), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
two-byte sequence split	`xé`	`xé`	Passed
four-byte sequence split into single bytes	`🙂`	`🙂`	Passed
ASCII stream	`ab`	`ab`	Passed
empty chunks within a sequence	`é`	`é`	Passed
incomplete final sequence rejected	`None`	`None`	Passed
invalid continuation rejected	`None`	`None`	Passed
no chunks			Passed

SHA-256 / 8648940600cd62dc7d0251b9f70da7dac83989107b1605c1c4b602a793a591a7

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:36:51.019074+00:00.

Case digest / 2a5f19a21ffc5a25574b1f469e8267fc5cf13a5b21ff00478ab94537fb4c173e