FAILURE MAP
← Case archive

FA-2961 / Encodings / Open access

Invalid UTF-8 bytes silently disappear · case 01

Invalid UTF-8 bytes silently disappear.

Verified by executionVariant 1 · 4 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The implementation applies an operation whose text or grammar semantics violate this contract: Decode UTF-8 using one standard replacement sequence per decoding error.

VERIFIED REPAIR

Implement the complete stated contract, including the boundary fixtures: Decode UTF-8 using one standard replacement sequence per decoding error.

Unsuccessful approach: The attempted repair handles the primary example but still violates a separate boundary of the same contract.

Case contract

Decode UTF-8 using one standard replacement sequence per decoding error.

Why this case matters

A local executable model for consumers of structured text; oracle values are authored literals, not outputs copied from the repaired implementation.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return bytes.fromhex(x).decode('utf-8','ignore')
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve('61ff62'), 'a�b')
check('boundary 2', solve('e282'), '�')
check('boundary 3', solve('f09f9880'), '😀')
check('boundary 4', solve(''), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
boundary 1aba�bFailed
boundary 2�Failed
boundary 3😀😀Passed
boundary 4Passed

SHA-256 / 356364cfa1c0ea9409bc8d946309bd3f078048ddb96a3b9a6e75eec6139cf0a6

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return bytes.fromhex(x).decode('latin1')
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve('61ff62'), 'a�b')
check('boundary 2', solve('e282'), '�')
check('boundary 3', solve('f09f9880'), '😀')
check('boundary 4', solve(''), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
boundary 1aÿba�bFailed
boundary 2�Failed
boundary 3😀😀Failed
boundary 4Passed

SHA-256 / 4c5083ce59e3cf8046cb159a55d8c683717dbda53b3826b562ff04fbb51e1cf9

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return bytes.fromhex(x).decode('utf-8','replace')
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve('61ff62'), 'a�b')
check('boundary 2', solve('e282'), '�')
check('boundary 3', solve('f09f9880'), '😀')
check('boundary 4', solve(''), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
boundary 1a�ba�bPassed
boundary 2��Passed
boundary 3😀😀Passed
boundary 4Passed

SHA-256 / 13cba4421fef998874aeac31168f551c8834f195f66537dd7cd9c9789d550adb

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:37:19.511211+00:00.

Case digest / e47c64f7487051a014eda6247ee6dd8258c42c71443557422c8fec04d08fa8cd