FA-3926 / Encodings / Open access

UTF-16 code units are interpreted as Unicode scalars · case 01

UTF-16 code units are interpreted as Unicode scalars.

Verified by executionVariant 1 · 4 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The implementation applies an operation whose text or grammar semantics violate this contract: Decode little-endian UTF-16 code units to text with replacement for unpaired surrogates.

VERIFIED REPAIR

Implement the complete stated contract, including the boundary fixtures: Decode little-endian UTF-16 code units to text with replacement for unpaired surrogates.

Unsuccessful approach: The attempted repair handles the primary example but still violates a separate boundary of the same contract.

Case contract

Decode little-endian UTF-16 code units to text with replacement for unpaired surrogates.

Why this case matters

A local executable model for consumers of structured text; oracle values are authored literals, not outputs copied from the repaired implementation.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return ''.join(chr(v) for v in x).encode('utf-8','replace').decode()
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve([55357, 56832]), '😀')
check('boundary 2', solve([55296]), '�')
check('boundary 3', solve([65]), 'A')
check('boundary 4', solve([]), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
boundary 1	`??`	`😀`	Failed
boundary 2	`?`	`�`	Failed
boundary 3	`A`	`A`	Passed
boundary 4			Passed

SHA-256 / df47615848fb3b84c07a8be5a003f721bfa1975f9bb86adf676055584413d7bd

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return ''.join(v.to_bytes(2,'little').decode('utf-16-le','replace') for v in x)
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve([55357, 56832]), '😀')
check('boundary 2', solve([55296]), '�')
check('boundary 3', solve([65]), 'A')
check('boundary 4', solve([]), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
boundary 1	`��`	`😀`	Failed
boundary 2	`�`	`�`	Passed
boundary 3	`A`	`A`	Passed
boundary 4			Passed

SHA-256 / 72392a0734b08099d9cb584846ad6c761ece129ef0014078b19a6e40ed9cfa57

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json
import re, unicodedata, json, csv, io, html, base64, binascii, codecs, struct
from urllib.parse import quote, unquote, unquote_to_bytes, urlencode, parse_qsl, urlsplit, urlunsplit
from email.header import decode_header, make_header
from email.utils import getaddresses
from xml.etree import ElementTree as ET
import shlex, string, textwrap

N = 1
observations = []
def solve(x):
    try:
        return b''.join(v.to_bytes(2,'little') for v in x).decode('utf-16-le','replace')
    except Exception as exc:
        return {"error": type(exc).__name__}
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('boundary 1', solve([55357, 56832]), '😀')
check('boundary 2', solve([55296]), '�')
check('boundary 3', solve([65]), 'A')
check('boundary 4', solve([]), '')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
boundary 1	`😀`	`😀`	Passed
boundary 2	`�`	`�`	Passed
boundary 3	`A`	`A`	Passed
boundary 4			Passed

SHA-256 / 4599cedf5bf1bfff10286cf1ff73610b099c59f44f51db3757b95a3434044a2c

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:37:33.084172+00:00.

Case digest / 0b8d91e0edee7c1a5bdacbbc238fca1b0c113d54100151652730bbe309c76381