FA-72236 / Error-correcting codes / Open access

LLR quantizer saturates to the unsigned range · case 01

Large samples produce LLRs outside the decoder word size.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The limit is 2^bits - 1, the unsigned range, instead of the symmetric signed limit.

THE FAILURE

The limit is 2^bits - 1, the unsigned range, instead of the symmetric signed limit.

Unsuccessful approach: Allowing the extra negative two-complement code makes negative LLRs stronger than positive ones.

Case contract

Quantise soft demodulator samples to signed integer LLRs for a hard/soft decoder. Each sample y (integer) is scaled (v = y * scale, in hundredths), rounded to the nearest integer with halves rounded away from zero, and saturated symmetrically to +-(2^(bits-1) - 1). Positive LLR means bit 0, negative bit 1, zero is an erasure (None). Return [llrs, hard decisions].

Why this case matters

Fixed-point receivers quantise LLRs before feeding Viterbi or LDPC decoders; bias at this stage costs coding gain.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(samples, scale, bits):
    lim = 2 ** (bits - 1) - 1
    llrs, hard = [], []
    for y in samples:
        v = y * scale
        q = (abs(v) + 50) // 100
        q = min(q, 2 ** bits - 1)
        q = q if v >= 0 else -q
        llrs.append(q)
        hard.append(None if q == 0 else (0 if q > 0 else 1))
    return [llrs, hard]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[['regression [[-316, 153, -380, 115, 285], 2, 3]', [[-316, 153, -380, 115, 285], 2, 3], [[-3, 3, -3, 2, 3], [1, 0, 1, 0, 0]]], ['regression [[-175, 400, -173], 4, 5]', [[-175, 400, -173], 4, 5], [[-7, 15, -7], [1, 0, 1]]], ['partial-repair [[-273, 202, 226, 129, -219, -131], 2, 3]', [[-273, 202, 226, 129, -219, -131], 2, 3], [[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]], ['control [[329, -1, -98, 396, 80, -221, -227], 3, 5]', [[329, -1, -98, 396, 80, -221, -227], 3, 5], [[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]], ['control [[-110, -296, 193], 1, 3]', [[-110, -296, 193], 1, 3], [[-1, -3, 2], [1, 1, 0]]], ['control [[-139, 193, -225, -10, -203], 3, 4]', [[-139, 193, -225, -10, -203], 3, 4], [[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]], ['control [[-325, -180, 220, -322, 21], 4, 5]', [[-325, -180, 220, -322, 21], 4, 5], [[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]], ['control [[-341, 242, -149, -142, 5, 301], 1, 4]', [[-341, 242, -149, -142, 5, 301], 1, 4], [[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]]], [['regression [[136, 302, -270, -121, 100, 337, -271], 2, 3]', [[136, 302, -270, -121, 100, 337, -271], 2, 3], [[3, 3, -3, -2, 2, 3, -3], [0, 0, 1, 1, 0, 0, 1]]], ['regression [[251, -383, 56, 115, -2], 2, 4]', [[251, -383, 56, 115, -2], 2, 4], [[5, -7, 1, 2, 0], [0, 1, 0, 0, None]]], ['partial-repair [[-308, 132, -336, -334, 391, 337, -203], 4, 4]', [[-308, 132, -336, -334, 391, 337, -203], 4, 4], [[-7, 5, -7, -7, 7, 7, -7], [1, 0, 1, 1, 0, 0, 1]]], ['control [[396, 308, -178], 1, 4]', [[396, 308, -178], 1, 4], [[4, 3, -2], [0, 0, 1]]], ['control [[161, 358], 1, 4]', [[161, 358], 1, 4], [[2, 4], [0, 0]]], ['control [[184, 146], 3, 5]', [[184, 146], 3, 5], [[6, 4], [0, 0]]], ['control [[-87, 93], 3, 4]', [[-87, 93], 3, 4], [[-3, 3], [1, 0]]], ['control [[-219, -66, -258, -177, 231, 304], 3, 5]', [[-219, -66, -258, -177, 231, 304], 3, 5], [[-7, -2, -8, -5, 7, 9], [1, 1, 1, 1, 0, 0]]]], [['regression [[-293, -398, 93, -6, -35, -78, -62], 4, 5]', [[-293, -398, 93, -6, -35, -78, -62], 4, 5], [[-12, -15, 4, 0, -1, -3, -2], [1, 1, 0, None, 1, 1, 1]]], ['regression [[-119, -57, 243, 368, -75, -270, -363], 1, 3]', [[-119, -57, 243, 368, -75, -270, -363], 1, 3], [[-1, -1, 2, 3, -1, -3, -3], [1, 1, 0, 0, 1, 1, 1]]], ['partial-repair [[900, -900, 700, -800], 1, 4]', [[900, -900, 700, -800], 1, 4], [[7, -7, 7, -7], [0, 1, 0, 1]]], ['control [[395, -217, -351], 1, 4]', [[395, -217, -351], 1, 4], [[4, -2, -4], [0, 1, 1]]], ['control [[-128, 321, 78, -33, 59, -182], 1, 3]', [[-128, 321, 78, -33, 59, -182], 1, 3], [[-1, 3, 1, 0, 1, -2], [1, 0, 0, None, 0, 1]]], ['control [[250, -250, 150, -150, 49, -50], 1, 4]', [[250, -250, 150, -150, 49, -50], 1, 4], [[3, -3, 2, -2, 0, -1], [0, 1, 0, 1, None, 1]]], ['control [[0, 1, -1], 1, 3]', [[0, 1, -1], 1, 3], [[0, 0, 0], [None, None, None]]], ['control [[-1049, -1050, -1051], 1, 8]', [[-1049, -1050, -1051], 1, 8], [[-10, -11, -11], [1, 1, 1]]]], [['regression [[-350, 350], 2, 3]', [[-350, 350], 2, 3], [[-3, 3], [1, 0]]], ['regression [[-316, 153, -380, 115, 285], 2, 3]', [[-316, 153, -380, 115, 285], 2, 3], [[-3, 3, -3, 2, 3], [1, 0, 1, 0, 0]]], ['partial-repair [[-273, 202, 226, 129, -219, -131], 2, 3]', [[-273, 202, 226, 129, -219, -131], 2, 3], [[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]], ['control [[329, -1, -98, 396, 80, -221, -227], 3, 5]', [[329, -1, -98, 396, 80, -221, -227], 3, 5], [[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]], ['control [[-110, -296, 193], 1, 3]', [[-110, -296, 193], 1, 3], [[-1, -3, 2], [1, 1, 0]]], ['control [[-139, 193, -225, -10, -203], 3, 4]', [[-139, 193, -225, -10, -203], 3, 4], [[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]], ['control [[-325, -180, 220, -322, 21], 4, 5]', [[-325, -180, 220, -322, 21], 4, 5], [[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]], ['control [[-341, 242, -149, -142, 5, 301], 1, 4]', [[-341, 242, -149, -142, 5, 301], 1, 4], [[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]]], [['regression [[-273, 202, 226, 129, -219, -131], 2, 3]', [[-273, 202, 226, 129, -219, -131], 2, 3], [[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]], ['regression [[136, 302, -270, -121, 100, 337, -271], 2, 3]', [[136, 302, -270, -121, 100, 337, -271], 2, 3], [[3, 3, -3, -2, 2, 3, -3], [0, 0, 1, 1, 0, 0, 1]]], ['partial-repair [[251, -383, 56, 115, -2], 2, 4]', [[251, -383, 56, 115, -2], 2, 4], [[5, -7, 1, 2, 0], [0, 1, 0, 0, None]]], ['partial-repair [[-308, 132, -336, -334, 391, 337, -203], 4, 4]', [[-308, 132, -336, -334, 391, 337, -203], 4, 4], [[-7, 5, -7, -7, 7, 7, -7], [1, 0, 1, 1, 0, 0, 1]]], ['control [[396, 308, -178], 1, 4]', [[396, 308, -178], 1, 4], [[4, 3, -2], [0, 0, 1]]], ['control [[161, 358], 1, 4]', [[161, 358], 1, 4], [[2, 4], [0, 0]]], ['control [[184, 146], 3, 5]', [[184, 146], 3, 5], [[6, 4], [0, 0]]], ['control [[-87, 93], 3, 4]', [[-87, 93], 3, 4], [[-3, 3], [1, 0]]]]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
regression [[-316, 153, -380, 115, 285], 2, 3]	`[[-6, 3, -7, 2, 6], [1, 0, 1, 0, 0]]`	`[[-3, 3, -3, 2, 3], [1, 0, 1, 0, 0]]`	Failed
regression [[-175, 400, -173], 4, 5]	`[[-7, 16, -7], [1, 0, 1]]`	`[[-7, 15, -7], [1, 0, 1]]`	Failed
partial-repair [[-273, 202, 226, 129, -219, -131], 2, 3]	`[[-5, 4, 5, 3, -4, -3], [1, 0, 0, 0, 1, 1]]`	`[[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]`	Failed
control [[329, -1, -98, 396, 80, -221, -227], 3, 5]	`[[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]`	`[[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]`	Passed
control [[-110, -296, 193], 1, 3]	`[[-1, -3, 2], [1, 1, 0]]`	`[[-1, -3, 2], [1, 1, 0]]`	Passed
control [[-139, 193, -225, -10, -203], 3, 4]	`[[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]`	`[[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]`	Passed
control [[-325, -180, 220, -322, 21], 4, 5]	`[[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]`	`[[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]`	Passed
control [[-341, 242, -149, -142, 5, 301], 1, 4]	`[[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]`	`[[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]`	Passed

SHA-256 / 29eb6ad7e608b294fe0ee170d1e358bccb75cb3c255ceb27360375c097f29114

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(samples, scale, bits):
    lim = 2 ** (bits - 1) - 1
    llrs, hard = [], []
    for y in samples:
        v = y * scale
        q = (abs(v) + 50) // 100
        q = min(q, lim if v >= 0 else lim + 1)
        q = q if v >= 0 else -q
        llrs.append(q)
        hard.append(None if q == 0 else (0 if q > 0 else 1))
    return [llrs, hard]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[['regression [[-316, 153, -380, 115, 285], 2, 3]', [[-316, 153, -380, 115, 285], 2, 3], [[-3, 3, -3, 2, 3], [1, 0, 1, 0, 0]]], ['regression [[-175, 400, -173], 4, 5]', [[-175, 400, -173], 4, 5], [[-7, 15, -7], [1, 0, 1]]], ['partial-repair [[-273, 202, 226, 129, -219, -131], 2, 3]', [[-273, 202, 226, 129, -219, -131], 2, 3], [[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]], ['control [[329, -1, -98, 396, 80, -221, -227], 3, 5]', [[329, -1, -98, 396, 80, -221, -227], 3, 5], [[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]], ['control [[-110, -296, 193], 1, 3]', [[-110, -296, 193], 1, 3], [[-1, -3, 2], [1, 1, 0]]], ['control [[-139, 193, -225, -10, -203], 3, 4]', [[-139, 193, -225, -10, -203], 3, 4], [[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]], ['control [[-325, -180, 220, -322, 21], 4, 5]', [[-325, -180, 220, -322, 21], 4, 5], [[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]], ['control [[-341, 242, -149, -142, 5, 301], 1, 4]', [[-341, 242, -149, -142, 5, 301], 1, 4], [[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]]], [['regression [[136, 302, -270, -121, 100, 337, -271], 2, 3]', [[136, 302, -270, -121, 100, 337, -271], 2, 3], [[3, 3, -3, -2, 2, 3, -3], [0, 0, 1, 1, 0, 0, 1]]], ['regression [[251, -383, 56, 115, -2], 2, 4]', [[251, -383, 56, 115, -2], 2, 4], [[5, -7, 1, 2, 0], [0, 1, 0, 0, None]]], ['partial-repair [[-308, 132, -336, -334, 391, 337, -203], 4, 4]', [[-308, 132, -336, -334, 391, 337, -203], 4, 4], [[-7, 5, -7, -7, 7, 7, -7], [1, 0, 1, 1, 0, 0, 1]]], ['control [[396, 308, -178], 1, 4]', [[396, 308, -178], 1, 4], [[4, 3, -2], [0, 0, 1]]], ['control [[161, 358], 1, 4]', [[161, 358], 1, 4], [[2, 4], [0, 0]]], ['control [[184, 146], 3, 5]', [[184, 146], 3, 5], [[6, 4], [0, 0]]], ['control [[-87, 93], 3, 4]', [[-87, 93], 3, 4], [[-3, 3], [1, 0]]], ['control [[-219, -66, -258, -177, 231, 304], 3, 5]', [[-219, -66, -258, -177, 231, 304], 3, 5], [[-7, -2, -8, -5, 7, 9], [1, 1, 1, 1, 0, 0]]]], [['regression [[-293, -398, 93, -6, -35, -78, -62], 4, 5]', [[-293, -398, 93, -6, -35, -78, -62], 4, 5], [[-12, -15, 4, 0, -1, -3, -2], [1, 1, 0, None, 1, 1, 1]]], ['regression [[-119, -57, 243, 368, -75, -270, -363], 1, 3]', [[-119, -57, 243, 368, -75, -270, -363], 1, 3], [[-1, -1, 2, 3, -1, -3, -3], [1, 1, 0, 0, 1, 1, 1]]], ['partial-repair [[900, -900, 700, -800], 1, 4]', [[900, -900, 700, -800], 1, 4], [[7, -7, 7, -7], [0, 1, 0, 1]]], ['control [[395, -217, -351], 1, 4]', [[395, -217, -351], 1, 4], [[4, -2, -4], [0, 1, 1]]], ['control [[-128, 321, 78, -33, 59, -182], 1, 3]', [[-128, 321, 78, -33, 59, -182], 1, 3], [[-1, 3, 1, 0, 1, -2], [1, 0, 0, None, 0, 1]]], ['control [[250, -250, 150, -150, 49, -50], 1, 4]', [[250, -250, 150, -150, 49, -50], 1, 4], [[3, -3, 2, -2, 0, -1], [0, 1, 0, 1, None, 1]]], ['control [[0, 1, -1], 1, 3]', [[0, 1, -1], 1, 3], [[0, 0, 0], [None, None, None]]], ['control [[-1049, -1050, -1051], 1, 8]', [[-1049, -1050, -1051], 1, 8], [[-10, -11, -11], [1, 1, 1]]]], [['regression [[-350, 350], 2, 3]', [[-350, 350], 2, 3], [[-3, 3], [1, 0]]], ['regression [[-316, 153, -380, 115, 285], 2, 3]', [[-316, 153, -380, 115, 285], 2, 3], [[-3, 3, -3, 2, 3], [1, 0, 1, 0, 0]]], ['partial-repair [[-273, 202, 226, 129, -219, -131], 2, 3]', [[-273, 202, 226, 129, -219, -131], 2, 3], [[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]], ['control [[329, -1, -98, 396, 80, -221, -227], 3, 5]', [[329, -1, -98, 396, 80, -221, -227], 3, 5], [[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]], ['control [[-110, -296, 193], 1, 3]', [[-110, -296, 193], 1, 3], [[-1, -3, 2], [1, 1, 0]]], ['control [[-139, 193, -225, -10, -203], 3, 4]', [[-139, 193, -225, -10, -203], 3, 4], [[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]], ['control [[-325, -180, 220, -322, 21], 4, 5]', [[-325, -180, 220, -322, 21], 4, 5], [[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]], ['control [[-341, 242, -149, -142, 5, 301], 1, 4]', [[-341, 242, -149, -142, 5, 301], 1, 4], [[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]]], [['regression [[-273, 202, 226, 129, -219, -131], 2, 3]', [[-273, 202, 226, 129, -219, -131], 2, 3], [[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]], ['regression [[136, 302, -270, -121, 100, 337, -271], 2, 3]', [[136, 302, -270, -121, 100, 337, -271], 2, 3], [[3, 3, -3, -2, 2, 3, -3], [0, 0, 1, 1, 0, 0, 1]]], ['partial-repair [[251, -383, 56, 115, -2], 2, 4]', [[251, -383, 56, 115, -2], 2, 4], [[5, -7, 1, 2, 0], [0, 1, 0, 0, None]]], ['partial-repair [[-308, 132, -336, -334, 391, 337, -203], 4, 4]', [[-308, 132, -336, -334, 391, 337, -203], 4, 4], [[-7, 5, -7, -7, 7, 7, -7], [1, 0, 1, 1, 0, 0, 1]]], ['control [[396, 308, -178], 1, 4]', [[396, 308, -178], 1, 4], [[4, 3, -2], [0, 0, 1]]], ['control [[161, 358], 1, 4]', [[161, 358], 1, 4], [[2, 4], [0, 0]]], ['control [[184, 146], 3, 5]', [[184, 146], 3, 5], [[6, 4], [0, 0]]], ['control [[-87, 93], 3, 4]', [[-87, 93], 3, 4], [[-3, 3], [1, 0]]]]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
regression [[-316, 153, -380, 115, 285], 2, 3]	`[[-4, 3, -4, 2, 3], [1, 0, 1, 0, 0]]`	`[[-3, 3, -3, 2, 3], [1, 0, 1, 0, 0]]`	Failed
regression [[-175, 400, -173], 4, 5]	`[[-7, 15, -7], [1, 0, 1]]`	`[[-7, 15, -7], [1, 0, 1]]`	Passed
partial-repair [[-273, 202, 226, 129, -219, -131], 2, 3]	`[[-4, 3, 3, 3, -4, -3], [1, 0, 0, 0, 1, 1]]`	`[[-3, 3, 3, 3, -3, -3], [1, 0, 0, 0, 1, 1]]`	Failed
control [[329, -1, -98, 396, 80, -221, -227], 3, 5]	`[[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]`	`[[10, 0, -3, 12, 2, -7, -7], [0, None, 1, 0, 0, 1, 1]]`	Passed
control [[-110, -296, 193], 1, 3]	`[[-1, -3, 2], [1, 1, 0]]`	`[[-1, -3, 2], [1, 1, 0]]`	Passed
control [[-139, 193, -225, -10, -203], 3, 4]	`[[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]`	`[[-4, 6, -7, 0, -6], [1, 0, 1, None, 1]]`	Passed
control [[-325, -180, 220, -322, 21], 4, 5]	`[[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]`	`[[-13, -7, 9, -13, 1], [1, 1, 0, 1, 0]]`	Passed
control [[-341, 242, -149, -142, 5, 301], 1, 4]	`[[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]`	`[[-3, 2, -1, -1, 0, 3], [1, 0, 1, 1, None, 0]]`	Passed

SHA-256 / 76041365f76b651417fb7e07676b0cac7f34b4cbacfea178c6bbc8d0f9af4f86

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 8 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Verification & scope

A deterministic, bounded teaching model of the named code under the stated contract; not a production codec. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:48:36.679537+00:00.

Case digest / 7f8009c2549d5596e7e19274d1fa604d98fbaa46e3b9d6355cd26a6f54b6b98e