FA-80816 / Bidirectional text layout / Open access

Paragraph separation and direction: separator ownership · case 01

Paragraph separators appear at the start of the following paragraph.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The separator is added to the next paragraph instead of the one it ends.

VERIFIED REPAIR

Keep the separator with the paragraph it terminates.

Unsuccessful approach: Dropping the separator loses text.

Case contract

Input [text, default level]. Paragraph separators (\n, \r, U+2029; \r\n counts once) end a paragraph and stay with it. A trailing non-empty remainder is a paragraph. Each paragraph takes level 1 if its first letter is uppercase (stand-in for RTL), 0 if lowercase, else the default. Return [[paragraph, level]].

Why this case matters

Mixed right-to-left and left-to-right text must resolve levels and visual order exactly, or words, numbers and carets land in the wrong place.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(x):
    text, default = x
    paras = []
    cur = ''
    i = 0
    while i < len(text):
        ch = text[i]
        if ch == '\r' and text[i + 1:i + 2] == '\n':
            cur += '\r\n'
            paras.append(cur)
            cur = ''
            i += 2
            continue
        if ch in '\n\r\u2029':
            paras.append(cur)
            cur = ''
        cur += ch
        i += 1
    if cur:
        paras.append(cur)
    out = []
    for p in paras:
        level = default
        for ch in p:
            if ch.isalpha():
                level = 1 if ch.isupper() else 0
                break
        out.append([p, level])
    return out
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('regression: separator ownership', [' Y,1Y\n\r\n', 1], [[' Y,1Y\n', 1], ['\r\n', 1]]), ('regression: separator ownership', ['.a1 \r2a2\n', 0], [['.a1 \r', 0], ['2a2\n', 0]]), ('regression: separator ownership', ['bX YXX\nX', 1], [['bX YXX\n', 0], ['X', 1]]), ('separator only paragraph', ['a\n\nZ', 1], [['a\n', 0], ['\n', 1], ['Z', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['Z,b\r\nZ1a,,Y2211', 0], [['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]), ('control layout', ['Yb.', 0], [['Yb.', 1]])], [('regression: separator ownership', [' 1c!b.Y2\u2029\n\u2029\nY.b\n', 1], [[' 1c!b.Y2\u2029', 0], ['\n', 1], ['\u2029', 1], ['\n', 1], ['Y.b\n', 1]]), ('regression: separator ownership', ['\nc2Z\r\n', 0], [['\n', 0], ['c2Z\r\n', 0]]), ('regression: separator ownership', ['XbZ,.Y c\rYb', 0], [['XbZ,.Y c\r', 1], ['Yb', 1]]), ('regression: separator ownership', ['bb\n.Z. ', 1], [['bb\n', 0], ['.Z. ', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['\r\n1,X1 \r\n.b,', 1], [['\r\n', 1], ['1,X1 \r\n', 1], ['.b,', 0]]), ('control layout', ['b!X', 0], [['b!X', 0]])], [('regression: separator ownership', ['\n!, \r,1\nZ', 1], [['\n', 1], ['!, \r', 1], [',1\n', 1], ['Z', 1]]), ('regression: separator ownership', ['.,bZ\nbXY', 1], [['.,bZ\n', 0], ['bXY', 0]]), ('regression: separator ownership', ['1 \n\u2029cX\r\n1\n\n\n.c', 0], [['1 \n', 0], ['\u2029', 0], ['cX\r\n', 0], ['1\n', 0], ['\n', 0], ['\n', 0], ['.c', 0]]), ('regression: separator ownership', ['..Z\r\na\n\r\na!Ya b', 1], [['..Z\r\n', 1], ['a\n', 0], ['\r\n', 1], ['a!Ya b', 0]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['aabY!.! XZa .Yc', 0], [['aabY!.! XZa .Yc', 0]]), ('control layout', ['a! \r\nY.a1', 1], [['a! \r\n', 0], ['Y.a1', 1]])], [('regression: separator ownership', ['Ya\nb c,,,a1\r\n\r\n\n\u2029\n', 0], [['Ya\n', 1], ['b c,,,a1\r\n', 0], ['\r\n', 0], ['\n', 0], ['\u2029', 0], ['\n', 0]]), ('regression: separator ownership', ['21!b!YcZ\r\n\n.Y Y!', 1], [['21!b!YcZ\r\n', 0], ['\n', 1], ['.Y Y!', 1]]), ('regression: separator ownership', ['a\r\n\nY1X111,2Z', 1], [['a\r\n', 0], ['\n', 1], ['Y1X111,2Z', 1]]), ('regression: separator ownership', ['Z\u2029\r\n Z', 0], [['Z\u2029', 1], ['\r\n', 0], [' Z', 1]]), ('separator only paragraph', ['a\n\nZ', 1], [['a\n', 0], ['\n', 1], ['Z', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('control layout', [',Yb!', 0], [[',Yb!', 1]]), ('control layout', ['a  \r\n1', 1], [['a  \r\n', 0], ['1', 1]])], [('regression: separator ownership', [' ZZ\nXb', 0], [[' ZZ\n', 1], ['Xb', 1]]), ('regression: separator ownership', ['c1\u2029\nb2', 1], [['c1\u2029', 0], ['\n', 1], ['b2', 0]]), ('regression: separator ownership', ['c\rbX\r\n', 0], [['c\r', 0], ['bX\r\n', 0]]), ('regression: separator ownership', [',Y. X a,\r\n1c1,\n2', 1], [[',Y. X a,\r\n', 1], ['1c1,\n', 0], ['2', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('control layout', ['12.,cZX', 0], [['12.,cZX', 0]]), ('control layout', ['a,1.2Z', 1], [['a,1.2Z', 0]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
digits then letter	`[['12 X', 1], ['\n', 0]]`	`[['12 X\n', 1]]`	Failed
regression: separator ownership	`[[' Y,1Y', 1], ['\n\r\n', 1]]`	`[[' Y,1Y\n', 1], ['\r\n', 1]]`	Failed
regression: separator ownership	`[['.a1 ', 0], ['\r2a2', 0], ['\n', 0]]`	`[['.a1 \r', 0], ['2a2\n', 0]]`	Failed
regression: separator ownership	`[['bX YXX', 0], ['\nX', 1]]`	`[['bX YXX\n', 0], ['X', 1]]`	Failed
separator only paragraph	`[['a', 0], ['\n', 1], ['\nZ', 1]]`	`[['a\n', 0], ['\n', 1], ['Z', 1]]`	Failed
CRLF is one separator	`[['ab\r\n', 0], ['XY', 1]]`	`[['ab\r\n', 0], ['XY', 1]]`	Passed
control layout	`[['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]`	`[['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]`	Passed
control layout	`[['Yb.', 1]]`	`[['Yb.', 1]]`	Passed

SHA-256 / a04b69495af27d15de86341f2c9b83315203e84a3a2823af3eeebfeaf9847d2a

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(x):
    text, default = x
    paras = []
    cur = ''
    i = 0
    while i < len(text):
        ch = text[i]
        if ch == '\r' and text[i + 1:i + 2] == '\n':
            cur += '\r\n'
            paras.append(cur)
            cur = ''
            i += 2
            continue
        if ch in '\n\r\u2029':
            paras.append(cur)
            cur = ''
        else:
            cur += ch
        i += 1
    if cur:
        paras.append(cur)
    out = []
    for p in paras:
        level = default
        for ch in p:
            if ch.isalpha():
                level = 1 if ch.isupper() else 0
                break
        out.append([p, level])
    return out
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('regression: separator ownership', [' Y,1Y\n\r\n', 1], [[' Y,1Y\n', 1], ['\r\n', 1]]), ('regression: separator ownership', ['.a1 \r2a2\n', 0], [['.a1 \r', 0], ['2a2\n', 0]]), ('regression: separator ownership', ['bX YXX\nX', 1], [['bX YXX\n', 0], ['X', 1]]), ('separator only paragraph', ['a\n\nZ', 1], [['a\n', 0], ['\n', 1], ['Z', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['Z,b\r\nZ1a,,Y2211', 0], [['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]), ('control layout', ['Yb.', 0], [['Yb.', 1]])], [('regression: separator ownership', [' 1c!b.Y2\u2029\n\u2029\nY.b\n', 1], [[' 1c!b.Y2\u2029', 0], ['\n', 1], ['\u2029', 1], ['\n', 1], ['Y.b\n', 1]]), ('regression: separator ownership', ['\nc2Z\r\n', 0], [['\n', 0], ['c2Z\r\n', 0]]), ('regression: separator ownership', ['XbZ,.Y c\rYb', 0], [['XbZ,.Y c\r', 1], ['Yb', 1]]), ('regression: separator ownership', ['bb\n.Z. ', 1], [['bb\n', 0], ['.Z. ', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['\r\n1,X1 \r\n.b,', 1], [['\r\n', 1], ['1,X1 \r\n', 1], ['.b,', 0]]), ('control layout', ['b!X', 0], [['b!X', 0]])], [('regression: separator ownership', ['\n!, \r,1\nZ', 1], [['\n', 1], ['!, \r', 1], [',1\n', 1], ['Z', 1]]), ('regression: separator ownership', ['.,bZ\nbXY', 1], [['.,bZ\n', 0], ['bXY', 0]]), ('regression: separator ownership', ['1 \n\u2029cX\r\n1\n\n\n.c', 0], [['1 \n', 0], ['\u2029', 0], ['cX\r\n', 0], ['1\n', 0], ['\n', 0], ['\n', 0], ['.c', 0]]), ('regression: separator ownership', ['..Z\r\na\n\r\na!Ya b', 1], [['..Z\r\n', 1], ['a\n', 0], ['\r\n', 1], ['a!Ya b', 0]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['aabY!.! XZa .Yc', 0], [['aabY!.! XZa .Yc', 0]]), ('control layout', ['a! \r\nY.a1', 1], [['a! \r\n', 0], ['Y.a1', 1]])], [('regression: separator ownership', ['Ya\nb c,,,a1\r\n\r\n\n\u2029\n', 0], [['Ya\n', 1], ['b c,,,a1\r\n', 0], ['\r\n', 0], ['\n', 0], ['\u2029', 0], ['\n', 0]]), ('regression: separator ownership', ['21!b!YcZ\r\n\n.Y Y!', 1], [['21!b!YcZ\r\n', 0], ['\n', 1], ['.Y Y!', 1]]), ('regression: separator ownership', ['a\r\n\nY1X111,2Z', 1], [['a\r\n', 0], ['\n', 1], ['Y1X111,2Z', 1]]), ('regression: separator ownership', ['Z\u2029\r\n Z', 0], [['Z\u2029', 1], ['\r\n', 0], [' Z', 1]]), ('separator only paragraph', ['a\n\nZ', 1], [['a\n', 0], ['\n', 1], ['Z', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('control layout', [',Yb!', 0], [[',Yb!', 1]]), ('control layout', ['a  \r\n1', 1], [['a  \r\n', 0], ['1', 1]])], [('regression: separator ownership', [' ZZ\nXb', 0], [[' ZZ\n', 1], ['Xb', 1]]), ('regression: separator ownership', ['c1\u2029\nb2', 1], [['c1\u2029', 0], ['\n', 1], ['b2', 0]]), ('regression: separator ownership', ['c\rbX\r\n', 0], [['c\r', 0], ['bX\r\n', 0]]), ('regression: separator ownership', [',Y. X a,\r\n1c1,\n2', 1], [[',Y. X a,\r\n', 1], ['1c1,\n', 0], ['2', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('control layout', ['12.,cZX', 0], [['12.,cZX', 0]]), ('control layout', ['a,1.2Z', 1], [['a,1.2Z', 0]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
digits then letter	`[['12 X', 1]]`	`[['12 X\n', 1]]`	Failed
regression: separator ownership	`[[' Y,1Y', 1], ['\r\n', 1]]`	`[[' Y,1Y\n', 1], ['\r\n', 1]]`	Failed
regression: separator ownership	`[['.a1 ', 0], ['2a2', 0]]`	`[['.a1 \r', 0], ['2a2\n', 0]]`	Failed
regression: separator ownership	`[['bX YXX', 0], ['X', 1]]`	`[['bX YXX\n', 0], ['X', 1]]`	Failed
separator only paragraph	`[['a', 0], ['', 1], ['Z', 1]]`	`[['a\n', 0], ['\n', 1], ['Z', 1]]`	Failed
CRLF is one separator	`[['ab\r\n', 0], ['XY', 1]]`	`[['ab\r\n', 0], ['XY', 1]]`	Passed
control layout	`[['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]`	`[['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]`	Passed
control layout	`[['Yb.', 1]]`	`[['Yb.', 1]]`	Passed

SHA-256 / 2b8445c1e36149c7cfe614c99ca8698bd3f692665c32a6dfe6f8adca490fdcfa

3 / The verified repair

Exit 0

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(x):
    text, default = x
    paras = []
    cur = ''
    i = 0
    while i < len(text):
        ch = text[i]
        if ch == '\r' and text[i + 1:i + 2] == '\n':
            cur += '\r\n'
            paras.append(cur)
            cur = ''
            i += 2
            continue
        cur += ch
        if ch in '\n\r\u2029':
            paras.append(cur)
            cur = ''
        i += 1
    if cur:
        paras.append(cur)
    out = []
    for p in paras:
        level = default
        for ch in p:
            if ch.isalpha():
                level = 1 if ch.isupper() else 0
                break
        out.append([p, level])
    return out
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('regression: separator ownership', [' Y,1Y\n\r\n', 1], [[' Y,1Y\n', 1], ['\r\n', 1]]), ('regression: separator ownership', ['.a1 \r2a2\n', 0], [['.a1 \r', 0], ['2a2\n', 0]]), ('regression: separator ownership', ['bX YXX\nX', 1], [['bX YXX\n', 0], ['X', 1]]), ('separator only paragraph', ['a\n\nZ', 1], [['a\n', 0], ['\n', 1], ['Z', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['Z,b\r\nZ1a,,Y2211', 0], [['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]), ('control layout', ['Yb.', 0], [['Yb.', 1]])], [('regression: separator ownership', [' 1c!b.Y2\u2029\n\u2029\nY.b\n', 1], [[' 1c!b.Y2\u2029', 0], ['\n', 1], ['\u2029', 1], ['\n', 1], ['Y.b\n', 1]]), ('regression: separator ownership', ['\nc2Z\r\n', 0], [['\n', 0], ['c2Z\r\n', 0]]), ('regression: separator ownership', ['XbZ,.Y c\rYb', 0], [['XbZ,.Y c\r', 1], ['Yb', 1]]), ('regression: separator ownership', ['bb\n.Z. ', 1], [['bb\n', 0], ['.Z. ', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['\r\n1,X1 \r\n.b,', 1], [['\r\n', 1], ['1,X1 \r\n', 1], ['.b,', 0]]), ('control layout', ['b!X', 0], [['b!X', 0]])], [('regression: separator ownership', ['\n!, \r,1\nZ', 1], [['\n', 1], ['!, \r', 1], [',1\n', 1], ['Z', 1]]), ('regression: separator ownership', ['.,bZ\nbXY', 1], [['.,bZ\n', 0], ['bXY', 0]]), ('regression: separator ownership', ['1 \n\u2029cX\r\n1\n\n\n.c', 0], [['1 \n', 0], ['\u2029', 0], ['cX\r\n', 0], ['1\n', 0], ['\n', 0], ['\n', 0], ['.c', 0]]), ('regression: separator ownership', ['..Z\r\na\n\r\na!Ya b', 1], [['..Z\r\n', 1], ['a\n', 0], ['\r\n', 1], ['a!Ya b', 0]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('control layout', ['aabY!.! XZa .Yc', 0], [['aabY!.! XZa .Yc', 0]]), ('control layout', ['a! \r\nY.a1', 1], [['a! \r\n', 0], ['Y.a1', 1]])], [('regression: separator ownership', ['Ya\nb c,,,a1\r\n\r\n\n\u2029\n', 0], [['Ya\n', 1], ['b c,,,a1\r\n', 0], ['\r\n', 0], ['\n', 0], ['\u2029', 0], ['\n', 0]]), ('regression: separator ownership', ['21!b!YcZ\r\n\n.Y Y!', 1], [['21!b!YcZ\r\n', 0], ['\n', 1], ['.Y Y!', 1]]), ('regression: separator ownership', ['a\r\n\nY1X111,2Z', 1], [['a\r\n', 0], ['\n', 1], ['Y1X111,2Z', 1]]), ('regression: separator ownership', ['Z\u2029\r\n Z', 0], [['Z\u2029', 1], ['\r\n', 0], [' Z', 1]]), ('separator only paragraph', ['a\n\nZ', 1], [['a\n', 0], ['\n', 1], ['Z', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('control layout', [',Yb!', 0], [[',Yb!', 1]]), ('control layout', ['a  \r\n1', 1], [['a  \r\n', 0], ['1', 1]])], [('regression: separator ownership', [' ZZ\nXb', 0], [[' ZZ\n', 1], ['Xb', 1]]), ('regression: separator ownership', ['c1\u2029\nb2', 1], [['c1\u2029', 0], ['\n', 1], ['b2', 0]]), ('regression: separator ownership', ['c\rbX\r\n', 0], [['c\r', 0], ['bX\r\n', 0]]), ('regression: separator ownership', [',Y. X a,\r\n1c1,\n2', 1], [[',Y. X a,\r\n', 1], ['1c1,\n', 0], ['2', 1]]), ('CRLF is one separator', ['ab\r\nXY', 0], [['ab\r\n', 0], ['XY', 1]]), ('digits then letter', ['12 X\n', 0], [['12 X\n', 1]]), ('control layout', ['12.,cZX', 0], [['12.,cZX', 0]]), ('control layout', ['a,1.2Z', 1], [['a,1.2Z', 0]])]]
for label, args, expected in fixtures[N - 1]:
    check(label, solve(args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
digits then letter	`[['12 X\n', 1]]`	`[['12 X\n', 1]]`	Passed
regression: separator ownership	`[[' Y,1Y\n', 1], ['\r\n', 1]]`	`[[' Y,1Y\n', 1], ['\r\n', 1]]`	Passed
regression: separator ownership	`[['.a1 \r', 0], ['2a2\n', 0]]`	`[['.a1 \r', 0], ['2a2\n', 0]]`	Passed
regression: separator ownership	`[['bX YXX\n', 0], ['X', 1]]`	`[['bX YXX\n', 0], ['X', 1]]`	Passed
separator only paragraph	`[['a\n', 0], ['\n', 1], ['Z', 1]]`	`[['a\n', 0], ['\n', 1], ['Z', 1]]`	Passed
CRLF is one separator	`[['ab\r\n', 0], ['XY', 1]]`	`[['ab\r\n', 0], ['XY', 1]]`	Passed
control layout	`[['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]`	`[['Z,b\r\n', 1], ['Z1a,,Y2211', 1]]`	Passed
control layout	`[['Yb.', 1]]`	`[['Yb.', 1]]`	Passed

SHA-256 / be980281c8771c7f8b3106e64d0a2f11e7a54a55989b3e2faab39310cc3df993

Verification & scope

A deterministic toy bidi model over stipulated class labels and integer levels; it is inspired by, but does not claim conformance to, any published algorithm. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:49:57.369703+00:00.

Case digest / a537566ac8112e61db5dbb6a5bd42ac5171b2c88dc8517835b5d9908bde6112d