FAILURE MAP
← Case archive

FA-5306 / Python evaluation / Open access

Chained comparison evaluation: Reevaluates the shared middle expression after the first comparison · case 01

Reevaluates the shared middle expression after the first comparison and changes the required Python-language result.

Verified by executionVariant 1 · 5 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Reevaluates the shared middle expression after the first comparison. The executable reproducer isolates this operation from the rest of the contract.

VERIFIED REPAIR

Evaluate lhs() < middle() < rhs(). Evaluate the middle expression once, evaluate each operand left-to-right, and skip the final expression when the first comparison fails. Both inequalities are strict.

Unsuccessful approach: The alternative still fails because it evaluates the right operand before knowing whether the first comparison passes.

Case contract

Evaluate lhs() < middle() < rhs(). Evaluate the middle expression once, evaluate each operand left-to-right, and skip the final expression when the first comparison fails. Both inequalities are strict. Inputs use the finite Python types shown in the fixtures. An error string denotes the specified caught exception, not an unhandled process failure.

Why this case matters

Runs Python standard-library language operations in a deterministic local reproducer. Literal expected answers are authored independently. Related defects share an evaluation group to avoid benchmark split leakage.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(left, mid, right):
    trace=[]
    def lhs(): trace.append('left'); return left
    def middle(): trace.append('middle'); return mid
    def rhs(): trace.append('right'); return right
    result=lhs() < middle() and middle() < rhs()
    return [result,trace]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('increasing triple', solve(1, 2, 3), [True, ['left', 'middle', 'right']])
check('first comparison fails', solve(3, 2, 9), [False, ['left', 'middle']])
check('equal first operands', solve(2, 2, 3), [False, ['left', 'middle']])
check('equal last operands', solve(1, 2, 2), [False, ['left', 'middle', 'right']])
check('second comparison fails', solve(1, 3, 2), [False, ['left', 'middle', 'right']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
increasing triple[True, ['left', 'middle', 'middle', 'right']][True, ['left', 'middle', 'right']]Failed
first comparison fails[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal first operands[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal last operands[False, ['left', 'middle', 'middle', 'right']][False, ['left', 'middle', 'right']]Failed
second comparison fails[False, ['left', 'middle', 'middle', 'right']][False, ['left', 'middle', 'right']]Failed

SHA-256 / 5449700986cd046e3bd19211ea793aa411034cfb31d2066ad21be721f1589b87

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(left, mid, right):
    trace=[]
    def lhs(): trace.append('left'); return left
    def middle(): trace.append('middle'); return mid
    def rhs(): trace.append('right'); return right
    a,b,c=lhs(),middle(),rhs()
    return [a<b<c,trace]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('increasing triple', solve(1, 2, 3), [True, ['left', 'middle', 'right']])
check('first comparison fails', solve(3, 2, 9), [False, ['left', 'middle']])
check('equal first operands', solve(2, 2, 3), [False, ['left', 'middle']])
check('equal last operands', solve(1, 2, 2), [False, ['left', 'middle', 'right']])
check('second comparison fails', solve(1, 3, 2), [False, ['left', 'middle', 'right']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
increasing triple[True, ['left', 'middle', 'right']][True, ['left', 'middle', 'right']]Passed
first comparison fails[False, ['left', 'middle', 'right']][False, ['left', 'middle']]Failed
equal first operands[False, ['left', 'middle', 'right']][False, ['left', 'middle']]Failed
equal last operands[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed
second comparison fails[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed

SHA-256 / b49e3b76420f30a7db98ae43fa9d7edb7c354be43d59a1b05f93bdbb502404cd

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(left, mid, right):
    trace=[]
    def lhs(): trace.append('left'); return left
    def middle(): trace.append('middle'); return mid
    def rhs(): trace.append('right'); return right
    result=lhs() < middle() < rhs()
    return [result,trace]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('increasing triple', solve(1, 2, 3), [True, ['left', 'middle', 'right']])
check('first comparison fails', solve(3, 2, 9), [False, ['left', 'middle']])
check('equal first operands', solve(2, 2, 3), [False, ['left', 'middle']])
check('equal last operands', solve(1, 2, 2), [False, ['left', 'middle', 'right']])
check('second comparison fails', solve(1, 3, 2), [False, ['left', 'middle', 'right']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
increasing triple[True, ['left', 'middle', 'right']][True, ['left', 'middle', 'right']]Passed
first comparison fails[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal first operands[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal last operands[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed
second comparison fails[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed

SHA-256 / 2e00ac85bd278ba821bd7a185101168446ef53f78f007ccb5a4ae6cc027c6315

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:37:47.939942+00:00.

Case digest / b505bf5a258cb479e26e4eed44788b07285d801a5cdd50cc4bd112e555321b7a