FAILURE MAP
← Case archive

FA-5326 / Python evaluation / Open access

Chained comparison evaluation: Reverses the direction of the second comparison · case 01

Reverses the direction of the second comparison and changes the required Python-language result.

Verified by executionVariant 1 · 5 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Reverses the direction of the second comparison. The executable reproducer isolates this operation from the rest of the contract.

VERIFIED REPAIR

Evaluate lhs() < middle() < rhs(). Evaluate the middle expression once, evaluate each operand left-to-right, and skip the final expression when the first comparison fails. Both inequalities are strict.

Unsuccessful approach: The alternative still fails because it reevaluates the shared middle expression after the first comparison.

Case contract

Evaluate lhs() < middle() < rhs(). Evaluate the middle expression once, evaluate each operand left-to-right, and skip the final expression when the first comparison fails. Both inequalities are strict. Inputs use the finite Python types shown in the fixtures. An error string denotes the specified caught exception, not an unhandled process failure.

Why this case matters

Runs Python standard-library language operations in a deterministic local reproducer. Literal expected answers are authored independently. Related defects share an evaluation group to avoid benchmark split leakage.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(left, mid, right):
    trace=[]
    def lhs(): trace.append('left'); return left
    def middle(): trace.append('middle'); return mid
    def rhs(): trace.append('right'); return right
    result=lhs() < middle() > rhs()
    return [result,trace]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('increasing triple', solve(1, 2, 3), [True, ['left', 'middle', 'right']])
check('first comparison fails', solve(3, 2, 9), [False, ['left', 'middle']])
check('equal first operands', solve(2, 2, 3), [False, ['left', 'middle']])
check('equal last operands', solve(1, 2, 2), [False, ['left', 'middle', 'right']])
check('second comparison fails', solve(1, 3, 2), [False, ['left', 'middle', 'right']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
increasing triple[False, ['left', 'middle', 'right']][True, ['left', 'middle', 'right']]Failed
first comparison fails[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal first operands[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal last operands[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed
second comparison fails[True, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Failed

SHA-256 / 8ea0ef74128758db9040171a96cab9a5660d9681938cc048ec36fe5f032de6b7

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(left, mid, right):
    trace=[]
    def lhs(): trace.append('left'); return left
    def middle(): trace.append('middle'); return mid
    def rhs(): trace.append('right'); return right
    result=lhs() < middle() and middle() < rhs()
    return [result,trace]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('increasing triple', solve(1, 2, 3), [True, ['left', 'middle', 'right']])
check('first comparison fails', solve(3, 2, 9), [False, ['left', 'middle']])
check('equal first operands', solve(2, 2, 3), [False, ['left', 'middle']])
check('equal last operands', solve(1, 2, 2), [False, ['left', 'middle', 'right']])
check('second comparison fails', solve(1, 3, 2), [False, ['left', 'middle', 'right']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
increasing triple[True, ['left', 'middle', 'middle', 'right']][True, ['left', 'middle', 'right']]Failed
first comparison fails[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal first operands[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal last operands[False, ['left', 'middle', 'middle', 'right']][False, ['left', 'middle', 'right']]Failed
second comparison fails[False, ['left', 'middle', 'middle', 'right']][False, ['left', 'middle', 'right']]Failed

SHA-256 / 5449700986cd046e3bd19211ea793aa411034cfb31d2066ad21be721f1589b87

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(left, mid, right):
    trace=[]
    def lhs(): trace.append('left'); return left
    def middle(): trace.append('middle'); return mid
    def rhs(): trace.append('right'); return right
    result=lhs() < middle() < rhs()
    return [result,trace]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('increasing triple', solve(1, 2, 3), [True, ['left', 'middle', 'right']])
check('first comparison fails', solve(3, 2, 9), [False, ['left', 'middle']])
check('equal first operands', solve(2, 2, 3), [False, ['left', 'middle']])
check('equal last operands', solve(1, 2, 2), [False, ['left', 'middle', 'right']])
check('second comparison fails', solve(1, 3, 2), [False, ['left', 'middle', 'right']])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
increasing triple[True, ['left', 'middle', 'right']][True, ['left', 'middle', 'right']]Passed
first comparison fails[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal first operands[False, ['left', 'middle']][False, ['left', 'middle']]Passed
equal last operands[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed
second comparison fails[False, ['left', 'middle', 'right']][False, ['left', 'middle', 'right']]Passed

SHA-256 / 2e00ac85bd278ba821bd7a185101168446ef53f78f007ccb5a4ae6cc027c6315

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:37:48.295671+00:00.

Case digest / dc37ba191b225cf3948c27f047e1d25a5fa7823bf80c97189b724b3d695eb4f7