FAILURE MAP
← Case archive

FA-92346 / HVAC thermostat control / Open access

Heating call starts without its differential · case 01

The furnace starts as soon as the room dips below setpoint and short-cycles around it.

Verified by executionVariant 1 · 8 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The heat start test compares against heat_sp instead of heat_sp minus the differential.

VERIFIED REPAIR

Start heating only below heat_sp - 1.0 F.

Unsuccessful approach: Using half the differential still starts heating too early.

Case contract

Auto heat/cool thermostat sampled once per minute (temps in tenths F). If the cooling setpoint is less than 3.0 F above the heating setpoint it is pushed up to heat_sp + 30 (the heating setpoint wins). Heating starts below heat_sp - 1.0 F and ends at temp >= heat_sp; cooling starts above cool_sp + 1.0 F and ends at temp <= cool_sp. After a heating call ends, cooling may not start until 5 minutes later, and vice versa. Return heat, cool or idle per minute.

Why this case matters

Residential and light-commercial thermostats make these decisions sample by sample; each defect changes equipment calls in a way that shows up as short cycling, comfort complaints or equipment stress.

1 / The failure

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(heat_sp, cool_sp, temps):
    diff = 10
    if cool_sp - heat_sp < 30:
        cool_sp = heat_sp + 30
    state = 'idle'
    last_heat_end = None
    last_cool_end = None
    out = []
    for minute, temp in enumerate(temps):
        if state == 'heat' and temp >= heat_sp:
            state = 'idle'
            last_heat_end = minute
        elif state == 'cool' and temp <= cool_sp:
            state = 'idle'
            last_cool_end = minute
        if state == 'idle':
            if temp < heat_sp and (last_cool_end is None or minute - last_cool_end >= 5):
                state = 'heat'
            elif temp > cool_sp + diff and (last_heat_end is None or minute - last_heat_end >= 5):
                state = 'cool'
        out.append(state)
    return out
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['regression: scenario 6',
   [680,
    720,
    [666, 650, 675, 664, 671, 693, 693, 695, 674, 675, 657, 635, 629, 632, 629, 632, 636, 632, 620]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['regression: scenario 1',
   [680, 690, [693, 687, 709, 709, 685, 704, 684, 680, 686, 687, 679]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: scenario 2',
   [680, 700, [681, 686, 698, 703, 724, 744, 752, 732, 755, 742, 762, 772]],
   ['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']],
  ['control: scenario 3',
   [680,
    740,
    [649,
     667,
     643,
     622,
     637,
     657,
     634,
     649,
     674,
     654,
     672,
     675,
     698,
     687,
     666,
     654,
     670,
     645,
     654,
     637,
     651,
     672]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']]],
 [['regression: scenario 2',
   [680, 710, [674, 681, 688, 699, 718, 698, 716, 709, 691, 672, 678, 701, 724, 699, 683, 698, 691, 701]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle']],
  ['regression: scenario 9',
   [680, 720, [674, 698, 687, 683, 659, 647, 658, 656, 658, 674, 664, 656, 672, 650]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 700, [687, 695, 710, 721, 712, 724, 736, 738, 737, 740, 748]],
   ['idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']],
  ['regression: scenario 3',
   [680, 690, [686, 679, 704, 696, 678, 658, 635, 648, 643, 657, 650, 661, 672, 691, 687, 685, 708, 687]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle']]],
 [['regression: scenario 5',
   [680, 740, [676, 673, 693, 690, 714, 709, 715, 740, 727, 735, 738, 752, 754, 729, 752, 746]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'idle',
    'cool',
    'cool']],
  ['regression: scenario 16',
   [680, 740, [684, 671, 648, 650, 634, 633, 624, 643, 667, 647, 630, 639]],
   ['idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 710, [758, 778, 769, 788, 780, 787, 790, 773, 789, 766, 785, 779, 790, 790, 790, 777, 790]],
   ['cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']],
  ['control: scenario 2',
   [680, 740, [646, 654, 672, 672, 692, 717, 736, 759, 769, 790, 790]],
   ['heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']]],
 [['regression: scenario 11',
   [680, 690, [661, 683, 677, 656, 633, 621, 620, 620, 620, 629, 635, 658, 682, 657, 659, 665]],
   ['heat',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'heat',
    'heat',
    'heat']],
  ['regression: scenario 12',
   [680, 710, [710, 715, 700, 681, 698, 674, 698, 701, 681, 675, 664]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['regression: scenario 1',
   [680,
    690,
    [671, 646, 636, 635, 620, 625, 635, 653, 659, 636, 645, 633, 620, 628, 620, 638, 653, 655, 666, 678]],
   ['idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: scenario 2',
   [680, 740, [655, 638, 663, 641, 622, 636, 628, 621, 621, 620, 635, 632, 644, 651, 656, 675, 687]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle']]],
 [['regression: scenario 9',
   [680, 690, [673, 694, 686, 686, 699, 720, 727, 736, 726, 719]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool']],
  ['regression: scenario 34',
   [680,
    700,
    [735,
     716,
     727,
     705,
     684,
     664,
     652,
     673,
     670,
     688,
     692,
     705,
     701,
     720,
     743,
     746,
     741,
     746,
     763,
     788,
     790]],
   ['cool',
    'cool',
    'cool',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 720, [774, 779, 785, 790, 790, 781, 778, 777, 770, 767, 743, 730, 708, 690]],
   ['cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'idle',
    'idle']],
  ['regression: scenario 2',
   [680,
    710,
    [701,
     679,
     702,
     721,
     727,
     748,
     743,
     720,
     741,
     738,
     762,
     769,
     758,
     740,
     744,
     725,
     748,
     728,
     742,
     755,
     769]],
   ['idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']]]]
for label, args, expected in fixtures[N-1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
regression: exact thresholds['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']Failed
regression: scenario 6['heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']['heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']Failed
control: heat then locked cool['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']Passed
control: short heat cycle then hot['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']Passed
control: deadband squeeze['idle', 'cool', 'cool', 'idle', 'idle']['idle', 'cool', 'cool', 'idle', 'idle']Passed
regression: scenario 1['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat']['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']Failed
control: scenario 2['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']Passed
control: scenario 3['heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']['heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']Passed

SHA-256 / 67846e5521136de4612e54048da95f303d53336c69c283a3be919c6e88668746

2 / The unsuccessful fix

Exit 1
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(heat_sp, cool_sp, temps):
    diff = 10
    if cool_sp - heat_sp < 30:
        cool_sp = heat_sp + 30
    state = 'idle'
    last_heat_end = None
    last_cool_end = None
    out = []
    for minute, temp in enumerate(temps):
        if state == 'heat' and temp >= heat_sp:
            state = 'idle'
            last_heat_end = minute
        elif state == 'cool' and temp <= cool_sp:
            state = 'idle'
            last_cool_end = minute
        if state == 'idle':
            if temp < heat_sp - diff // 2 and (last_cool_end is None or minute - last_cool_end >= 5):
                state = 'heat'
            elif temp > cool_sp + diff and (last_heat_end is None or minute - last_heat_end >= 5):
                state = 'cool'
        out.append(state)
    return out
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['regression: scenario 6',
   [680,
    720,
    [666, 650, 675, 664, 671, 693, 693, 695, 674, 675, 657, 635, 629, 632, 629, 632, 636, 632, 620]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['regression: scenario 1',
   [680, 690, [693, 687, 709, 709, 685, 704, 684, 680, 686, 687, 679]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: scenario 2',
   [680, 700, [681, 686, 698, 703, 724, 744, 752, 732, 755, 742, 762, 772]],
   ['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']],
  ['control: scenario 3',
   [680,
    740,
    [649,
     667,
     643,
     622,
     637,
     657,
     634,
     649,
     674,
     654,
     672,
     675,
     698,
     687,
     666,
     654,
     670,
     645,
     654,
     637,
     651,
     672]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']]],
 [['regression: scenario 2',
   [680, 710, [674, 681, 688, 699, 718, 698, 716, 709, 691, 672, 678, 701, 724, 699, 683, 698, 691, 701]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle']],
  ['regression: scenario 9',
   [680, 720, [674, 698, 687, 683, 659, 647, 658, 656, 658, 674, 664, 656, 672, 650]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 700, [687, 695, 710, 721, 712, 724, 736, 738, 737, 740, 748]],
   ['idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']],
  ['regression: scenario 3',
   [680, 690, [686, 679, 704, 696, 678, 658, 635, 648, 643, 657, 650, 661, 672, 691, 687, 685, 708, 687]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle']]],
 [['regression: scenario 5',
   [680, 740, [676, 673, 693, 690, 714, 709, 715, 740, 727, 735, 738, 752, 754, 729, 752, 746]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'idle',
    'cool',
    'cool']],
  ['regression: scenario 16',
   [680, 740, [684, 671, 648, 650, 634, 633, 624, 643, 667, 647, 630, 639]],
   ['idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 710, [758, 778, 769, 788, 780, 787, 790, 773, 789, 766, 785, 779, 790, 790, 790, 777, 790]],
   ['cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']],
  ['control: scenario 2',
   [680, 740, [646, 654, 672, 672, 692, 717, 736, 759, 769, 790, 790]],
   ['heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']]],
 [['regression: scenario 11',
   [680, 690, [661, 683, 677, 656, 633, 621, 620, 620, 620, 629, 635, 658, 682, 657, 659, 665]],
   ['heat',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'heat',
    'heat',
    'heat']],
  ['regression: scenario 12',
   [680, 710, [710, 715, 700, 681, 698, 674, 698, 701, 681, 675, 664]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['regression: scenario 1',
   [680,
    690,
    [671, 646, 636, 635, 620, 625, 635, 653, 659, 636, 645, 633, 620, 628, 620, 638, 653, 655, 666, 678]],
   ['idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: scenario 2',
   [680, 740, [655, 638, 663, 641, 622, 636, 628, 621, 621, 620, 635, 632, 644, 651, 656, 675, 687]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle']]],
 [['regression: scenario 9',
   [680, 690, [673, 694, 686, 686, 699, 720, 727, 736, 726, 719]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool']],
  ['regression: scenario 34',
   [680,
    700,
    [735,
     716,
     727,
     705,
     684,
     664,
     652,
     673,
     670,
     688,
     692,
     705,
     701,
     720,
     743,
     746,
     741,
     746,
     763,
     788,
     790]],
   ['cool',
    'cool',
    'cool',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 720, [774, 779, 785, 790, 790, 781, 778, 777, 770, 767, 743, 730, 708, 690]],
   ['cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'idle',
    'idle']],
  ['regression: scenario 2',
   [680,
    710,
    [701,
     679,
     702,
     721,
     727,
     748,
     743,
     720,
     741,
     738,
     762,
     769,
     758,
     740,
     744,
     725,
     748,
     728,
     742,
     755,
     769]],
   ['idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']]]]
for label, args, expected in fixtures[N-1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
regression: exact thresholds['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']Failed
regression: scenario 6['heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']['heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']Failed
control: heat then locked cool['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']Passed
control: short heat cycle then hot['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']Passed
control: deadband squeeze['idle', 'cool', 'cool', 'idle', 'idle']['idle', 'cool', 'cool', 'idle', 'idle']Passed
regression: scenario 1['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']Passed
control: scenario 2['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']Passed
control: scenario 3['heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']['heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']Passed

SHA-256 / 5275c7da69c905bf71aeac0133124a4db95a59d626a99654a0031b216d749c04

3 / The verified repair

Exit 0
"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(heat_sp, cool_sp, temps):
    diff = 10
    if cool_sp - heat_sp < 30:
        cool_sp = heat_sp + 30
    state = 'idle'
    last_heat_end = None
    last_cool_end = None
    out = []
    for minute, temp in enumerate(temps):
        if state == 'heat' and temp >= heat_sp:
            state = 'idle'
            last_heat_end = minute
        elif state == 'cool' and temp <= cool_sp:
            state = 'idle'
            last_cool_end = minute
        if state == 'idle':
            if temp < heat_sp - diff and (last_cool_end is None or minute - last_cool_end >= 5):
                state = 'heat'
            elif temp > cool_sp + diff and (last_heat_end is None or minute - last_heat_end >= 5):
                state = 'cool'
        out.append(state)
    return out
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
fixtures = [[['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['regression: scenario 6',
   [680,
    720,
    [666, 650, 675, 664, 671, 693, 693, 695, 674, 675, 657, 635, 629, 632, 629, 632, 636, 632, 620]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['regression: scenario 1',
   [680, 690, [693, 687, 709, 709, 685, 704, 684, 680, 686, 687, 679]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: scenario 2',
   [680, 700, [681, 686, 698, 703, 724, 744, 752, 732, 755, 742, 762, 772]],
   ['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']],
  ['control: scenario 3',
   [680,
    740,
    [649,
     667,
     643,
     622,
     637,
     657,
     634,
     649,
     674,
     654,
     672,
     675,
     698,
     687,
     666,
     654,
     670,
     645,
     654,
     637,
     651,
     672]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']]],
 [['regression: scenario 2',
   [680, 710, [674, 681, 688, 699, 718, 698, 716, 709, 691, 672, 678, 701, 724, 699, 683, 698, 691, 701]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle']],
  ['regression: scenario 9',
   [680, 720, [674, 698, 687, 683, 659, 647, 658, 656, 658, 674, 664, 656, 672, 650]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 700, [687, 695, 710, 721, 712, 724, 736, 738, 737, 740, 748]],
   ['idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']],
  ['regression: scenario 3',
   [680, 690, [686, 679, 704, 696, 678, 658, 635, 648, 643, 657, 650, 661, 672, 691, 687, 685, 708, 687]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle']]],
 [['regression: scenario 5',
   [680, 740, [676, 673, 693, 690, 714, 709, 715, 740, 727, 735, 738, 752, 754, 729, 752, 746]],
   ['idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'idle',
    'cool',
    'cool']],
  ['regression: scenario 16',
   [680, 740, [684, 671, 648, 650, 634, 633, 624, 643, 667, 647, 630, 639]],
   ['idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 710, [758, 778, 769, 788, 780, 787, 790, 773, 789, 766, 785, 779, 790, 790, 790, 777, 790]],
   ['cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']],
  ['control: scenario 2',
   [680, 740, [646, 654, 672, 672, 692, 717, 736, 759, 769, 790, 790]],
   ['heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']]],
 [['regression: scenario 11',
   [680, 690, [661, 683, 677, 656, 633, 621, 620, 620, 620, 629, 635, 658, 682, 657, 659, 665]],
   ['heat',
    'idle',
    'idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle',
    'heat',
    'heat',
    'heat']],
  ['regression: scenario 12',
   [680, 710, [710, 715, 700, 681, 698, 674, 698, 701, 681, 675, 664]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['regression: scenario 1',
   [680,
    690,
    [671, 646, 636, 635, 620, 625, 635, 653, 659, 636, 645, 633, 620, 628, 620, 638, 653, 655, 666, 678]],
   ['idle',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat']],
  ['control: scenario 2',
   [680, 740, [655, 638, 663, 641, 622, 636, 628, 621, 621, 620, 635, 632, 644, 651, 656, 675, 687]],
   ['heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'heat',
    'idle']]],
 [['regression: scenario 9',
   [680, 690, [673, 694, 686, 686, 699, 720, 727, 736, 726, 719]],
   ['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool']],
  ['regression: scenario 34',
   [680,
    700,
    [735,
     716,
     727,
     705,
     684,
     664,
     652,
     673,
     670,
     688,
     692,
     705,
     701,
     720,
     743,
     746,
     741,
     746,
     763,
     788,
     790]],
   ['cool',
    'cool',
    'cool',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']],
  ['control: heat then locked cool',
   [680, 700, [650, 660, 690, 700, 712, 725, 725, 725, 715, 710]],
   ['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']],
  ['regression: exact thresholds',
   [680, 720, [670, 669, 680, 731, 730, 720, 721]],
   ['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']],
  ['control: short heat cycle then hot',
   [680, 760, [660, 685, 790, 790, 790, 790, 790, 790]],
   ['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']],
  ['control: deadband squeeze',
   [700, 705, [740, 741, 735, 730, 729]],
   ['idle', 'cool', 'cool', 'idle', 'idle']],
  ['control: scenario 1',
   [680, 720, [774, 779, 785, 790, 790, 781, 778, 777, 770, 767, 743, 730, 708, 690]],
   ['cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'idle',
    'idle']],
  ['regression: scenario 2',
   [680,
    710,
    [701,
     679,
     702,
     721,
     727,
     748,
     743,
     720,
     741,
     738,
     762,
     769,
     758,
     740,
     744,
     725,
     748,
     728,
     742,
     755,
     769]],
   ['idle',
    'idle',
    'idle',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool',
    'cool']]]]
for label, args, expected in fixtures[N-1]:
    check(label, solve(*args), expected)
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)
Boundary fixtureActualExpectedOutcome
regression: exact thresholds['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']['idle', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle']Passed
regression: scenario 6['heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']['heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']Passed
control: heat then locked cool['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']['heat', 'heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'idle']Passed
control: short heat cycle then hot['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']['heat', 'idle', 'idle', 'idle', 'idle', 'idle', 'cool', 'cool']Passed
control: deadband squeeze['idle', 'cool', 'cool', 'idle', 'idle']['idle', 'cool', 'cool', 'idle', 'idle']Passed
regression: scenario 1['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']['idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle', 'idle']Passed
control: scenario 2['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']['idle', 'idle', 'idle', 'idle', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool', 'cool']Passed
control: scenario 3['heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']['heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'idle', 'idle', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat', 'heat']Passed

SHA-256 / 916988fe422badcccd9cb0b4165300b97a2bbfd44ba59ce78b6368411e980a64

Verification & scope

A deterministic bounded teaching model of one thermostat or HVAC controller decision evaluated per sample. Temperatures are integer tenths of a degree Fahrenheit unless stated otherwise. The contract is a stipulated toy convention, not a claim of conformance to any manufacturer, ASHRAE guideline or code. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:51:44.945803+00:00.

Case digest / c73db85caca40c9f8d49b0cad1af079eeeee165783c209a24624a7cf50e5b916