FAILURE MAP

Understand the failure.
Verify the repair.

Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.

Explore the cases ↓How results are verified ↗
100840Executable case variants
20168Distinct failure mechanisms
302520Executed implementations
20168Open-access cases

WHAT THE ARCHIVE CONTAINS

100840 executable cases. 20168 are open.

Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.

The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗

A RECORD OF WHAT WENT WRONG

Browse the archive / 100840

Python · Standard library
REFERENCEFAILURE MECHANISMDOMAINACCESS
FA-8501

Successful form control collection: Multiple selections are submitted as one nested value · case 01

Multiple selections are submitted as one nested value.

Form submission● Open access↗
FA-8502

Successful form control collection: Multiple selections are submitted as one nested value · case 02

Multiple selections are submitted as one nested value.

Form submission◈ Members↗
FA-8503

Successful form control collection: Multiple selections are submitted as one nested value · case 03

Multiple selections are submitted as one nested value.

Form submission◈ Members↗
FA-8504

Successful form control collection: Multiple selections are submitted as one nested value · case 04

Multiple selections are submitted as one nested value.

Form submission◈ Members↗
FA-8505

Successful form control collection: Multiple selections are submitted as one nested value · case 05

Multiple selections are submitted as one nested value.

Form submission◈ Members↗
FA-8506

Application form reset semantics: Reset restores a file input path that cannot be retained · case 01

Reset restores a file input path that cannot be retained.

Form lifecycle● Open access↗
FA-8507

Application form reset semantics: Reset restores a file input path that cannot be retained · case 02

Reset restores a file input path that cannot be retained.

Form lifecycle◈ Members↗
FA-8508

Application form reset semantics: Reset restores a file input path that cannot be retained · case 03

Reset restores a file input path that cannot be retained.

Form lifecycle◈ Members↗
FA-8509

Application form reset semantics: Reset restores a file input path that cannot be retained · case 04

Reset restores a file input path that cannot be retained.

Form lifecycle◈ Members↗
FA-8510

Application form reset semantics: Reset restores a file input path that cannot be retained · case 05

Reset restores a file input path that cannot be retained.

Form lifecycle◈ Members↗
FA-8511

Application form reset semantics: Reset clears file text but keeps selected file objects · case 01

Reset clears file text but keeps selected file objects.

Form lifecycle● Open access↗
FA-8512

Application form reset semantics: Reset clears file text but keeps selected file objects · case 02

Reset clears file text but keeps selected file objects.

Form lifecycle◈ Members↗
FA-8513

Application form reset semantics: Reset clears file text but keeps selected file objects · case 03

Reset clears file text but keeps selected file objects.

Form lifecycle◈ Members↗
FA-8514

Application form reset semantics: Reset clears file text but keeps selected file objects · case 04

Reset clears file text but keeps selected file objects.

Form lifecycle◈ Members↗
FA-8515

Application form reset semantics: Reset clears file text but keeps selected file objects · case 05

Reset clears file text but keeps selected file objects.

Form lifecycle◈ Members↗
FA-8516

Application form reset semantics: Reset unchecks choices whose default is checked · case 01

Reset unchecks choices whose default is checked.

Form lifecycle● Open access↗
FA-8517

Application form reset semantics: Reset unchecks choices whose default is checked · case 02

Reset unchecks choices whose default is checked.

Form lifecycle◈ Members↗
FA-8518

Application form reset semantics: Reset unchecks choices whose default is checked · case 03

Reset unchecks choices whose default is checked.

Form lifecycle◈ Members↗
FA-8519

Application form reset semantics: Reset unchecks choices whose default is checked · case 04

Reset unchecks choices whose default is checked.

Form lifecycle◈ Members↗
FA-8520

Application form reset semantics: Reset unchecks choices whose default is checked · case 05

Reset unchecks choices whose default is checked.

Form lifecycle◈ Members↗
FA-8521

Application form reset semantics: Reset discards default text values · case 01

Reset discards default text values.

Form lifecycle● Open access↗
FA-8522

Application form reset semantics: Reset discards default text values · case 02

Reset discards default text values.

Form lifecycle◈ Members↗
FA-8523

Application form reset semantics: Reset discards default text values · case 03

Reset discards default text values.

Form lifecycle◈ Members↗
FA-8524

Application form reset semantics: Reset discards default text values · case 04

Reset discards default text values.

Form lifecycle◈ Members↗
FA-8525

Application form reset semantics: Reset discards default text values · case 05

Reset discards default text values.

Form lifecycle◈ Members↗
FA-8526

Application form reset semantics: Reset leaves stale custom validation errors · case 01

Reset leaves stale custom validation errors.

Form lifecycle● Open access↗
FA-8527

Application form reset semantics: Reset leaves stale custom validation errors · case 02

Reset leaves stale custom validation errors.

Form lifecycle◈ Members↗
FA-8528

Application form reset semantics: Reset leaves stale custom validation errors · case 03

Reset leaves stale custom validation errors.

Form lifecycle◈ Members↗
FA-8529

Application form reset semantics: Reset leaves stale custom validation errors · case 04

Reset leaves stale custom validation errors.

Form lifecycle◈ Members↗
FA-8530

Application form reset semantics: Reset leaves stale custom validation errors · case 05

Reset leaves stale custom validation errors.

Form lifecycle◈ Members↗
FA-8531

Submission state transitions: Repeated submission cancels the visible pending state · case 01

Repeated submission cancels the visible pending state.

Form submission● Open access↗
FA-8532

Submission state transitions: Repeated submission cancels the visible pending state · case 02

Repeated submission cancels the visible pending state.

Form submission◈ Members↗
FA-8533

Submission state transitions: Repeated submission cancels the visible pending state · case 03

Repeated submission cancels the visible pending state.

Form submission◈ Members↗
FA-8534

Submission state transitions: Repeated submission cancels the visible pending state · case 04

Repeated submission cancels the visible pending state.

Form submission◈ Members↗
FA-8535

Submission state transitions: Repeated submission cancels the visible pending state · case 05

Repeated submission cancels the visible pending state.

Form submission◈ Members↗
FA-8536

Submission state transitions: Invalid forms enter the submitting state · case 01

Invalid forms enter the submitting state.

Form submission● Open access↗
FA-8537

Submission state transitions: Invalid forms enter the submitting state · case 02

Invalid forms enter the submitting state.

Form submission◈ Members↗
FA-8538

Submission state transitions: Invalid forms enter the submitting state · case 03

Invalid forms enter the submitting state.

Form submission◈ Members↗
FA-8539

Submission state transitions: Invalid forms enter the submitting state · case 04

Invalid forms enter the submitting state.

Form submission◈ Members↗
FA-8540

Submission state transitions: Invalid forms enter the submitting state · case 05

Invalid forms enter the submitting state.

Form submission◈ Members↗
FA-8541

Submission state transitions: Successful submission retains stale errors · case 01

Successful submission retains stale errors.

Form submission● Open access↗
FA-8542

Submission state transitions: Successful submission retains stale errors · case 02

Successful submission retains stale errors.

Form submission◈ Members↗
FA-8543

Submission state transitions: Successful submission retains stale errors · case 03

Successful submission retains stale errors.

Form submission◈ Members↗
FA-8544

Submission state transitions: Successful submission retains stale errors · case 04

Successful submission retains stale errors.

Form submission◈ Members↗
FA-8545

Submission state transitions: Successful submission retains stale errors · case 05

Successful submission retains stale errors.

Form submission◈ Members↗
FA-8546

Submission state transitions: Server failure details never replace old error messages · case 01

Server failure details never replace old error messages.

Form submission● Open access↗
FA-8547

Submission state transitions: Server failure details never replace old error messages · case 02

Server failure details never replace old error messages.

Form submission◈ Members↗
FA-8548

Submission state transitions: Server failure details never replace old error messages · case 03

Server failure details never replace old error messages.

Form submission◈ Members↗
FA-8549

Submission state transitions: Server failure details never replace old error messages · case 04

Server failure details never replace old error messages.

Form submission◈ Members↗
FA-8550

Submission state transitions: Server failure details never replace old error messages · case 05

Server failure details never replace old error messages.

Form submission◈ Members↗
FA-8551

Submission state transitions: Editing keeps a stale completed submission state · case 01

Editing keeps a stale completed submission state.

Form submission● Open access↗
FA-8552

Submission state transitions: Editing keeps a stale completed submission state · case 02

Editing keeps a stale completed submission state.

Form submission◈ Members↗
FA-8553

Submission state transitions: Editing keeps a stale completed submission state · case 03

Editing keeps a stale completed submission state.

Form submission◈ Members↗
FA-8554

Submission state transitions: Editing keeps a stale completed submission state · case 04

Editing keeps a stale completed submission state.

Form submission◈ Members↗
FA-8555

Submission state transitions: Editing keeps a stale completed submission state · case 05

Editing keeps a stale completed submission state.

Form submission◈ Members↗
FA-8556

Constraint validation priority: A required zero value is mistaken for missing input · case 01

A required zero value is mistaken for missing input.

Form validation● Open access↗
FA-8557

Constraint validation priority: A required zero value is mistaken for missing input · case 02

A required zero value is mistaken for missing input.

Form validation◈ Members↗
FA-8558

Constraint validation priority: A required zero value is mistaken for missing input · case 03

A required zero value is mistaken for missing input.

Form validation◈ Members↗
FA-8559

Constraint validation priority: A required zero value is mistaken for missing input · case 04

A required zero value is mistaken for missing input.

Form validation◈ Members↗
FA-8560

Constraint validation priority: A required zero value is mistaken for missing input · case 05

A required zero value is mistaken for missing input.

Form validation◈ Members↗
FA-8561

Constraint validation priority: Unparseable user input is reported valid · case 01

Unparseable user input is reported valid.

Form validation● Open access↗
FA-8562

Constraint validation priority: Unparseable user input is reported valid · case 02

Unparseable user input is reported valid.

Form validation◈ Members↗
FA-8563

Constraint validation priority: Unparseable user input is reported valid · case 03

Unparseable user input is reported valid.

Form validation◈ Members↗
FA-8564

Constraint validation priority: Unparseable user input is reported valid · case 04

Unparseable user input is reported valid.

Form validation◈ Members↗
FA-8565

Constraint validation priority: Unparseable user input is reported valid · case 05

Unparseable user input is reported valid.

Form validation◈ Members↗
FA-8566

Constraint validation priority: Optional blank fields are subjected to a pattern failure · case 01

Optional blank fields are subjected to a pattern failure.

Form validation● Open access↗
FA-8567

Constraint validation priority: Optional blank fields are subjected to a pattern failure · case 02

Optional blank fields are subjected to a pattern failure.

Form validation◈ Members↗
FA-8568

Constraint validation priority: Optional blank fields are subjected to a pattern failure · case 03

Optional blank fields are subjected to a pattern failure.

Form validation◈ Members↗
FA-8569

Constraint validation priority: Optional blank fields are subjected to a pattern failure · case 04

Optional blank fields are subjected to a pattern failure.

Form validation◈ Members↗
FA-8570

Constraint validation priority: Optional blank fields are subjected to a pattern failure · case 05

Optional blank fields are subjected to a pattern failure.

Form validation◈ Members↗
FA-8571

Constraint validation priority: A value equal to the inclusive minimum is rejected · case 01

A value equal to the inclusive minimum is rejected.

Form validation● Open access↗
FA-8572

Constraint validation priority: A value equal to the inclusive minimum is rejected · case 02

A value equal to the inclusive minimum is rejected.

Form validation◈ Members↗
FA-8573

Constraint validation priority: A value equal to the inclusive minimum is rejected · case 03

A value equal to the inclusive minimum is rejected.

Form validation◈ Members↗
FA-8574

Constraint validation priority: A value equal to the inclusive minimum is rejected · case 04

A value equal to the inclusive minimum is rejected.

Form validation◈ Members↗
FA-8575

Constraint validation priority: A value equal to the inclusive minimum is rejected · case 05

A value equal to the inclusive minimum is rejected.

Form validation◈ Members↗
FA-8576

Constraint validation priority: A value equal to the inclusive maximum is rejected · case 01

A value equal to the inclusive maximum is rejected.

Form validation● Open access↗
FA-8577

Constraint validation priority: A value equal to the inclusive maximum is rejected · case 02

A value equal to the inclusive maximum is rejected.

Form validation◈ Members↗
FA-8578

Constraint validation priority: A value equal to the inclusive maximum is rejected · case 03

A value equal to the inclusive maximum is rejected.

Form validation◈ Members↗
FA-8579

Constraint validation priority: A value equal to the inclusive maximum is rejected · case 04

A value equal to the inclusive maximum is rejected.

Form validation◈ Members↗
FA-8580

Constraint validation priority: A value equal to the inclusive maximum is rejected · case 05

A value equal to the inclusive maximum is rejected.

Form validation◈ Members↗
FA-8581

Tab selection and activation mode: Disabled tabs remain keyboard navigable · case 01

Disabled tabs remain keyboard navigable.

Tab interfaces● Open access↗
FA-8582

Tab selection and activation mode: Disabled tabs remain keyboard navigable · case 02

Disabled tabs remain keyboard navigable.

Tab interfaces◈ Members↗
FA-8583

Tab selection and activation mode: Disabled tabs remain keyboard navigable · case 03

Disabled tabs remain keyboard navigable.

Tab interfaces◈ Members↗
FA-8584

Tab selection and activation mode: Disabled tabs remain keyboard navigable · case 04

Disabled tabs remain keyboard navigable.

Tab interfaces◈ Members↗
FA-8585

Tab selection and activation mode: Disabled tabs remain keyboard navigable · case 05

Disabled tabs remain keyboard navigable.

Tab interfaces◈ Members↗
FA-8586

Tab selection and activation mode: Removing the focused tab selects the wrong fallback · case 01

Removing the focused tab selects the wrong fallback.

Tab interfaces● Open access↗
FA-8587

Tab selection and activation mode: Removing the focused tab selects the wrong fallback · case 02

Removing the focused tab selects the wrong fallback.

Tab interfaces◈ Members↗
FA-8588

Tab selection and activation mode: Removing the focused tab selects the wrong fallback · case 03

Removing the focused tab selects the wrong fallback.

Tab interfaces◈ Members↗
FA-8589

Tab selection and activation mode: Removing the focused tab selects the wrong fallback · case 04

Removing the focused tab selects the wrong fallback.

Tab interfaces◈ Members↗
FA-8590

Tab selection and activation mode: Removing the focused tab selects the wrong fallback · case 05

Removing the focused tab selects the wrong fallback.

Tab interfaces◈ Members↗
FA-8591

Tab selection and activation mode: Next-tab traversal runs backwards · case 01

Next-tab traversal runs backwards.

Tab interfaces● Open access↗
FA-8592

Tab selection and activation mode: Next-tab traversal runs backwards · case 02

Next-tab traversal runs backwards.

Tab interfaces◈ Members↗
FA-8593

Tab selection and activation mode: Next-tab traversal runs backwards · case 03

Next-tab traversal runs backwards.

Tab interfaces◈ Members↗
FA-8594

Tab selection and activation mode: Next-tab traversal runs backwards · case 04

Next-tab traversal runs backwards.

Tab interfaces◈ Members↗
FA-8595

Tab selection and activation mode: Next-tab traversal runs backwards · case 05

Next-tab traversal runs backwards.

Tab interfaces◈ Members↗
FA-8596

Tab selection and activation mode: Manual-activation tabs switch panels on arrow navigation · case 01

Manual-activation tabs switch panels on arrow navigation.

Tab interfaces● Open access↗
FA-8597

Tab selection and activation mode: Manual-activation tabs switch panels on arrow navigation · case 02

Manual-activation tabs switch panels on arrow navigation.

Tab interfaces◈ Members↗
FA-8598

Tab selection and activation mode: Manual-activation tabs switch panels on arrow navigation · case 03

Manual-activation tabs switch panels on arrow navigation.

Tab interfaces◈ Members↗
FA-8599

Tab selection and activation mode: Manual-activation tabs switch panels on arrow navigation · case 04

Manual-activation tabs switch panels on arrow navigation.

Tab interfaces◈ Members↗
FA-8600

Tab selection and activation mode: Manual-activation tabs switch panels on arrow navigation · case 05

Manual-activation tabs switch panels on arrow navigation.

Tab interfaces◈ Members↗

INSPECTABLE BY DESIGN

Every result has a runnable source.

Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.

Read the methodology ↗