VERIFICATION / RELEASE 2026.09.3

Results you can
run yourself.

The archive pairs software failure mechanisms with runnable implementations, regression fixtures, and recorded results.

What is in a case

Each case includes an implementation that fails its checks, an attempted repair that still fails at least one check, and a corrected implementation that passes every recorded check. The source uses only the Python standard library.

How results are verified

Each of the 293520 implementations was executed in a separate Python process. Case records preserve stdout, stderr, exit status, elapsed time, boundary observations, and SHA-256 source hashes. The recorded inputs, expected values, and actual values make each result inspectable.

Working with case families

The 97840 variants cover 19568 distinct mechanisms. Five numbered cases in each family share a contract. Parameterized fixtures vary where the model supports them; fixed boundary fixtures can repeat across cases. Keep each family and every shared evaluation_group together when designing an evaluation split, and report both mechanism coverage and variant counts. Related faults may share a corrected model; a different fault does not imply an independent benchmark task.

Scope of the results

Each case defines behavior under its stated contract and fixtures. Passing results establish that behavior for the recorded checks; broader inputs and integration conditions need their own validation. Execution times describe the recorded run and are not performance benchmarks.

Access & reuse

One complete case per family is open access. The remaining cases require member access. Released case sources and observations use CC0-1.0. The catalog API provides metadata, and public cases include source bundles.

Reproduce a result

unzip FA-001.zip
python3 broken.py   # expected exit: 1
python3 attempt.py  # expected exit: 1
python3 fixed.py    # expected exit: 0