Understand the failure.
Verify the repair.
Small, reproducible software failures. The broken implementation, the fix that didn’t work, and the one that passed—preserved together.
Explore the cases ↓How results are verified ↗WHAT THE ARCHIVE CONTAINS
100840 executable cases. 20168 are open.
Every case records the implementation that fails, the fix that did not work, and the repair that passed its checks—with recorded outputs and source hashes. This release adds 100840 cases across 20168 failure mechanisms and 254 domains.
The open tier gives you the failure and the unsuccessful fix for one case in every mechanism. The remaining 80672 cases, 5 variants per mechanism, are member-only: the verified repair, its recorded checks, and the full fixture suite are held in the member archive. Read the methodology ↗
A RECORD OF WHAT WENT WRONG
Browse the archive / 100840
Configuration change plan: Changed keys appear in the unchanged report · case 01
Changed keys appear in the unchanged report.
Configuration change plan: Changed keys appear in the unchanged report · case 02
Changed keys appear in the unchanged report.
Configuration change plan: Changed keys appear in the unchanged report · case 03
Changed keys appear in the unchanged report.
Configuration change plan: Changed keys appear in the unchanged report · case 04
Changed keys appear in the unchanged report.
Configuration change plan: Changed keys appear in the unchanged report · case 05
Changed keys appear in the unchanged report.
Command option assignment: Assignment preserves an obsolete default · case 01
Assignment preserves an obsolete default.
Command option assignment: Assignment preserves an obsolete default · case 02
Assignment preserves an obsolete default.
Command option assignment: Assignment preserves an obsolete default · case 03
Assignment preserves an obsolete default.
Command option assignment: Assignment preserves an obsolete default · case 04
Assignment preserves an obsolete default.
Command option assignment: Assignment preserves an obsolete default · case 05
Assignment preserves an obsolete default.
Command option assignment: Appending replaces earlier list elements · case 01
Appending replaces earlier list elements.
Command option assignment: Appending replaces earlier list elements · case 02
Appending replaces earlier list elements.
Command option assignment: Appending replaces earlier list elements · case 03
Appending replaces earlier list elements.
Command option assignment: Appending replaces earlier list elements · case 04
Appending replaces earlier list elements.
Command option assignment: Appending replaces earlier list elements · case 05
Appending replaces earlier list elements.
Command option assignment: Unset stores a null instead of removing the setting · case 01
Unset stores a null instead of removing the setting.
Command option assignment: Unset stores a null instead of removing the setting · case 02
Unset stores a null instead of removing the setting.
Command option assignment: Unset stores a null instead of removing the setting · case 03
Unset stores a null instead of removing the setting.
Command option assignment: Unset stores a null instead of removing the setting · case 04
Unset stores a null instead of removing the setting.
Command option assignment: Unset stores a null instead of removing the setting · case 05
Unset stores a null instead of removing the setting.
Command option assignment: Enable depends on an unused operand · case 01
Enable depends on an unused operand.
Command option assignment: Enable depends on an unused operand · case 02
Enable depends on an unused operand.
Command option assignment: Enable depends on an unused operand · case 03
Enable depends on an unused operand.
Command option assignment: Enable depends on an unused operand · case 04
Enable depends on an unused operand.
Command option assignment: Enable depends on an unused operand · case 05
Enable depends on an unused operand.
Command option assignment: Disable deletes the option instead of setting false · case 01
Disable deletes the option instead of setting false.
Command option assignment: Disable deletes the option instead of setting false · case 02
Disable deletes the option instead of setting false.
Command option assignment: Disable deletes the option instead of setting false · case 03
Disable deletes the option instead of setting false.
Command option assignment: Disable deletes the option instead of setting false · case 04
Disable deletes the option instead of setting false.
Command option assignment: Disable deletes the option instead of setting false · case 05
Disable deletes the option instead of setting false.
Deprecated option alias migration: Alias migration erases unrelated settings · case 01
Alias migration erases unrelated settings.
Deprecated option alias migration: Alias migration erases unrelated settings · case 02
Alias migration erases unrelated settings.
Deprecated option alias migration: Alias migration erases unrelated settings · case 03
Alias migration erases unrelated settings.
Deprecated option alias migration: Alias migration erases unrelated settings · case 04
Alias migration erases unrelated settings.
Deprecated option alias migration: Alias migration erases unrelated settings · case 05
Alias migration erases unrelated settings.
Deprecated option alias migration: Conflicting aliases do not emit diagnostics · case 01
Conflicting aliases do not emit diagnostics.
Deprecated option alias migration: Conflicting aliases do not emit diagnostics · case 02
Conflicting aliases do not emit diagnostics.
Deprecated option alias migration: Conflicting aliases do not emit diagnostics · case 03
Conflicting aliases do not emit diagnostics.
Deprecated option alias migration: Conflicting aliases do not emit diagnostics · case 04
Conflicting aliases do not emit diagnostics.
Deprecated option alias migration: Conflicting aliases do not emit diagnostics · case 05
Conflicting aliases do not emit diagnostics.
Deprecated option alias migration: Deprecated aliases override canonical settings · case 01
Deprecated aliases override canonical settings.
Deprecated option alias migration: Deprecated aliases override canonical settings · case 02
Deprecated aliases override canonical settings.
Deprecated option alias migration: Deprecated aliases override canonical settings · case 03
Deprecated aliases override canonical settings.
Deprecated option alias migration: Deprecated aliases override canonical settings · case 04
Deprecated aliases override canonical settings.
Deprecated option alias migration: Deprecated aliases override canonical settings · case 05
Deprecated aliases override canonical settings.
Deprecated option alias migration: Deprecated keys remain in the migrated configuration · case 01
Deprecated keys remain in the migrated configuration.
Deprecated option alias migration: Deprecated keys remain in the migrated configuration · case 02
Deprecated keys remain in the migrated configuration.
Deprecated option alias migration: Deprecated keys remain in the migrated configuration · case 03
Deprecated keys remain in the migrated configuration.
Deprecated option alias migration: Deprecated keys remain in the migrated configuration · case 04
Deprecated keys remain in the migrated configuration.
Deprecated option alias migration: Deprecated keys remain in the migrated configuration · case 05
Deprecated keys remain in the migrated configuration.
Deprecated option alias migration: Multiple aliases repeat the same conflict · case 01
Multiple aliases repeat the same conflict.
Deprecated option alias migration: Multiple aliases repeat the same conflict · case 02
Multiple aliases repeat the same conflict.
Deprecated option alias migration: Multiple aliases repeat the same conflict · case 03
Multiple aliases repeat the same conflict.
Deprecated option alias migration: Multiple aliases repeat the same conflict · case 04
Multiple aliases repeat the same conflict.
Deprecated option alias migration: Multiple aliases repeat the same conflict · case 05
Multiple aliases repeat the same conflict.
Configuration provenance tracking: Deleted settings remain visible · case 01
Deleted settings remain visible.
Configuration provenance tracking: Deleted settings remain visible · case 02
Deleted settings remain visible.
Configuration provenance tracking: Deleted settings remain visible · case 03
Deleted settings remain visible.
Configuration provenance tracking: Deleted settings remain visible · case 04
Deleted settings remain visible.
Configuration provenance tracking: Deleted settings remain visible · case 05
Deleted settings remain visible.
Configuration provenance tracking: Deleted settings retain stale provenance · case 01
Deleted settings retain stale provenance.
Configuration provenance tracking: Deleted settings retain stale provenance · case 02
Deleted settings retain stale provenance.
Configuration provenance tracking: Deleted settings retain stale provenance · case 03
Deleted settings retain stale provenance.
Configuration provenance tracking: Deleted settings retain stale provenance · case 04
Deleted settings retain stale provenance.
Configuration provenance tracking: Deleted settings retain stale provenance · case 05
Deleted settings retain stale provenance.
Configuration provenance tracking: Later values retain their previous assignment · case 01
Later values retain their previous assignment.
Configuration provenance tracking: Later values retain their previous assignment · case 02
Later values retain their previous assignment.
Configuration provenance tracking: Later values retain their previous assignment · case 03
Later values retain their previous assignment.
Configuration provenance tracking: Later values retain their previous assignment · case 04
Later values retain their previous assignment.
Configuration provenance tracking: Later values retain their previous assignment · case 05
Later values retain their previous assignment.
Configuration provenance tracking: Override provenance points to defaults · case 01
Override provenance points to defaults.
Configuration provenance tracking: Override provenance points to defaults · case 02
Override provenance points to defaults.
Configuration provenance tracking: Override provenance points to defaults · case 03
Override provenance points to defaults.
Configuration provenance tracking: Override provenance points to defaults · case 04
Override provenance points to defaults.
Configuration provenance tracking: Override provenance points to defaults · case 05
Override provenance points to defaults.
Configuration provenance tracking: False values erase settings and provenance · case 01
False values erase settings and provenance.
Configuration provenance tracking: False values erase settings and provenance · case 02
False values erase settings and provenance.
Configuration provenance tracking: False values erase settings and provenance · case 03
False values erase settings and provenance.
Configuration provenance tracking: False values erase settings and provenance · case 04
False values erase settings and provenance.
Configuration provenance tracking: False values erase settings and provenance · case 05
False values erase settings and provenance.
Access policy decision combination: Disabled deny rules remain effective · case 01
Disabled deny rules remain effective.
Access policy decision combination: Disabled deny rules remain effective · case 02
Disabled deny rules remain effective.
Access policy decision combination: Disabled deny rules remain effective · case 03
Disabled deny rules remain effective.
Access policy decision combination: Disabled deny rules remain effective · case 04
Disabled deny rules remain effective.
Access policy decision combination: Disabled deny rules remain effective · case 05
Disabled deny rules remain effective.
Access policy decision combination: Wildcard policies do not match resources · case 01
Wildcard policies do not match resources.
Access policy decision combination: Wildcard policies do not match resources · case 02
Wildcard policies do not match resources.
Access policy decision combination: Wildcard policies do not match resources · case 03
Wildcard policies do not match resources.
Access policy decision combination: Wildcard policies do not match resources · case 04
Wildcard policies do not match resources.
Access policy decision combination: Wildcard policies do not match resources · case 05
Wildcard policies do not match resources.
Access policy decision combination: An explicit deny is overridden by allows · case 01
An explicit deny is overridden by allows.
Access policy decision combination: An explicit deny is overridden by allows · case 02
An explicit deny is overridden by allows.
Access policy decision combination: An explicit deny is overridden by allows · case 03
An explicit deny is overridden by allows.
Access policy decision combination: An explicit deny is overridden by allows · case 04
An explicit deny is overridden by allows.
Access policy decision combination: An explicit deny is overridden by allows · case 05
An explicit deny is overridden by allows.
Access policy decision combination: Unknown effects grant access · case 01
Unknown effects grant access.
Access policy decision combination: Unknown effects grant access · case 02
Unknown effects grant access.
Access policy decision combination: Unknown effects grant access · case 03
Unknown effects grant access.
Access policy decision combination: Unknown effects grant access · case 04
Unknown effects grant access.
Access policy decision combination: Unknown effects grant access · case 05
Unknown effects grant access.
INSPECTABLE BY DESIGN
Every result has a runnable source.
Runnable implementations with recorded outputs, source hashes, and explicit contracts. Related variants share a failure mechanism and belong together in evaluation splits.
Read the methodology ↗