FA-74627 / Experiment statistics / Member archive

Guardrailed ship decision: Non-inferiority margin has the wrong sign · case 02

Higher-is-better guardrails demand a gain larger than the margin.

Member previewVariant 2 · 3 implementations · 8 checks per implementation

Case contract

primary is [lift, ci_low, ci_high]. Each guardrail [name, ci_low, ci_high, margin, direction] passes non-inferiority when higher_is_better and ci_low > -margin, or lower_is_better and ci_high < margin. Decision: no_ship if primary ci_high < 0; else if primary ci_low > 0 then "blocked:<first failing guardrail in input order>" or ship; else inconclusive. Return [decision, failing names in input order].

Why this case matters

Launch reviews combine a primary win with guardrail checks; the combination rule must be exact.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
non-inferiority allows a small loss	`["blocked:revenue", ["revenue"]]`	`["ship", []]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.