FA-74582 / Experiment statistics / Member archive

Bayesian conversion comparison: Posterior beta ignores observed successes · case 02

Posterior means are too low because every trial is counted as a failure.

Member previewVariant 2 · 3 implementations · 8 checks per implementation

Case contract

With an integer Beta(prior_a, prior_b) prior, arm posteriors are Beta(prior_a + s, prior_b + n - s). P(B > A) uses the exact integer-parameter sum over i < alpha_B of exp(lnB(alpha_A + i, beta_A + beta_B) - ln(beta_B + i) - lnB(1 + i, beta_B) - lnB(alpha_A, beta_A)). Return [posterior mean A, posterior mean B, P(B > A)] rounded to 6.

Why this case matters

Bayesian dashboards report a "chance to beat control" that product teams act on directly.

One recorded failure

Sample boundary fixture

This sample comes from the broken implementation of a controlled reproducer.

Boundary fixture	Actual	Expected	Outcome
informative prior shifts means	`[0.166667, 0.230769, 0.670807]`	`[0.166667, 0.25, 0.706767]`	Failed

MEMBER ARCHIVE

The complete case is available to members.

This record includes three runnable implementations, regression fixtures, execution results, and source hashes.

Member access is invitation-based. Sign in with your invited account to inspect the sources.