FA-6281 / Statistics / Open access

Binary f one score · case 01

Intersection-over-union replaces the F1 overlap normalization.

Verified by executionVariant 1 · 4 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Intersection-over-union replaces the F1 overlap normalization.

THE FAILURE

Intersection-over-union replaces the F1 overlap normalization.

Unsuccessful approach: The arithmetic mean of precision and recall differs from their harmonic mean.

Case contract

Integer finite observations and equal lengths for paired samples. Counts and weights are nonnegative. Rational results use reduced Fraction strings. Empty or undefined statistics return None where shown. tp,fp,fn are nonnegative counts; zero total returns None. Exact operational definition: str(Fraction(2*tp,2*tp+fp+fn)) if 2*tp+fp+fn else None

Why this case matters

Small exact fixtures expose this error without platform timing, external services, or probabilistic observations. Statistical estimators results depend on the stated convention.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import math
import calendar
import statistics
import itertools
from fractions import Fraction
from datetime import date, datetime, timedelta, timezone
from decimal import Decimal, ROUND_HALF_UP, ROUND_DOWN, ROUND_CEILING, ROUND_FLOOR

N = 1
observations = []
def solve(tp, fp, fn):
    return str(Fraction(tp,tp+fp+fn)) if tp+fp+fn else None
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('fixture 1: (3, 1, 2)', solve(*(3, 1, 2)), '2/3')
check('fixture 2: (0, 2, 1)', solve(*(0, 2, 1)), '0')
check('fixture 3: (0, 0, 0)', solve(*(0, 0, 0)), None)
check('fixture 4: (5, 0, 0)', solve(*(5, 0, 0)), '1')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
fixture 1: (3, 1, 2)	`1/2`	`2/3`	Failed
fixture 2: (0, 2, 1)	`0`	`0`	Passed
fixture 3: (0, 0, 0)	`None`	`None`	Passed
fixture 4: (5, 0, 0)	`1`	`1`	Passed

SHA-256 / 39f9e525f86a70363da8524a99ca7383125721ab98c57024d0f6350aa9951989

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json
import math
import calendar
import statistics
import itertools
from fractions import Fraction
from datetime import date, datetime, timedelta, timezone
from decimal import Decimal, ROUND_HALF_UP, ROUND_DOWN, ROUND_CEILING, ROUND_FLOOR

N = 1
observations = []
def solve(tp, fp, fn):
    return str((Fraction(tp,tp+fp)+Fraction(tp,tp+fn))/2) if tp+fp and tp+fn else None
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('fixture 1: (3, 1, 2)', solve(*(3, 1, 2)), '2/3')
check('fixture 2: (0, 2, 1)', solve(*(0, 2, 1)), '0')
check('fixture 3: (0, 0, 0)', solve(*(0, 0, 0)), None)
check('fixture 4: (5, 0, 0)', solve(*(5, 0, 0)), '1')
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
fixture 1: (3, 1, 2)	`27/40`	`2/3`	Failed
fixture 2: (0, 2, 1)	`0`	`0`	Passed
fixture 3: (0, 0, 0)	`None`	`None`	Passed
fixture 4: (5, 0, 0)	`1`	`1`	Passed

SHA-256 / a7fa7277bb8bffe95993ca29bbd4d67180b791cc43cd797f7035d4c1b249d4a8

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 4 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Verification & scope

This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-29T14:37:59.652836+00:00.

Case digest / a1d29c5b8d081333749812127a6260e3a99fe75cca6626a031ab0176adb0bdc1