FA-98886 / Gradient accumulation weighting / Open access

A parameter absent from a microbatch is counted as a zero update · case 01

A parameter absent from a microbatch is counted as a zero update.

Verified by executionVariant 1 · 7 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

The broken divisor sums activity counters rather than counting microbatches; absent parameters contribute zero in the contract.

THE FAILURE

The broken divisor sums activity counters rather than counting microbatches; absent parameters contribute zero in the contract.

Unsuccessful approach: Dividing only by active participation omits the zero contribution of absent parameters.

Case contract

Rows are microbatch coordinate sums; None means a parameter was absent and contributes zero to the mean over all microbatches. active_counts documents participation but is not a divisor.

Why this case matters

A deterministic toy model isolates one local accumulation contract.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(rows, active_counts):
    return [sum((row[j] or 0) for row in rows) / max(1,sum(active_counts)) for j in range(len(rows[0]))] if rows else []
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('missing treated as zero',solve([[2*N,None],[4*N,6*N]],[1,2]),[3*N,3*N])
check('all present',solve([[2*N,4*N],[6*N,8*N]],[2,2]),[4*N,6*N])
check('single missing',solve([[None,5*N]],[0,1]),[0,5*N])
check('empty rows',solve([],[]),[])
check('zero active count',solve([[0,None],[0,None]],[0,0]),[0,0])
check('late participation',solve([[None,2*N],[9*N,None],[3*N,4*N]],[1,1,2]),[4*N,2*N])
check('zero gradient',solve([[0,None],[4*N,2*N]],[1,1]),[2*N,N])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
missing treated as zero	`[2.0, 2.0]`	`[3, 3]`	Failed
all present	`[2.0, 3.0]`	`[4, 6]`	Failed
single missing	`[0.0, 5.0]`	`[0, 5]`	Passed
empty rows	`[]`	`[]`	Passed
zero active count	`[0.0, 0.0]`	`[0, 0]`	Passed
late participation	`[3.0, 1.5]`	`[4, 2]`	Failed
zero gradient	`[2.0, 1.0]`	`[2, 1]`	Passed

SHA-256 / 0d02e4e185a14d0abe6a78c97453c415215147b7e1e137b771b44d785053713e

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(rows, active_counts):
    return [sum(row[j] for row in rows if row[j] is not None) / max(1,sum(row[j] is not None for row in rows)) for j in range(len(rows[0]))] if rows else []
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('missing treated as zero',solve([[2*N,None],[4*N,6*N]],[1,2]),[3*N,3*N])
check('all present',solve([[2*N,4*N],[6*N,8*N]],[2,2]),[4*N,6*N])
check('single missing',solve([[None,5*N]],[0,1]),[0,5*N])
check('empty rows',solve([],[]),[])
check('zero active count',solve([[0,None],[0,None]],[0,0]),[0,0])
check('late participation',solve([[None,2*N],[9*N,None],[3*N,4*N]],[1,1,2]),[4*N,2*N])
check('zero gradient',solve([[0,None],[4*N,2*N]],[1,1]),[2*N,N])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
missing treated as zero	`[3.0, 6.0]`	`[3, 3]`	Failed
all present	`[4.0, 6.0]`	`[4, 6]`	Passed
single missing	`[0.0, 5.0]`	`[0, 5]`	Passed
empty rows	`[]`	`[]`	Passed
zero active count	`[0.0, 0.0]`	`[0, 0]`	Passed
late participation	`[6.0, 3.0]`	`[4, 2]`	Failed
zero gradient	`[2.0, 2.0]`	`[2, 1]`	Failed

SHA-256 / 1a9d2715a2c4215a4d3f225539e3f46a8c83e2919ffaf5aa9fed89d781f34312

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 7 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Verification & scope

Inputs are scalar or short-list summaries and do not model autodiff, devices, or an optimizer. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-30T16:48:18.065493+00:00.

Case digest / 50867ada14df70af7b437561368252e3fac7073de9b016a339466a3ed562e47d