FA-99291 / Distributed training aggregation / Open access

Sparse gradient coordinates are combined by list position · case 01

Sparse gradient coordinates are combined by list position.

Verified by executionVariant 1 · 6 checks per implementationDownload source bundle ↓JSON ↗

ROOT CAUSE

Workers may emit sparse coordinates in different orders, but the reducer uses each pair’s position as its parameter ID.

THE FAILURE

Workers may emit sparse coordinates in different orders, but the reducer uses each pair’s position as its parameter ID.

Unsuccessful approach: Replacing positional IDs with explicit IDs but overwriting duplicate coordinates discards another worker’s contribution.

Case contract

Rows [worker, [(coordinate,value),...]]; sum values by coordinate and return sorted pairs; empty coordinates omitted.

Why this case matters

A toy distributed-training contract makes replica and shard contribution rules explicit without claiming framework compatibility or performance.

1 / The failure

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(rows):
    return [[j,sum(vals[j][1] for w,vals in rows if j<len(vals))] for j in range(max((len(vals) for w,vals in rows),default=0))]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('sparse order differs',solve([['a',[[2,N],[0,1]]],['b',[[0,3],[2,4]]]]),[[0,4],[2,N+4]])
check('empty workers',solve([]),[])
check('empty coordinate rows',solve([['a',[]]]),[])
check('one coordinate',solve([['a',[[N,N]]]]),[[N,N]])
check('negative coordinate values',solve([['a',[[-1,N]]],['b',[[-1,-N]]]]),[[-1,0]])
check('missing coordinate treated zero',solve([['a',[[1,2]]],['b',[[3,4]]]]),[[1,2],[3,4]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
sparse order differs	`[[0, 4], [1, 5]]`	`[[0, 4], [2, 5]]`	Failed
empty workers	`[]`	`[]`	Passed
empty coordinate rows	`[]`	`[]`	Passed
one coordinate	`[[0, 1]]`	`[[1, 1]]`	Failed
negative coordinate values	`[[0, 0]]`	`[[-1, 0]]`	Failed
missing coordinate treated zero	`[[0, 6]]`	`[[1, 2], [3, 4]]`	Failed

SHA-256 / 44809c03c23b597ec94ca7daa9ab85d5f7d5bc97fca360277b234bcf64574ba3

2 / The unsuccessful fix

Exit 1

"""Failure Map reference implementation. Python standard library only."""
import json

N = 1
observations = []
def solve(rows):
    d={}
    for w,vals in rows:
        for j,v in vals: d[j]=v
    return [[j,d[j]] for j in sorted(d)]
def check(label, actual, expected):
    observations.append({"check": label, "actual": actual, "expected": expected, "passed": actual == expected})
check('sparse order differs',solve([['a',[[2,N],[0,1]]],['b',[[0,3],[2,4]]]]),[[0,4],[2,N+4]])
check('empty workers',solve([]),[])
check('empty coordinate rows',solve([['a',[]]]),[])
check('one coordinate',solve([['a',[[N,N]]]]),[[N,N]])
check('negative coordinate values',solve([['a',[[-1,N]]],['b',[[-1,-N]]]]),[[-1,0]])
check('missing coordinate treated zero',solve([['a',[[1,2]]],['b',[[3,4]]]]),[[1,2],[3,4]])
print(json.dumps({"observations": observations, "passed": all(x["passed"] for x in observations)}, ensure_ascii=False))
raise SystemExit(0 if all(x["passed"] for x in observations) else 1)

Boundary fixture	Actual	Expected	Outcome
sparse order differs	`[[0, 3], [2, 4]]`	`[[0, 4], [2, 5]]`	Failed
empty workers	`[]`	`[]`	Passed
empty coordinate rows	`[]`	`[]`	Passed
one coordinate	`[[1, 1]]`	`[[1, 1]]`	Passed
negative coordinate values	`[[-1, -1]]`	`[[-1, 0]]`	Failed
missing coordinate treated zero	`[[1, 2], [3, 4]]`	`[[1, 2], [3, 4]]`	Passed

SHA-256 / 933b123046997c2ca34a77ace0d03b6f06d72bb4c893f57340464bd66769749f

HELD IN THE MEMBER ARCHIVE

The verified repair and its recorded checks are member-only.

This mechanism has 6 recorded checks per implementation. The open-access tier publishes the failure and the unsuccessful fix; the repaired source that passes every check, and the observations that prove it, are available to members.

Every case sharing this mechanism uses the same contract and the same repair, so this one record is held back for all of them.

Member access is invitation-based. Sign in with your invited account to inspect the repair.

Verification & scope

Inputs are already materialized Python values from a fixed toy collective; no transport, failure recovery, optimizer state, or real model behavior is represented. This reproducer isolates one failure mechanism. Results cover the supplied fixtures. Variants within a family share a test contract and should remain grouped when constructing evaluation splits. Related mechanisms with a shared evaluation_group must also remain together; these controlled models are not independent production incidents.

Observations recorded using Python 3.12.14 at 2026-09-30T16:48:22.633383+00:00.

Case digest / 2ecf2459731ad253493f23733ec9a70dfe0121d921d9f943c5f288da93ddcb4f