Thermodynamic Routing of Stale Gradients via Non-Associative Information Geometry
| Paper: 218 | Portfolio C (AI) — AI & Deep Learning |
Abstract
Introduces Non-Associative Information Geometry (NAIG) Routing for distributed LLM training. By mapping gradient drift into the 14-dimensional Lie algebra $\mathfrak{g}_2$, NAIG evaluates staleness not by Euclidean magnitude but by topological contradiction against a ground-truth reference, via the rank-4 Fano-Fisher metric. The MGE thermodynamically freezes out Fano-incompatible gradients while executing Topological Rescue on geometrically coherent stale updates. NAIG operates as a pure topological control layer over standard Euclidean SGD, requiring no modification to the optimizer or hardware. Demonstrated on GPT-2 (124M) with a 35,000× dimensional compression of the routing signal.
Key Results
- Experiment A (Gram-Schmidt Audit): NAIG detects topology invisible to cosine similarity — workers with identical cosine distance are correctly separated by Fano compatibility.
- Experiment B (GPT-2 cluster): NAIG achieves −31% final loss vs. Hogwild! with an effective routing dimensionality of 4 (not 5M).
- Auto-annealing: No schedule required — G₂ geometry spontaneously freezes contradictions during exploration and thaws at convergence.
Zenodo
Related Papers
- Paper 221 — Non-Associative Information Geometry: Fano-Fisher Decomposition Theorem (proves the rank-4 result used here)
- Paper 201 — The Maslov-Gibbs Einsum (MGE) (the thermodynamic routing operator)
- Paper 202 — Topological Resonance Synthesis (TRS)