Tagged continual-learning

Frozen Trunk, Adaptive Periphery: Continual Learning and Anytime Early Exit for GLM-5.2 (753B) via Portable Sufficient Statistics→

We instrumented GLM-5.2, a 753B-parameter open frontier MoE (MLA + sparse attention + IndexShare), at three levels: weight space (no forwards), per-token generation forensics (512 to 32k-token contexts), and a learned tap periphery. The measurements locate a late, scale-invariant decision front (median L76/78, invariant across a 64x context change) behind a motor band that weight anatomy independently predicts. Exploiting this, we attach per-layer linear translators trained closed-form from streaming Gram statistics; no gradients ever pass through the trunk. The same ~302 MB/tap statistical object is simultaneously the training set, the continual-learning memory, and the exit policy's substrate: streaming 49k tokens of a held-out language into the frontier model's Grams and re-solving lifts tap agreement 0.076 to 0.732 in 5.1 minutes (~$12) on a same-corpus holdout. This is continual readout adaptation, deliberately scoped: the trunk already carries the language, so the stream re-aims the periphery's transport rather than adding a capability. The -0.011 dip on code is not catastrophic forgetting, which pooled solves rule out by construction; it is mixture dilution, which compounds as streams accumulate unless reweighted. Live early-exit decoding with margin gates and MLA-cheap KV backfill reproduces the full model token-for-token on math over the full 80-token horizon (~82% exit rate, delta-NLL 0.000); ablating backfill collapses code from 40/40 to 10/40 matched tokens, putting measured causal weight on the skipped-layer KV problem CALM first named. On out-of-distribution input the margin gate exits confidently wrong. A ridge-leverage veto computed from the same Grams closes that hole live (wrong-exit rate 26% to 3-8%).