Topic
Adam
Latest news
- 6 OctOptimiser history causes loss of answer probability during fine‑tuning, study finds
Researchers show that optimiser memory (momentum) can reduce probability mass assigned to previously learned answers during fine‑tuning even when current gradients oppose that loss. They find older stored gradients drive leakage of answer mass while recent gradients protect old answers, and interventions that reset momentum can recover retention.
Research · 1 source
Articles
No articles about Adam yet.