Topic
self-attention
Latest news
- 17 OctPost explains how transformer layers act as memory stores
A Hacker News post presents an intuitive view of linear maps, MLPs and self-attention as different kinds of memory stores for large language models. It frames recall capacity using an interference-to-signal ratio and derives heuristics for how many memories each layer can hold before recall degrades.
Research · 1 source
Articles
No articles about self-attention yet.