Models · Updated 21 Oct, 02:44 am IST
i14 trains and serves Laplace 3B linear-attention model on AWS Trainium2
Why it matters for readers: Explains a technical approach that helps models remember long context without growing memory for each token.
- Laplace 3B uses a linear attention (Gated Linear Attention) design so recurrent state stays at 29.5 MB per sequence, independent of context length.1
- i14 used native PyTorch support through TorchNeuron Native to run its custom architecture and Muon optimizer on AWS Trainium2.1
- On a single trn2.48xlarge instance, Laplace 3B trained at about 120,000 tokens per second.1
- The model served 11,544 output tokens per second across 1,280 concurrent sequences, improving concurrency and reducing KV cache growth compared to softmax attention.1
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get started