Hardware · Updated 25 Oct, 07:03 am IST
Cerebras shows how wafer-scale chips run multi‑trillion parameter models
Why it matters for readers: Offers a clear example of how very large AI models can be split across many chips to run efficiently.
- A single Cerebras WSE-3T wafer has 44 GB of on‑chip memory, which the post uses as the basis for sharding models across wafers.1
- Cerebras illustrates deploying a ~3 trillion‑parameter open source model (Kimi K3) by splitting its 93 layers across multiple wafers.1
- The example assumes 4‑bit weight quantization, an 8‑bit KV cache, and a 256k average request size to compute memory per layer and wafer requirements.1
- Using these assumptions, the post shows each layer needs on the order of ~16 GB, and describes mapping dense and attention layers differently because dense layers require extra KV cache space.1
- Cerebras says this approach can support multi‑trillion parameter models at throughput in the 1000–2000+ tokens per second range.1
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get started