Research · Updated 15 Oct, 04:17 pm IST
Google speeds up video diffusion spatio‑temporal attention on TPUs
Why it matters for readers: It explains why attention is the bottleneck in video generation and how skipping low‑importance computations speeds things up.
- Video diffusion is slow because it uses many denoising steps and each step is expensive, with self‑attention becoming dominant at long sequence lengths.1
- For an 81‑frame 720p video, sequence lengths for current open models range from about 50,000 to 400,000, and attention can account for over half of per‑layer latency.1
- Scaling resolution (e.g., 720p to 1440p) can quadruple sequence length and push attention's share of latency from ~55% to ~88% due to quadratic scaling.1
- Because many query‑key interactions carry little attention mass, Google demonstrates using sparse attention masks to skip low‑importance computations and turn theoretical sparsity into real end‑to‑end TPU speedups.1
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get started