Model Runner V2

Latest news

1. 6 Oct
   vLLM releases v0.31.0 with performance and restart improvements

   vLLM published version 0.31.0 featuring many performance optimizations, MoE and KV-cache improvements, and fast-restart tooling. The release also adds Model Runner V2 features like speculative decoding and custom logits processors.

   Tools · 1 source

Articles

No articles about Model Runner V2 yet.