Chips & infra updates · Official

• vLLM 0.32.0 adds a structured-decision endpoint, NVFP4 KV cache on more GPUs and makes Model Runner V2 the ROCm default

  vLLM v0.32.0 bundles 814 commits from 363 contributors. It adds a POST /v1/systemone endpoint that answers structured choice and score questions, a generic hf tool and reasoning parser that reads a checkpoint's response_template, and NVFP4 KV cache support on SM8x, SM90 and SM12x GPUs. Model Runner V2 gains a GPU n-gram speculator and becomes the default for all models on ROCm, and new model support includes EmbeddingGemma 2, MiniCPM-V 4.7 and Nemotron 3.5 ASR. The release also fixes three security advisories, restricts default CORS origins and drops Python 3.10.

  OfficialOct 18Source: vLLM (GitHub)Part of vLLM (GitHub)

RSS for this view → · By the numbers