vLLM 0.33.0 ships as an unauthenticated crash resurfaces

Unauthenticated EngineCore crash via out-of-range prompt token id under skip_tokenizer_init (incomplete fix of GHSA-25q3-v2hm-8vpf) vllm-project/vllm

A prompt token id outside the model's vocabulary still takes down EngineCore with no credentials required when skip_tokenizer_init is set, because the earlier GHSA-25q3-v2hm-8vpf fix did not close the path. If you run vLLM behind that flag with untrusted input, treat the server as crashable until a follow-up lands.

v0.33.0 vllm-project/vllm

by khluu

The release adds Reflection Beam-1.0, a 501B MoE with 23B active and its DFlash2 drafter, plus Yandex AliceAI-Foundation-80B-A3B, Recursive Flint (40B MoE, 2.5B active) and Muse Spark. Budget time for the upgrade: it is marked breaking, and the three open advisories predate it.

llama + CUDA: flash_attn_ext_rows (fix unified kv for multi-seq) ggml-org/llama.cpp

by am17an

The new flash_attn_ext_rows path lets a backend send an optimized mask representation instead of filtering other sequences' tokens out of the mask each step. That is the cost unified KV cache pays during multi-seq inference; the CUDA backend implements it first.

llama : fix on-device state copy sizes for quantized KV caches (#30541) ggml-org/llama.cpp

by Hongqiang Wang

On-device sequence state copies computed their element count as size divided by ggml_element_size(), which is wrong for a quantized type such as q8, so quantized KV caches copied the wrong number of bytes. Anyone saving or restoring sequence state on a quantized cache should be on this build.

Add freqs and partial_rotary_factor support to RoPE ml-explore/mlx

by shadyuwugurl

mlx.nn.RoPE now forwards a caller-supplied freqs array and partial_rotary_factor, so rotary encodings that are not a plain power law no longer need a Python reimplementation on top of MLX. Models with partial rotation, including multimodal mRoPE, can now use the fused kernel.

[Core][CUDA Graph] Separate prefill (token) and decode (request) CUDA graph capture sizes vllm-project/vllm

by Pb314314

Prefill and mixed batches are sized by tokens, uniform decode batches by requests, and a single capture size could not serve both well. vLLM now captures separate CUDA graph sizes for each, which changes the memory cost of your graph capture configuration.

[Responses] Normalize OpenResponses input items to match openai-python SDK requirements vllm-project/vllm

by anencore94

Incomplete assistant and reasoning history in Responses API requests, including empty assistant content, was losing turns or crashing. The fix normalizes input items through the existing validator to match what the openai-python SDK sends, which matters if you proxy Codex-style clients.

mlx: load bundled CUDA libraries in-process on Linux ollama/ollama

by dhiltgen

Follow-up to moving libcublas and libnvJitLink into cuda_v13: libcusolver searches only its own directory for dependencies, so the server picked up the host's CUDA libraries instead of the bundled ones. Alongside, Ollama's MLX runner now honors YaRN RoPE scaling in Qwen3 and Qwen3.5 and rejects GPUs its build has no architecture for at load time rather than later.