vLLM drops compile-based sequence parallelism, llama.cpp fixes slot checkpoints and a silent CUDA fallback

Cross-tenant DoS via uninitialized LogprobsTensors with prompt_logprobs vllm-project/vllm

vLLM disclosed a cross-tenant denial-of-service reachable through prompt_logprobs, where uninitialized LogprobsTensors are returned to a caller from another tenant's request. If you serve multi-tenant traffic with prompt logprobs enabled, treat this as the day's one thing to act on.

[Compile] Remove torch.compile-based sequence parallelism and AsyncTP vllm-project/vllm

by WoosukKwon

Woosuk Kwon deleted SequenceParallelismPass and AsyncTPPass, the two torch.compile-based passes, since the feature is unsupported in Model Runner V2 and the project is cutting its torch.compile reliance; manual model-level sequence parallelism under vllm/models/ is untouched. Any workload or config still depending on the compile-based SP and AsyncTP path loses it, and the PR supersedes the open...

server : preserve context checkpoints across slot save/restore (#26004) ggml-org/llama.cpp

by Leebr Data Consulting

The server now appends context checkpoints to the packed server_tokens payload and counts them in n_written/n_read, so a restored slot can roll back to a checkpoint instead of re-processing the entire prompt. The same change drops draft checkpoint data whose draft KV cache type does not match the draft context; restoring a slot saved with a different draft KV cache type previously aborted.

CUDA: fix CCCL version guard breaking on major version rollover (#29453) ggml-org/llama.cpp

by hey-gm

The CCCL guard tested major and minor independently, so on CCCL 4.x it evaluated as 4 >= 3 && 0 >= 1 and STRIDED_ITERATOR_AVAILABLE silently went undefined, dropping argsort back to the init_offsets path with nothing warning the user. The guard now tests the version as a whole; if you build CUDA from source against a rolled-over CCCL series, this is the fix.

Add 1-bit affine quantization support (Metal) ml-explore/mlx

by khosravipasha

MLX extended affine quantization from {2, 3, 4, 5, 6, 8} bits to include 1-bit, with full 1-bit affine dequantization and quantized matmul kernels on both CPU and Metal. It is the first sub-2-bit affine option in the framework.

server: hide duplicate and downgrade guards from list (#18874) ollama/ollama

by Daniel Hiltgen

Ollama stopped listing the duplicate and shadow v1 manifest tags it creates when lazily converting a legacy GGUF that needs the llama.cpp patch set, along with the retained old blobs held so a downgraded server cannot delete them. The conversion machinery still works; it is just no longer visible to list output and API consumers, and the legacy copies are slated for removal in a future release ...

[MemCache] Delete `ChunkCache` sgl-project/sglang

by hnyls2002

SGLang deleted chunk_cache.py, including ChunkCache, SWAChunkCache, and PureSWAChunkCache, after the routing changes below left nothing constructing them; disabled pure-SWA models now build PureSWARadixCache(disable=True) and no longer fall through to the lmcache arms. Scheduler-side, waiting requests under fcfs, lof, random, and routing_key now refresh their cached prefixes instead of sitting ...