$ cat local-llm/week/2026-09-07.log
the week in review · Sep 7 – Sep 13, 2026
vLLM EngineCore can be killed by crafted stop_token_ids
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
Two vLLM reports this week show out-of-range stop_token_ids with min_tokens terminating EngineCore, including via the Rust HTTP/gRPC path.
Out-of-range `stop_token_ids` with `min_tokens` can kill vLLM EngineCore vllm-project/vllm
An out-of-range stop_token_ids combined with min_tokens can kill EngineCore, so validate token ids against the model vocab before passing them in. No fixed version is stated in the report.
Rust HTTP/gRPC stop_token_ids bypass Python vocab-bound fix and can terminate EngineCore vllm-project/vllm
The Rust HTTP/gRPC path bypasses the Python vocab-bound fix and terminates EngineCore, which means Python-side validation alone will not protect a server fronted by the Rust HTTP/gRPC layer.
v0.29.0 vllm-project/vllm
Model Runner V2 is now the default for all models, completing the rollout that began with pooling models, and MRV2 also gained CUDA graph memory profiling for KV cache auto-sizing. The release covers 594 commits from 277 contributors, 91 of them new.
[Model] Support for Spark2_5ForCausalLM implementation ggml-org/llama.cpp
llama.cpp gained end-to-end Spark2_5 support: GGUF conversion, architecture registration and tensor mappings, tokenizer pre-tokenizer, and model loading and inference graphs. New architecture support lands in convert and inference together, so you can convert and run Spark2_5 checkpoints with the same toolchain.
[Scheduler] Make request-timeout aborts rank-consistent to fix TP collective hangs sgl-project/sglang
SGLang changed request-timeout aborts to be rank-consistent, fixing TP collective hangs. The previous wall-clock scan ran independently on every rank, so sub-millisecond skew could abort a request on whichever rank crossed the deadline first, leaving the waiting queue inconsistent across ranks.
Revert "[kernel] add fused silu mul quant fp8" sgl-project/sglang
SGLang reverted the fused silu mul quant fp8 kernel on the block-wise FP8 MoE path because it is not numerically equivalent to the two-kernel path it replaces, and the difference cost speculative-decoding acceptance on GLM-5.2-FP8 EAGLE MTP. If you are on that path, the previous numerics are back.
Disaggregated generate skips decoder prompt-length validation for some multimodal processors vllm-project/vllm
Disaggregated generate skips decoder prompt-length validation for some multimodal processors, so requests that should be rejected can reach the decoder. Validate prompt length ahead of the call on the affected processors until this is fixed.
$ ls local-llm/week/ # the briefings behind this review
Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.