$ cat local-llm/week/2026-10-12.log
the week in review · Oct 12 – Oct 18, 2026
vLLM v0.32.0 lands systemone, security fixes behind it
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
llama.cpp's v0.7.0 shipped a host-side MoE expert cache and speculative decoding, while vLLM closed a ZMQ socket exhaustion hole and tightened CORS.
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) vllm-project/vllm
Unbounded peer sessions could exhaust P2P ZMQ sockets, and the fix is flagged breaking, so anything running P2P on vLLM should move to v0.32.0 rather than a patch line. The same release restricts default CORS origins, meaning permissive defaults your frontend may rely on are no longer the default.
v0.7.0 ggml-org/llama.cpp
The release adds a GPU cache for MoE experts held in host memory via `--moe-cache-mib`, which the release notes describe as up to ~2.2x decode speedup when experts are offloaded, plus LiliCorr speculative decoding and MTP for GLM5-Next. It also carries a batch of new model architectures, including K2 Horizon in dense and MoVA forms. The version is marked breaking.
v0.32.0 vllm-project/vllm
The new `POST /v1/systemone` endpoint answers `choice`, `noul` and `score` questions over a structured state and is served by default, with generic parsers behind it. Before upgrading, check anything that assumed the endpoint was absent or gated.
llama: update llama.cpp to b11521 for decision models and EmbeddingGemma 2 ollama/ollama
GGUF decision models now send `/v1/systemone` requests straight to llama.cpp, which builds each model's prompt, and Ollama's own llama-server scorer and the Clef patch are gone. If you pinned to that internal path, the routing has moved out from under you.
metal : fix mul_mat_id with duplicate ids (#30373) ggml-org/llama.cpp
`kernel_mul_mm_id_map0` assumed at most one match per token and a fixed capacity of ne21 entries per expert, producing wrong or dropped output rows when ids repeated. MoE inference on Metal produced silently incorrect output in that case, which makes this a correctness fix worth taking rather than a routine one.
[PD] Recompute retracted DeepSeek-V4 requests on prefill instead of crashing decode sgl-project/sglang
Every DeepSeek-V4 decode server now recovers a retracted request by recomputing its prefix on prefill, where previously only V4.1 with DSpark did and the rest resolved to a `cpu_tensor` backup that crashed decode. Anyone serving DeepSeek-V4 without DSpark was exposed to a decode-side crash.
server : escape the special tokens without std::regex ggml-org/llama.cpp
A `/v1/systemone` request could crash `llama-server` on Linux with a d1-omni model; the cause was libstdc++ matching `<\|([A-Za-z0-9_]+)\|>` through a recursive `std::regex_replace`. The escape path now avoids `std::regex` entirely.
cuda, hip: GDN chunked kernel ggml-org/llama.cpp
Alongside: a GDN chunked CUDA/hip kernel upstreamed from QVAC fabric, a multi-query Triton split-K path for spec-decode verify in vLLM, an EAGLE replay boundary fix for Mooncake sliding-window tails, and a layerwise offload for LTX-2.5's 23.9 GB Gemma-4 text encoder that closes host-memory pressure in sglang.
$ ls local-llm/week/ # the briefings behind this review
Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.