$ cat local-llm/week/2026-11-16.log
the week in review · Nov 16 – Nov 22, 2026
vLLM 0.34.0 drops Model Runner V1, enables extensible KV cache
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
The release spans 1,061 commits from 412 contributors and adds 10 models; five security fixes closed out the week.
v0.34.0 vllm-project/vllm
Extensible KV cache now defaults on, sizing the cache from memory actually free after warmup, and the release ships 10 new models including Mistral Large 4 and MiniMax-M3.1-Flash. Model Runner V1 is gone, so any custom attention or sampler plugging into that path needs porting before you take the bump.
[Bugfix][Security] Check the uncompressed size of embedding archives before torch.load (#67316) vllm-project/vllm
The loader now checks an archive's uncompressed size before torch.load runs, closing the gap where a small file could expand past the memory it was given. Part of a five-fix security batch that also saw SGLang swap DiffusionServer's inbound path to safe_pickle_loads.
[Refactor] server_args: Remove deprecated flags, choice values and environment variables sgl-project/sglang
Every spelling deprecated in 0.5.x is now rejected: CLI flags, choice values, environment variables and YAML keys stop the launch with a message naming the replacement rather than being remapped with a warning. Scripts that still passed the old names will fail at start instead of drifting silently.
mlxrunner: batch speculative verification ollama/ollama
Drafting stays active when several requests overlap, so MTP, DFlash and DFlash2 models keep their speedup under concurrent load instead of falling back to the single-client rate. A companion change lets a new request reuse a saved prefix from a busy slot.
server: answer reasoning_effort levels the template rejects with 400 (#32332) ggml-org/llama.cpp
The reasoning effort probe now surfaces levels the template rejects, so a request asking for something like max on Qwen3.8 gets a 400 instead of a 500 from /v1/chat/completions. Clients validating effort values before sending can finally trust the error they get back.
ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032) ggml-org/llama.cpp
The chat UI moved to ui/apps/chat and is now called Llama Chat, the first step toward splitting the UI into a shared library and the apps built on it. The code is unchanged but the paths are not, so anything mounting or patching the old location breaks.
convert : fix FP8 qwen4exp checkpoints (n-gram table scale, block-scaled FP8) ggml-org/llama.cpp
Conversion now handles FP8 qwen4exp checkpoints that store the n-gram table as 128 F8_E4M3 shards behind a single BF16 scale, including Qwen3.8-Flash-Next-FP8 and its NVFP4 sibling. Block-scaled FP8 gets the same treatment.
Fail `test-backend-ops` and exit with error status 1 when missing and/or unsupported types and/or unsupported backends are provided in the command line ggml-org/llama.cpp
test-backend-ops exits 1 when given a missing or unsupported type or backend, replacing the 0/0 PASS and success code a mistyped filter used to produce. Alongside: Vulkan's radix top-k tie selection is now deterministic, and Ollama route on Windows iGPUs moved CPU-side weights out of pinned host memory, which shares the WDDM budget with device buffers.
$ ls local-llm/week/ # the briefings behind this review
Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.