RepoJournal

$ cat local-llm/week/2026-11-16.log

Local LLMs

Local LLMs

the week in review · Nov 16 – Nov 22, 2026

vLLM 0.34.0 drops Model Runner V1, enables extensible KV cache

By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology

♥

The release spans 1,061 commits from 412 contributors and adds 10 models; five security fixes closed out the week.

943 commits 939 PRs merged 71 releases 1 security advisories 7 briefings covered

all local-llm reviews →

v0.34.0 vllm-project/vllm

by khluu

Extensible KV cache now defaults on, sizing the cache from memory actually free after warmup, and the release ships 10 new models including Mistral Large 4 and MiniMax-M3.1-Flash. Model Runner V1 is gone, so any custom attention or sampler plugging into that path needs porting before you take the bump.

[Bugfix][Security] Check the uncompressed size of embedding archives before torch.load (#67316) vllm-project/vllm

by Leizhen Zhang

The loader now checks an archive's uncompressed size before torch.load runs, closing the gap where a small file could expand past the memory it was given. Part of a five-fix security batch that also saw SGLang swap DiffusionServer's inbound path to safe_pickle_loads.

[Refactor] server_args: Remove deprecated flags, choice values and environment variables sgl-project/sglang

by hnyls2002

Every spelling deprecated in 0.5.x is now rejected: CLI flags, choice values, environment variables and YAML keys stop the launch with a message naming the replacement rather than being remapped with a warning. Scripts that still passed the old names will fail at start instead of drifting silently.

mlxrunner: batch speculative verification ollama/ollama

by dhiltgen

Drafting stays active when several requests overlap, so MTP, DFlash and DFlash2 models keep their speedup under concurrent load instead of falling back to the single-client rate. A companion change lets a new request reuse a saved prefix from a busy slot.

server: answer reasoning_effort levels the template rejects with 400 (#32332) ggml-org/llama.cpp

by Pascal

The reasoning effort probe now surfaces levels the template rejects, so a request asking for something like max on Qwen3.8 gets a 400 instead of a 500 from /v1/chat/completions. Clients validating effort values before sending can finally trust the error they get back.

ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032) ggml-org/llama.cpp

by Aleksander Grygier

The chat UI moved to ui/apps/chat and is now called Llama Chat, the first step toward splitting the UI into a shared library and the apps built on it. The code is unchanged but the paths are not, so anything mounting or patching the old location breaks.

convert : fix FP8 qwen4exp checkpoints (n-gram table scale, block-scaled FP8) ggml-org/llama.cpp

by am17an

Conversion now handles FP8 qwen4exp checkpoints that store the n-gram table as 128 F8_E4M3 shards behind a single BF16 scale, including Qwen3.8-Flash-Next-FP8 and its NVFP4 sibling. Block-scaled FP8 gets the same treatment.

Fail `test-backend-ops` and exit with error status 1 when missing and/or unsupported types and/or unsupported backends are provided in the command line ggml-org/llama.cpp

by EAddario

test-backend-ops exits 1 when given a missing or unsupported type or backend, replacing the 0/0 PASS and success code a mistyped filter used to produce. Alongside: Vulkan's radix top-k tie selection is now deterministic, and Ollama route on Windows iGPUs moved CPU-side weights out of pinned host memory, which shares the WDDM budget with device buffers.

$ ls local-llm/week/ # the briefings behind this review

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

all local-llm reviews →