RepoJournal
Local LLMs Local LLMs
122 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-17
stories 285

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

vLLM 0.34.0 drops Model Runner V1, Ollama batches speculative decode

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

vLLM shipped 0.34.0 with extensible KV cache on by default and Model Runner V1 removed, while Ollama and llama.cpp spent the day making concurrent serving faster and more reproducible.

v0.34.0 vllm-project/vllm

by khluu

The release carries 1,061 commits from 412 contributors, turns on an extensible KV cache sized from memory actually free after warmup (more cache and throughput for most models), and removes Model Runner V1 entirely, leaving V2 as the only GPU runner. Model Runner V1 is gone: anything pinned to GPUModelRunner V1 via the V0 path stops working on this version.

mlxrunner: batch speculative verification ollama/ollama

by dhiltgen

Ollama's mlxrunner used to disable drafting whenever several generation requests overlapped, so models with MTP, DFlash or DFlash2 lost their speculative speedup under concurrency. Drafting now stays on and proposed tokens are verified in batches, at the cost of more temporary GPU memory and sometimes a slightly later first token; per-request sampling and output constraints stay separate.

mlxrunner: reuse saved prefixes across request slots ollama/ollama

by dhiltgen

The runner already reused a conversation held by an idle slot, but a request arriving when that slot was busy reprocessed the same prompt from scratch. A new request can now reuse a compatible saved prefix from a busy slot, picking the longest complete saved prefix, so overlapping conversations with shared prompts start responding sooner.

vulkan: make the radix top-k tie selection deterministic ggml-org/llama.cpp

by ServeurpersoCom

Radix top-k in the Vulkan backend (k >= 1024 and the fused TOPK_QSA path) broke ties with a shared atomic counter, so the kept columns changed from run to run when ties outnumbered open slots; it now follows the Metal kernel with each subgroup owning a contiguous chunk of the row. The nary and bitonic paths were already deterministic, and with this the Vulkan output matches the CUDA and Metal f...

[Perf][Muse Spark] FP8 KV cache for the specialized decode vllm-project/vllm

by jeejeelee

FP8 (E4M3) KV cache support lands for the Muse Spark latency_specialized decode on Blackwell, including the --kv-cache-dtype-skip-layers sliding_window option, measured on a 4x B300 TP4 setup with DSpark 7 speculative tokens at concurrency 1. vLLM reports TPOT down 6.0% versus the stock decode at 150k context, or 5.6% with SWA layers kept in BF16, with gsm8k unchanged in both cases.

ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032) ggml-org/llama.cpp

by Aleksander Grygier

Also landed: llama.cpp moved the web UI from tools/ui to ui/apps/chat and renamed it Llama Chat with no code change, a step toward splitting ui/lib from the apps built on it, so any CMake, Docker, Nix or workflow path pointing at tools/ui needs updating. Elsewhere, kv-cells replaced its per-removal O(n_cells) scan with a sorted position index, YuE2-3B support arrived in mtmd, and sglang fixed W...

Quick answers

What shipped in Local LLMs on November 17, 2026?
vLLM shipped 0.34.0 with extensible KV cache on by default and Model Runner V1 removed, while Ollama and llama.cpp spent the day making concurrent serving faster and more reproducible. In total, 138 commits, 136 pull requests, and 11 releases landed.
Who contributed to Local LLMs on November 17, 2026?
17 developers shipped this update, including dhiltgen, Gaurav Jain, Aleksander Grygier, ngxson, ServeurpersoCom, Titaniumtown, aldehir, and khluu, and 9 more.
What were the notable Local LLMs updates?
v0.34.0, mlxrunner: batch speculative verification, and mlxrunner: reuse saved prefixes across request slots.