RepoJournal
Local LLMs Local LLMs
46 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-09-05
stories 256
RSS Archive Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

vLLM Fast Start maps weights zero-copy; GLMGA sampling capped

By RepoJournal · Filed · About Local LLMs

ParthSareen hoyyeva Adrien Gallouët nikwen Little0o0 intel00000 Juan Pérez de Algaba
16 people shipped this
ParthSareen @ParthSareen 4 cited hoyyeva @hoyyeva 1 cited Adrien Gallouët @angt 1 cited nikwen @nikwen 1 cited Little0o0 @Little0o0 1 cited intel00000 @intel00000 1 cited Juan Pérez de Algaba @jperezdealgaba 1 cited Clinton Thomas @KernelClint 1 cited liusy58 @liusy58 1 cited zhang-keliang @zhang-keliang 1 cited gau-nernst @gau-nernst 1 cited Qiaolin-Yu @Qiaolin-Yu 1 cited JustinTong0323 @JustinTong0323 1 cited clintg6 @clintg6 1 cited zijiexia @zijiexia 1 cited alexnails @alexnails 1 cited

Two security patches and a zero-copy weight cache land across vLLM and llama.cpp, while sglang ships v0.5.19 with 786 PRs.

vLLM shipped two security fixes: GLMGA video sampling is now capped to prevent request-driven resource exhaustion [1], and cache salts are validated before they reach LMCache [2]. Both are security patches; validate your deployment before relying on the default behavior. Separately, the "Fast Start" PR introduces a persistent per-GPU daemon that holds post-quantized, TP-sharded weights in GPU memory, so engine restarts map them via CUDA IPC zero-copy instead of reloading from disk [3]. This is a new mechanism for weight caching, not yet a default; monitor it if you restart engines frequently.

In llama.cpp, ggml no longer crashes when the backend search path is inaccessible. The path is now skipped instead of terminating the process with an uncaught filesystem_error [11]. The UI export now reads from the database rather than the page-load cache, fixing a bug where exported conversations with branches had an empty "currNode" field [12]. On the model front, Tencent Hy 4 preview architecture support landed [13], and Metal gained fa-vec tuning rows for Q4_0/Q4_1/Q5_0/Q5_1 on M3 Max [14].

sglang released v0.5.19, which the maintainers describe as "*786 PRs from 214 contributors.*" [6]. The release adds support for Qwen3.8 models and other highlights. A separate PR adds support for the Hy4-preview architecture (hy_v4, text-only), with MLA plus DSA sparse attention, gated MLA, learned attention sinks, sigmoid-gated MoE, MTP/NextN speculative drafting, and MXFP8 quantization [7]. Two performance items also landed: one optimizes Kimi-K3 Triton MLA prefill on AMD gfx950 [8], and another vectorizes alloc_extend_naive to remove the per-request Python loop [10]. SGLang has added measured B300 1x8 numbers to the Kimi-K3 cookbook [9], though those are separate from the release.

Ollama improved Codex desktop proxy handling [15] and routes auto review through the selected model [16], while the OpenAI-compatible endpoint now preserves images through response compaction [17] and adds a compaction endpoint [18]. ChatGPT Desktop on macOS can now select Ollama models [19]. None of these are breaking changes.

vLLM also fixed a double BOS issue in LLM.chat() for multimodal models [4] and improved QSA sparse GQA for prefill and short-context decode [5].

Action items

References

  1. [1] [Security] Cap GLMGA video sampling to prevent request-driven resource exhaustion (#54935) ↗ vllm-project/vllm
  2. [2] [Security] Validate cache salts before they reach LMCache (#51444) ↗ vllm-project/vllm
  3. [3] Fast Start ↗ vllm-project/vllm
  4. [4] [Bugfix] Fix double BOS in LLM.chat() for multimodal models ↗ vllm-project/vllm
  5. [5] [Qwen3.8-Flash-Next] Improve QSA sparse GQA for prefill and short-ctx decode ↗ vllm-project/vllm
  6. [6] v0.5.19 ↗ sgl-project/sglang
  7. [7] Support Hy4-preview ↗ sgl-project/sglang
  8. [8] [AMD] Optimize Kimi-K3 Triton MLA prefill on gfx950 ↗ sgl-project/sglang
  9. [9] [Cookbook] Kimi-K3: add measured B300 1x8 Unified 8k/1k speed numbers ↗ sgl-project/sglang
  10. [10] [Perf] Vectorize alloc_extend_naive to remove the per-request Python loop ↗ sgl-project/sglang
  11. [11] ggml : don't crash when backend search path can't be read (#28271) ↗ ggml-org/llama.cpp
  12. [12] ui: export conversations from database instead of cached store ↗ ggml-org/llama.cpp
  13. [13] Model: add Tencent Hy 4 (hy_v4) preview architecture support ↗ ggml-org/llama.cpp
  14. [14] metal : add remaining fa-vec tunings for M3 Max ↗ ggml-org/llama.cpp
  15. [15] app: harden Codex desktop proxy handling ↗ ollama/ollama
  16. [16] app: route Codex auto review through selected model ↗ ollama/ollama
  17. [17] openai: preserve images through response compaction ↗ ollama/ollama
  18. [18] openai: add Codex compaction support ↗ ollama/ollama
  19. [19] app: add Ollama to ChatGPT Desktop ↗ ollama/ollama

$ ls local-llm/reviews/ # the step-back reads

the week in review Vulkan top_k radix select arrives for long rows the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive →

Quick answers

What shipped in Local LLMs on September 5, 2026?
Two security patches and a zero-copy weight cache land across vLLM and llama.cpp, while sglang ships v0.5.19 with 786 PRs. In total, 124 commits, 124 pull requests, and 8 releases landed.
Who contributed to Local LLMs on September 5, 2026?
16 developers shipped this update, including ParthSareen, hoyyeva, Adrien Gallouët, nikwen, Little0o0, intel00000, Juan Pérez de Algaba, and Clinton Thomas, and 8 more.
What were the notable Local LLMs updates?
[Security] Cap GLMGA video sampling to prevent request-driven resource exhaustion (#54935), [Security] Validate cache salts before they reach LMCache (#51444), and Fast Start.
♥

$ log Local LLMs --recent

Vulkan adds TQ1_0, Spark2.5 lands in llama.cpp 180 shipped vLLM closes scale-out multimodal handoff hole 86 shipped Ollama enables speculative decoding under structured output, vLLM adds LoRA and fusion 259 shipped Error propagation hardened in ollama's MLX bindings, gemma4 gains multimodal support 258 shipped

$ log --all # elsewhere on the wire

Meta React Native adds Stable API guards to renderer attributed string FastAPI & Pydantic Pydantic fixes Mypy plugin crash, tightens datetime constraints

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.