RepoJournal
Local LLMs Local LLMs
59 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-09-12
stories 253
RSS Archive Reviews Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

Ollama drops its CLI agent, llama.cpp cuts build times, vLLM fixes crash under DP>1

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

ParthSareen dhiltgen Xuan-Son Nguyen danbev masterFoad max-krasnyansky TheEpicDolphin
16 people shipped this
ParthSareen @ParthSareen 2 cited dhiltgen @dhiltgen 2 cited Xuan-Son Nguyen @ngxson 2 cited danbev @danbev 1 cited masterFoad @masterFoad 1 cited max-krasnyansky @max-krasnyansky 1 cited TheEpicDolphin @TheEpicDolphin 1 cited njhill @njhill 1 cited chengchengpei @chengchengpei 1 cited aurickq @aurickq 1 cited YAMY1234 @YAMY1234 1 cited jybsuper @jybsuper 1 cited mgoin @mgoin eastwood-c @eastwood-c Xinyu-Kang @Xinyu-Kang zijiexia @zijiexia

Ollama reverted its CLI to the old chat interface by removing the built-in agent, while llama.cpp landed PCH and unity build support and a memory-safety fix for concurrent blob downloads.

Ollama removed the built-in agent from the CLI, reverting to the previous chat interface on the command line [1][2]. Separately, dhiltgen shipped a guard that keeps Gemma3n's MobileNetV5 projector off the CPU backend [3][4]. The PR notes that the projector "silently produces corrupted image embeddings on the CPU backend - no error, the model just describes the wrong image," reproduced on llama.cpp b10760, where gemma4's encoder is fine on CPU [3]. Without the guard, partial-offload, limited-VRAM, and OOM-retry fallbacks would select the CPU projector on exactly the small GPUs where gemma3n lands [3][4].

On llama.cpp, danbev added precompiled headers and unity build support to cmake, targeting the most expensive headers in the parsing/frontend stage; the change does not affect codegen/backend [5]. ngxson refactored server subprocess handling onto a single-thread poll()/select() model and fixed a concurrent-download corruption bug where two processes writing the same blob path plus .downloadInProgress could grab each other's partial transfer; the in-progress file now carries the writer's pid [6][7]. On the Metal side, masterFoad split the kernel_mul_mm_id token tile into two NR1H=16 halves and skips the upper half when nr1 <= NR1H, avoiding multiplication of unused expert rows [8]. max-krasnyansky added --split-mode row support for the Hexagon backend, letting model tensors split across multiple NPUs [9].

In vLLM, TheEpicDolphin fixed a DP token-padding mismatch in dflash-derived speculators where extra padding tokens from the DP sync could desync query_start_loc_cpu from num_tokens, tripping an assertion on the FlashInfer backend with DP > 1 [10]. njhill scoped breakable cudagraphs to the piecewise path: VLLM_USE_BREAKABLE_CUDAGRAPH=1 no longer overrides FULL cudagraphs in ModelRunnerV1, so with cudagraph_mode=FULL_AND_PIECEWISE, decode dispatch now uses the standard CUDAGraphWrapper(FULL) [11]. chengchengpei fixed a Kimi-K3 KDA projection-overlap failure on H100/SM90a caused by skinny N/K kernels emitting PTX fma.f32.bf16 requiring SM100 or newer, and by FlashInfer's cute-dsl QKVG backend rejecting SM90 [12].

sglang's YAMY1234 fixed a paged sparse-decode gather on GB300 (SM103) where Qwen3.8-Flash-Next plain decode degenerated into endless token 0 for 40 to 60% of AIME25 samples at 32K max_tokens, and where fp8_e4m3 KV cache collapsed MTP/NEXTN accept length to 1.0 across 86 to 88% of AIME/GPQA samples, via zero-filled scratch, int64 offsets, and FP8 dequant on gather [14]. aurickq fixed RunAI object-storage checkpoint index filtering to read the cached index and validate shards against the remote listing, preventing stale-shard loads [13]. jybsuper enabled MoE LoRA in full and breakable prefill CUDA graphs, removing Triton's 32-request cap [15].

Action items

References

  1. [1] cmd: remove built-in agent ↗ ollama/ollama
  2. [2] cmd: remove built-in agent (#18393) ↗ ollama/ollama
  3. [3] llm: keep gemma3n projector off the CPU ↗ ollama/ollama
  4. [4] llm: keep gemma3n projector off the CPU (#18376) ↗ ollama/ollama
  5. [5] cmake : add PCH and unity build to improve build times ↗ ggml-org/llama.cpp
  6. [6] server: refactor subproc handling (#28555) ↗ ggml-org/llama.cpp
  7. [7] server: refactor subproc handling ↗ ggml-org/llama.cpp
  8. [8] metal : skip the empty half of the mul_mm_id token tile, load iq2/iq3 codebooks as uint32 ↗ ggml-org/llama.cpp
  9. [9] hexagon: support for multi-device model split (aka row-split) ↗ ggml-org/llama.cpp
  10. [10] [BugFix] Fix DP token padding in dflash attention metadata ↗ vllm-project/vllm
  11. [11] [MRV1] Scope breakable cudagraphs to the piecewise path only ↗ vllm-project/vllm
  12. [12] [Bugfix][Kimi-K3] Fix KDA projection overlap on Hopper ↗ vllm-project/vllm
  13. [13] Fix RunAI object-storage checkpoint index filtering ↗ sgl-project/sglang
  14. [14] fix(qsa): make the paged sparse-decode gather memory-safe (zero-fill scratch, int64 offsets, dequant FP8 on gather) ↗ sgl-project/sglang
  15. [15] [LoRA] Support MoE in full and breakable prefill CUDA graphs ↗ sgl-project/sglang

$ ls local-llm/reviews/ # the step-back reads

the week in review Sparse attention lands across vLLM, SGLang, llama.cpp the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive → Weekly reviews →

Quick answers

What shipped in Local LLMs on September 12, 2026?
Ollama reverted its CLI to the old chat interface by removing the built-in agent, while llama.cpp landed PCH and unity build support and a memory-safety fix for concurrent blob downloads. In total, 123 commits, 123 pull requests, and 7 releases landed.
Who contributed to Local LLMs on September 12, 2026?
16 developers shipped this update, including ParthSareen, dhiltgen, danbev, Xuan-Son Nguyen, masterFoad, max-krasnyansky, TheEpicDolphin, and njhill, and 8 more.
What were the notable Local LLMs updates?
cmd: remove built-in agent, cmd: remove built-in agent (#18393), and llm: keep gemma3n projector off the CPU.
♥

$ log Local LLMs --recent

Metal MoE/SSM fusion lands, MiMo-V2.5 fp8 sharding fixed 158 shipped Ollama exposes thinking levels, llama.cpp widens Hexagon and architecture coverage 250 shipped GGUF offset corruption fixed, vLLM restores Dynamo KV metadata 244 shipped MLX engine graduates out of x/, ROCm fixes silent attention corruption 268 shipped

$ log --all # elsewhere on the wire

@openclaw Cold-start Gateway diagnostics stop crying wolf, doctor repairs plugin drift Agentic Coding Cline desktop bundles shrink, goose unblocks its own clicks

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.