RepoJournal
Local LLMs Local LLMs
48 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-09-09
stories 249
RSS Archive Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

Ollama retries compaction after context overflow, llama.cpp fuses Vulkan kernels

By RepoJournal · Filed · About Local LLMs

ParthSareen zcnrex Ankit Khandelwal Foad Abo Dahood WakeUpMorty fish-jiang LopezCastroRoberto
15 people shipped this
ParthSareen @ParthSareen 3 cited zcnrex @zcnrex 2 cited Ankit Khandelwal @Ankk98 1 cited Foad Abo Dahood @masterFoad 1 cited WakeUpMorty @WakeUpMorty 1 cited fish-jiang @fish-jiang 1 cited LopezCastroRoberto @LopezCastroRoberto 1 cited akshaver @akshaver 1 cited gcanlin @gcanlin 1 cited Roderick-Wu @Roderick-Wu 1 cited ankith117 @ankith117 1 cited Fridge003 @Fridge003 1 cited Qiaolin-Yu @Qiaolin-Yu 1 cited hoyyeva @hoyyeva yewentao256 @yewentao256

Ollama's server now recovers from context-overflow errors by retrying compaction with trimmed history, while llama.cpp fuses common activation-plus-multiply kernels on Vulkan to cut inference passes.

Ollama's server now retries compaction after a context-overflow error, addressing a failure mode where expanding a conversation into a summarization prompt could exceed the model's context limit and leave the client retrying oversized history. The retry removes roughly 20% of the oldest removable transcript text while preserving user and instruction messages, prior summaries, and active tool state. [1] Separately, Ollama's OpenAI compatibility layer now recognizes plaintext-labeled Codex agent messages and finalizes responses gracefully at the web search limit instead of erroring out. [2][3]

llama.cpp fused UNARY(GELU|SIGMOID|SILU|SOFTPLUS) operations with a following MUL on Vulkan, matching CUDA's unary_mul optimization. The fusion applies to adjacent nodes and supports OP-on-B cases such as `ffn_shexp * sigmoid(gate)`, which should cut kernel launches for per-layer gating in models like Gemma and Qwen. [4] A dedicated iq4_xs mat-vec shader for the dmmv path replaces the generic fallback, yielding roughly 6 to 17% faster token generation on RDNA4 depending on model. [5] Also on Vulkan, new f16 B-type matmul pipelines and warp tile size tuning target Intel coopmat1, with performance gains reported on Xe1-ARL_H. [6] The server-side checkpoint min-step eviction now only applies when the checkpoint list is full, preventing premature eviction of the resume checkpoint for short prompts. [7]

vLLM removed GPTQ group/dynamic activation ordering entirely, including the `has_g_idx` field, tests, and kernels, aligning with llm-compressor and compressed-tensors which already dropped the feature. [8] In the ongoing DSv4 warmup migration, common attention kernels (fused Q/K normalization, MTP input normalization, cache compression) moved to the shared warmup contract. [9] A spec-decode bugfix for EAGLE resumes now caches the Mamba state at the block-grid position of the resume, fixing a prefix-cache defect. [10] New B200 Triton MoE configurations were tuned for Qwen3.8-Flash-Next-FP8 at TP2 and TP4. [11]

SGLang added a Triton split-K router GEMV for MiniMax-M3, addressing poor vendor BLAS performance on gfx950 where the skinny GEMV shape ran at ~0.1 TB/s for a ~1.6MB weight read. The kernel reads the gate weight once at near-bandwidth. [12] It also shares the sparse index top-k across consecutive layers and reuses the decode top-k buffer, at least for value-disabled layers. [13] An AMD-specific fix skips returning paged SWA pages under per-request ring when using `--attention-backend dsv4 --page-size 256`, preventing a deterministic crash in warmup decode. [14] CP V1 deprecation continues with API name canonicalization in prefill. [15] Qwen 3.8 Flash Next is now supported. [16]

Action items

References

  1. [1] server: retry compaction after context overflow ↗ ollama/ollama
  2. [2] openai: accept plaintext-labeled Codex agent messages ↗ ollama/ollama
  3. [3] openai: finalize responses at the web search limit ↗ ollama/ollama
  4. [4] vulkan : fuse UNARY(GELU|SIGMOID|SILU|SOFTPLUS) + MUL ↗ ggml-org/llama.cpp
  5. [5] vulkan: add dedicated iq4_xs mat-vec shader (#28426) ↗ ggml-org/llama.cpp
  6. [6] vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 ↗ ggml-org/llama.cpp
  7. [7] server : apply checkpoint min-step eviction only when the checkpoint list is full (#28302) ↗ ggml-org/llama.cpp
  8. [8] [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ↗ vllm-project/vllm
  9. [9] [4/N][warmup][DSv4] Migrate common attention kernels ↗ vllm-project/vllm
  10. [10] [Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume ↗ vllm-project/vllm
  11. [11] [Qwen3.8-Flash-Next] Tune FP8 TP2/TP4 Triton MoE on B200 ↗ vllm-project/vllm
  12. [12] MiniMax-M3: Triton split-K router GEMV with in-kernel fixup ↗ sgl-project/sglang
  13. [13] MiniMax-M3: share the sparse index top-k across layers and reuse the decode top-k buffer ↗ sgl-project/sglang
  14. [14] [AMD][DSV4] Skip the paged SWA page return under the per-request ring ↗ sgl-project/sglang
  15. [15] [CP V1 Deprecation 4/5] Canonicalize prefill CP API names ↗ sgl-project/sglang
  16. [16] support qwen 3.8 flash next ↗ sgl-project/sglang

$ ls local-llm/reviews/ # the step-back reads

the week in review Vulkan top_k radix select arrives for long rows the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive →

Quick answers

What shipped in Local LLMs on September 9, 2026?
Ollama's server now recovers from context-overflow errors by retrying compaction with trimmed history, while llama.cpp fuses common activation-plus-multiply kernels on Vulkan to cut inference passes. In total, 120 commits, 119 pull requests, and 10 releases landed.
Who contributed to Local LLMs on September 9, 2026?
15 developers shipped this update, including ParthSareen, hoyyeva, Ankit Khandelwal, Foad Abo Dahood, WakeUpMorty, fish-jiang, LopezCastroRoberto, and akshaver, and 7 more.
What were the notable Local LLMs updates?
server: retry compaction after context overflow, openai: accept plaintext-labeled Codex agent messages, and openai: finalize responses at the web search limit.
♥

$ log Local LLMs --recent

SGLang fixes TP hangs and GLM-5.2 routing 242 shipped Vulkan adds TQ1_0, Spark2.5 lands in llama.cpp 180 shipped vLLM closes scale-out multimodal handoff hole 86 shipped vLLM Fast Start maps weights zero-copy; GLMGA sampling capped 256 shipped

$ log --all # elsewhere on the wire

@openclaw openclaw 2026.9.3 rehearses updates; discrawl guards publication compatibility AI Agents Cline collapses lingering undici 5.x to fix CVE-2026-1525

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.