RepoJournal
Local LLMs Local LLMs
46 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-08-31
stories 184
RSS Archive Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

Vulkan top-k radix sort, HIP Q2_0 optimizations, and vLLM CUDA-graph memory fix

By RepoJournal · Filed · About Local LLMs

LunalFresh itterative am17an 0cc4m khluu simondanielsson machero
14 people shipped this
LunalFresh @LunalFresh 1 cited itterative @itterative 1 cited am17an @am17an 1 cited 0cc4m @0cc4m 1 cited khluu @khluu 1 cited simondanielsson @simondanielsson 1 cited machero @machero 1 cited tugot17 @tugot17 1 cited BBuf @BBuf 1 cited ngxson @ngxson Juan Pérez de Algaba @jperezdealgaba mmangkad @mmangkad caihuali95 @caihuali95 hnyls2002 @hnyls2002

Across three repos, the most consequential changes tune performance on AMD and Vulkan, fix a vLLM memory regression, and extend speculative decoding support.

Vulkan's top-k path now uses a radix sort for k >= 1024, enabling Qwen 3.8 Flash Next support [1]. HIP's Q2_0 dot-product path for gfx1201 gets native amdgcn perm optimizations [2], and RDNA 3/4 mmq configs are further tuned [3]. A new SWIGLU_CLAMP op in ggml clamps the SWIGLU variant used by DSV4 and GLM, cutting 3 separate ops into one and improving decode throughput [4]. vLLM fixes a default CUDA-graph memory regression from #50488 by keeping default capture sizes memory-safe; on H200 this restores the established ceiling of 512 tokens [5]. The dual-stream decode with hipgraphs PR was re-opened after a root-cause fix for a gsm8k accuracy failure [6], and vLLM adds batch-invariance regression tests for Qwen3-VL [7]. sglang adds DSpark speculative decoding for LFM2 and LFM2-MoE, including interleaved RoPE in the fused-KV kernel [8], and absorbs Qwen-Image output-projection biases to remove 224 BF16 add kernels per profiled step [9].

Action items

References

  1. [1] vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next ↗ ggml-org/llama.cpp
  2. [2] hip : optimize Q2_0 dot-product path for gfx1201 (#26753) ↗ ggml-org/llama.cpp
  3. [3] HIP: tune rdna 3 mmq configs ↗ ggml-org/llama.cpp
  4. [4] ggml: add SWIGLU_CLAMP ↗ ggml-org/llama.cpp
  5. [5] [Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe ↗ vllm-project/vllm
  6. [6] [Perf][ROCm] Dual-stream decode with hipgraphs ↗ vllm-project/vllm
  7. [7] [Test][VLM] Add batch-invariance tests for Qwen3-VL ↗ vllm-project/vllm
  8. [8] [Spec] Add LFM2 and LFM2-MoE DSpark speculative decoding support ↗ sgl-project/sglang
  9. [9] [diffusion] perf: absorb Qwen-Image output projection biases ↗ sgl-project/sglang

$ ls local-llm/reviews/ # the step-back reads

the week in review Vulkan top_k radix select arrives for long rows the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive →

Quick answers

What shipped in Local LLMs on August 31, 2026?
Across three repos, the most consequential changes tune performance on AMD and Vulkan, fix a vLLM memory regression, and extend speculative decoding support. In total, 87 commits, 87 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 31, 2026?
14 developers shipped this update, including LunalFresh, itterative, ngxson, am17an, 0cc4m, Juan Pérez de Algaba, khluu, and simondanielsson, and 6 more.
What were the notable Local LLMs updates?
vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next, hip : optimize Q2_0 dot-product path for gfx1201 (#26753), and HIP: tune rdna 3 mmq configs.
♥

$ log Local LLMs --recent

Vulkan adds TQ1_0, Spark2.5 lands in llama.cpp 180 shipped vLLM closes scale-out multimodal handoff hole 86 shipped vLLM Fast Start maps weights zero-copy; GLMGA sampling capped 256 shipped Ollama enables speculative decoding under structured output, vLLM adds LoRA and fusion 259 shipped

$ log --all # elsewhere on the wire

Meta React Native adds Stable API guards to renderer attributed string FastAPI & Pydantic Pydantic fixes Mypy plugin crash, tightens datetime constraints

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.