RepoJournal
Local LLMs Local LLMs
46 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-09-07
stories 180
RSS Archive Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

Vulkan adds TQ1_0, Spark2.5 lands in llama.cpp

By RepoJournal · Filed · About Local LLMs

Aleksander Grygier Anjielon KnightYao JoursBleu Whamp Thien Tran wenhuach21
14 people shipped this
Aleksander Grygier @allozaur 2 cited Anjielon @Anjielon 1 cited KnightYao @KnightYao 1 cited JoursBleu @JoursBleu 1 cited Whamp @Whamp 1 cited Thien Tran @gau-nernst 1 cited wenhuach21 @wenhuach21 1 cited billishyahao @billishyahao 1 cited mickqian @mickqian 1 cited arathi-hlab @arathi-hlab 1 cited froststeam @froststeam 1 cited zhouyou9505 @zhouyou9505 jeejeelee @jeejeelee Xia-Weiwen @Xia-Weiwen

llama.cpp and its ecosystem shipped new hardware support and a model architecture, while vLLM and SGLang fixed bugs in prefill and warmup paths.

llama.cpp gained end-to-end support for the Spark2.5 model family [1] and an opt-in `--fuse-qkv` flag on `convert_hf_to_gguf.py` [2], which concatenates complete per-layer Q/K/V triples into a single `attn_qkv` tensor when enabled. Also, the Vulkan backend now supports the TQ1_0 quant format across matrix multiplication, mat-vec, dequant, and get_rows operations [3]. The included test run on gfx1151 shows "MUL_MAT 11/11, MUL_MAT_ID 6/6, GET_ROWS 4/4, unchanged."

The llama.cpp chat UI improved rendering performance through two related changes [4][5]: active conversation fields are now updated in place rather than replacing the whole `activeConversation` object, which previously changed its identity on every send and triggered a full refetch of all messages, and off-screen rows are now lazily mounted near the viewport, avoiding full tree builds for rows the garbage collector keeps walking.

vLLM fixed a chunked prefill bug in KV offload where the wrong SWA chunk count was used as the store-reachability horizon, causing chunks to be submitted that the load path never requests [6]. The fix uses the prompt's final offloadable length while prefill is active. vLLM also removed `torch.compile` from the NVIDIA implementation of the Qwen3.8-Flash-Next model [7], which may affect compile-time overhead and runtime behavior, and restored CUDA support for 2-bit and 3-bit AutoRound formats [8].

SGLang added qlen>1 support for the aiter gluon path on AMD, which previously only worked with qlen=1 [9]. The diffusion path now silences internal warmup frame searches, which were repeatedly logging Wan and causal-frame warnings for each candidate [10]. The XPU nightly dashboard was fixed so it renders data instead of empty tables [11]. Finally, SGLang added MUSA installation support with a Dockerfile and a Moore Threads GPU installation guide [12], though the PR does not change scheduler or runtime behavior.

Action items

References

  1. [1] [Model] Support for Spark2_5ForCausalLM implementation ↗ ggml-org/llama.cpp
  2. [2] convert : add `--fuse-qkv` flag to fuse Q/K/V into QKV during HF-to-GGUF conversion ↗ ggml-org/llama.cpp
  3. [3] vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765) ↗ ggml-org/llama.cpp
  4. [4] ui: Improve Chat Messages rendering performance (#28460) ↗ ggml-org/llama.cpp
  5. [5] ui: Improve Chat Messages rendering performance ↗ ggml-org/llama.cpp
  6. [6] [Bugfix][KV Offload] Fix SWA store reachability during chunked prefill ↗ vllm-project/vllm
  7. [7] [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272) ↗ vllm-project/vllm
  8. [8] add 2/3/5/6/7 CUDA support in AutoRound format ↗ vllm-project/vllm
  9. [9] [AMD] support qlen>1 for aiter gluon path for Kimi K3 ↗ sgl-project/sglang
  10. [10] [diffusion] fix: quiet internal warmup frame searches ↗ sgl-project/sglang
  11. [11] [XPU][CI] Fix empty nightly dashboard ↗ sgl-project/sglang
  12. [12] [MUSA] Add installation guide and Dockerfile ↗ sgl-project/sglang

$ ls local-llm/reviews/ # the step-back reads

the week in review Vulkan top_k radix select arrives for long rows the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive →

Quick answers

What shipped in Local LLMs on September 7, 2026?
llama.cpp and its ecosystem shipped new hardware support and a model architecture, while vLLM and SGLang fixed bugs in prefill and warmup paths. In total, 85 commits, 85 pull requests, and 10 releases landed.
Who contributed to Local LLMs on September 7, 2026?
14 developers shipped this update, including Aleksander Grygier, Anjielon, KnightYao, JoursBleu, Whamp, zhouyou9505, Thien Tran, and jeejeelee, and 6 more.
What were the notable Local LLMs updates?
[Model] Support for Spark2_5ForCausalLM implementation, convert : add `--fuse-qkv` flag to fuse Q/K/V into QKV during HF-to-GGUF conversion, and vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765).
♥

$ log Local LLMs --recent

vLLM closes scale-out multimodal handoff hole 86 shipped vLLM Fast Start maps weights zero-copy; GLMGA sampling capped 256 shipped Ollama enables speculative decoding under structured output, vLLM adds LoRA and fusion 259 shipped Error propagation hardened in ollama's MLX bindings, gemma4 gains multimodal support 258 shipped

$ log --all # elsewhere on the wire

Meta React Native adds Stable API guards to renderer attributed string FastAPI & Pydantic Pydantic fixes Mypy plugin crash, tightens datetime constraints

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.