RepoJournal
Local LLMs Local LLMs
59 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-07-24
stories 157
RSS Archive Reviews Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

LLAMA.CPP SHIPS FIVE RELEASES IN 24 HOURS, FIXES SPECULATIVE DECODING ACROSS THE STACK

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

github-actions[bot] Jesse Gross yuyue0225sc davidjpyu mgoin rainj-me polisettyvarma
12 people shipped this
github-actions[bot] @github-actions[bot] 4 cited Jesse Gross @jessegross 2 cited yuyue0225sc @yuyue0225sc 1 cited davidjpyu @davidjpyu 1 cited mgoin @mgoin 1 cited rainj-me @rainj-me 1 cited polisettyvarma @polisettyvarma 1 cited izhuhaoran @izhuhaoran chaeminlim-mb @chaeminlim-mb Qiaolin-Yu @Qiaolin-Yu hnyls2002 @hnyls2002 Jialin @Jialin

The inference stack is consolidating around speculative decoding, and the bugs surfacing there are getting squashed across three major projects simultaneously.

llama.cpp burned through five releases in the last day, each one tightening the hardware-specific kernels that matter at scale: CUDA's NVFP4 W4A4 quantization [4] now fuses per-channel amax and quantization to cut overhead, Hexagon activation ops [5] switched to striped DMA for non-contiguous tensors, and Metal got f16 support for leaky relu [3]. Meanwhile vLLM fixed a critical bug where speculative decoding silently broke grammar parsing by missing reasoning boundaries [7], a stop-string correctness bug that only surfaces under speculative decoding [8], and an OOM bug in rejection sampling that chunking now solves [9]. SGLang hit platform expansion: Intel XPU support shipped [11] and the Rust build switched to a workspace model [10]. The pattern is clear: speculative decoding is now the default path, and the three projects are racing to make it bulletproof. Ollama fixed an asymmetry in lm_head quantization [1] where untied heads now follow the same 8-bit rules as tied embedding models, bringing quality closer to bf16 baseline [2]. llama.cpp's speculative draft inference also got smarter: it now infers the draft type from repo sidecars when available [6], eliminating a manual flag. Across all desks, 76 commits and 76 PRs landed in the last 24 hours, with five releases shipped.

Action items

References

  1. [1] x/create: quantize lm_head at 8-bit in the requested family ↗ ollama/ollama
  2. [2] x/create: quantize lm_head at 8-bit in the requested family ↗ ollama/ollama
  3. [3] b10103 ↗ ggml-org/llama.cpp
  4. [4] b10099 ↗ ggml-org/llama.cpp
  5. [5] b10098 ↗ ggml-org/llama.cpp
  6. [6] b10094 ↗ ggml-org/llama.cpp
  7. [7] [Bugfix][Structured Output][Spec Decode] Advance grammar across reasoning boundary ↗ vllm-project/vllm
  8. [8] [Bugfix][Spec Decode] Select earliest-completing stop string in check_stop_strings ↗ vllm-project/vllm
  9. [9] [MRV2][Spec Decode] Avoid rejection sampler OOM by chunking ↗ vllm-project/vllm
  10. [10] create rust workspace ↗ sgl-project/sglang
  11. [11] [Intel GPU] Add XPU Platform support ↗ sgl-project/sglang

$ ls local-llm/reviews/ # the step-back reads

the week in review Sparse attention lands across vLLM, SGLang, llama.cpp the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive → Weekly reviews →

Quick answers

What shipped in Local LLMs on July 24, 2026?
The inference stack is consolidating around speculative decoding, and the bugs surfacing there are getting squashed across three major projects simultaneously. In total, 76 commits, 76 pull requests, and 5 releases landed.
Who contributed to Local LLMs on July 24, 2026?
12 developers shipped this update, including Jesse Gross, github-actions[bot], yuyue0225sc, davidjpyu, izhuhaoran, chaeminlim-mb, mgoin, and rainj-me, and 4 more.
What were the notable Local LLMs updates?
x/create: quantize lm_head at 8-bit in the requested family, x/create: quantize lm_head at 8-bit in the requested family, and b10103.
♥

$ log Local LLMs --recent

Metal MoE/SSM fusion lands, MiMo-V2.5 fp8 sharding fixed 158 shipped Ollama exposes thinking levels, llama.cpp widens Hexagon and architecture coverage 250 shipped GGUF offset corruption fixed, vLLM restores Dynamo KV metadata 244 shipped MLX engine graduates out of x/, ROCm fixes silent attention corruption 268 shipped

$ log --all # elsewhere on the wire

@openclaw Cold-start Gateway diagnostics stop crying wolf, doctor repairs plugin drift Agentic Coding Cline desktop bundles shrink, goose unblocks its own clicks

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.