RepoJournal
Local LLMs Local LLMs
122 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-13
stories 280

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

llama.cpp fuses NVFP4 weight scales into MMQ prefill, sglang fixes zeroed MoE routing under pipeline parallelism

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

The day's work is almost entirely performance and correctness fixes inside existing kernels: fused scales for FP4 prefill, a Mamba-aware eviction policy, and a three-line class of bugs where the wrong layer's parameters reached the wrong block.

CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints ggml-org/llama.cpp

by kmorennv

Prefill batches on FP4-quantized checkpoints used to run the weight scale as a separate multiply after MMQ; the epilogue now folds s_W directly into the fused kernel, so per-tensor scales on dense projections and per-expert scales on MoE expert projections come for free at prefill batch sizes. It matters wherever you serve NVFP4 weights with *_s tensors.

[Feat] mem_cache: Evict Mamba states by GreedyDual-Size (--radix-eviction-policy gds) sgl-project/sglang

by ischencheng

The Mamba state pool is far smaller than the KV pool (419 state slots against 2,969,600 KV tokens on GLM-5.3-Flash NVFP4 across 2x B200 at TP2), and LRU of one state at a time drops the prefix a returning agent session needs. --radix-eviction-policy gds evicts by GreedyDual-Size instead.

[Feat] pp: Relay routed experts through pipeline-parallel stages sgl-project/sglang

by Josephasafg

--enable-return-routed-experts under --pp-size > 1 only ever captured the first stage's layers and returned zeros for the rest: on Qwen3-30B-A3B with --pp-size 2, layers 24 to 47 came back empty, 49.6% of all ids. Routed experts now relay across pipeline stages, which is the difference between a usable and a poisoned RL routing replay.

[Fix] minimax-h3: preserve per-layer AdaLN parameters with Cache-DiT sgl-project/sglang

by forrestl111

Cache-DiT replaced the gathered AdaLN stack with a single-entry wrapper during forward, so every MiniMax H3 block received layer 0's modulation parameters, and a full-compute Cache-DiT step could diverge from the uncached model before any cache reuse at all. Each block now keeps its own layer parameters.

[Bugfix][Rust Frontend] Resolve auxiliary frames of add requests in the mock engine vllm-project/vllm

by BugenZhao

vllm-mock-engine silently dropped every add request carrying an auxiliary ZMQ frame, so any prompt of 64 tokens or more never got an answer. A new EngineCoreRequest::from_frames resolves every array sent that way. The quieter long tail: an Intel prefill FA shader and a Vulkan GGML_VULKAN_COORDINATE compile fix in llama.cpp, SpacemiT RVV and Q4_1 dispatch, a search_chats tool that ships disabled...

Quick answers

What shipped in Local LLMs on November 13, 2026?
The day's work is almost entirely performance and correctness fixes inside existing kernels: fused scales for FP4 prefill, a Mamba-aware eviction policy, and a three-line class of bugs where the wrong layer's parameters reached the wrong block. In total, 135 commits, 135 pull requests, and 10 releases landed.
Who contributed to Local LLMs on November 13, 2026?
17 developers shipped this update, including jberg5, anra8571, ParthSareen, kmorennv, sliu39, gitGurugu, ngxson, and Siddharth Nachane, and 9 more.
What were the notable Local LLMs updates?
CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints, [Feat] mem_cache: Evict Mamba states by GreedyDual-Size (--radix-eviction-policy gds), and [Feat] pp: Relay routed experts through pipeline-parallel stages.