RepoJournal
Local LLMs Local LLMs
122 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-12
stories 286

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

CUDA fattn picks up head sizes 40 and 72, ROCm stack fills in FP8 and sparse MLA

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

llama.cpp closes two long-standing CUDA gaps (unusual attention head sizes and OCP FP8) while vLLM and SGLang spend the day making ROCm and Ascend backends actually keep up with their model families.

Cuda fattn mma head sizes 40 72 ggml-org/llama.cpp

by sihanyu03

Head sizes 40 and 72, the geometry used by the vision towers in Gemma 3/4 and Qwen3-VL, were excluded from every tensor-core dispatch branch in CUDA fattn and fell back to the tile kernel; the MMA fragment covers 16 halfs, so the kernel now pads geometry to 40 -> 48 and 72 -> 80 in registers, shared-memory strides and loop bounds while loading and storing the real size from global memory. Enabl...

[Attention][MiniMax-M3] Add FlashInfer MSA backend for SM90 and SM120/SM121 vllm-project/vllm

by rmhaskarnvidia

MiniMax-M3 sparse attention gets a FlashInfer msa_ops backend on Hopper and consumer Blackwell, GPUs that until now ran the Triton fallback because the vendored MSA kernels are SM100-only. It requires FlashInfer 0.7.3; on older builds the feature probes keep the Triton path, so merge order against the pin bump does not matter.

[ROCm][DSv4/4.1] Enable Tensor scale FP8 KV Cache for Sparse MLA vllm-project/vllm

by cagrikymk

DeepSeek V4 and V4.1 on ROCm gfx950 can now run --kv-cache-dtype fp8_e4m3 with AITER's sparse MLA kernel, storing each token as a plain 512-element FP8 row ([448 NoPE | 64 RoPE], one per-tensor scale) instead of the packed fp8_ds_mla record, which stays the default. The prerequisite is a hard one: AITER with ROCm/aiter#7429 (4e91c7ea6) or later.

[Perf][DSv4.1] DSpark draft on the specialized decode vllm-project/vllm

by yaoyaoding

With decode_backend set to latency_specialized on GB300, the DSpark draft layers of DeepSeek-V4.1-Flash now run on the same decode executor as the target model, reusing its fused all-reduce + mHC kernel, attention and routed-expert GEMMs, and sampling through the model's sample_sequential hook with a Markov chain sharded by vocabulary across TP ranks. It is the speculative-decoding half of the ...

[Feat] paged-experts: a disk tier for experts that do not fit host memory sgl-project/sglang

by N3mezis

Paged Experts kept every expert in a pinned host store, so a model that did not fit in RAM could not start at all; the new disk tier addresses exactly that case, sized from DeepSeek-V4-Flash-0731's 137.1 GiB of routed experts against a 62 GB host with roughly 45 GiB pinnable. Elsewhere the long tail holds a MoE stream-k fixup that read one element past its padding buffer, an FP8 MMQ gap deferre...

Quick answers

What shipped in Local LLMs on November 12, 2026?
llama.cpp closes two long-standing CUDA gaps (unusual attention head sizes and OCP FP8) while vLLM and SGLang spend the day making ROCm and Ascend backends actually keep up with their model families. In total, 138 commits, 138 pull requests, and 10 releases landed.
Who contributed to Local LLMs on November 12, 2026?
15 developers shipped this update, including sihanyu03, ORippler, Feiteng, Aleksander Grygier, ggerganov, cagrikymk, olka-amd, and micah-wil, and 7 more.
What were the notable Local LLMs updates?
Cuda fattn mma head sizes 40 72, [Attention][MiniMax-M3] Add FlashInfer MSA backend for SM90 and SM120/SM121, and [ROCm][DSv4/4.1] Enable Tensor scale FP8 KV Cache for Sparse MLA.