Who contributed to Local LLMs on August 23, 2026?

11 developers shipped this update, including CISC, ngxson, Kartik Sirohi, superm1, DarkLight1337, taneem-ibrahim, yewentao256, and mmangkad, and 3 more.

What were the notable Local LLMs updates?

[VLM] Split Pixtral multi-image features before the CUDA IPC wrap, [diffusion] optimization: transfer mapped layers through a courier thread, and [Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement.

Local LLMs

Ollama, llama.cpp, vLLM, and SGLang - running and serving open models locally

Pick a date

Topics: Local LLMs AI / ML Full archive →

The Wire · Showcase

SGLang Fixes Pixtral 500s as vLLM Cuts TTFT 25%

By RepoJournal · Filed 06:59 UTC on August 23, 2026 · About Local LLMs

11 people shipped this

Kartik Sirohi @sirohikartik 1 cited

superm1 @superm1 1 cited

taneem-ibrahim @taneem-ibrahim 1 cited

yewentao256 @yewentao256 1 cited

mmangkad @mmangkad 1 cited

mickqian @mickqian 1 cited

CISC @CISC

ngxson @ngxson

DarkLight1337 @DarkLight1337

ShangmingCai @ShangmingCai

liusy58 @liusy58

The biggest news overnight: SGLang is patching a production-breaking Pixtral bug, and vLLM engineers found a way to shave up to 25% off time-to-first-token on Mamba models.

sglang's Pixtral multi-image requests have been throwing `HTTP 500` errors whenever CUDA IPC transport is on, with a clear culprit: splitting bundled images happens after the CUDA wrap, triggering a `TypeError` [1]. The fix moves feature splitting before the IPC wrap, and it's one of two diffusion-related PRs that also landed, including a courier thread that makes mapped-layer transfers genuinely asynchronous [2]. Over on vLLM, a new internal prefill checkpoint approach avoids a second full-model pass through attention, MoE, routing, and TP collectives; it claims "9%~25% TTFT improvement" for Mamba prefix caching [3]. vLLM also fixed a pooling throughput reporting bug: `vllm bench serve` now counts logical input cardinality for batched embedding or reranking requests, not just HTTP responses [4]. Meanwhile llama.cpp optimized the concat op by swapping per-element memcpy for row-level memcpy [5], and restored the Ubuntu ROCm CI job after fixing a `ccache` that was failing on mtime-based caching [6]. Both of those are solid quality-of-life wins for the C++ backend.

Action items

→ If you use Pixtral with CUDA IPC transport, pull the sglang fix before your next deploy sgl-project/sglang [immediate]
→ Upgrade vLLM to get the Mamba prefix caching prefill checkpoints if you serve Mamba with prefix caching vllm-project/vllm [plan]
→ Keep an eye on the llama.cpp concat optimization for potential CPU inference speedups ggml-org/llama.cpp [monitor]
→ Pull the latest sglang diffusion optimizations if you use serialized BnB4 components sgl-project/sglang [monitor]

References

[1] [VLM] Split Pixtral multi-image features before the CUDA IPC wrap ↗ sgl-project/sglang
[2] [diffusion] optimization: transfer mapped layers through a courier thread ↗ sgl-project/sglang
[3] [Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement ↗ vllm-project/vllm
[4] [Pooling] Report input throughput for batched requests ↗ vllm-project/vllm
[5] ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (#24575) ggml-org/llama.cpp
[6] Restore ROCm job for Ubuntu ↗ ggml-org/llama.cpp

Quick answers

What shipped in Local LLMs on August 23, 2026?: The biggest news overnight: SGLang is patching a production-breaking Pixtral bug, and vLLM engineers found a way to shave up to 25% off time-to-first-token on Mamba models. In total, 62 commits, 62 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 23, 2026?: 11 developers shipped this update, including CISC, ngxson, Kartik Sirohi, superm1, DarkLight1337, taneem-ibrahim, yewentao256, and mmangkad, and 3 more.
What were the notable Local LLMs updates?: [VLM] Split Pixtral multi-image features before the CUDA IPC wrap, [diffusion] optimization: transfer mapped layers through a courier thread, and [Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement.

SGLang ships v0.5.18 with 710 PRs and DeepSeek V4 FP4 auto-routing

The biggest release of the week just landed: SGLang v0.5.18 adds new models, fixes a silent disagg bug, and defaults DeepSeek V4 FP4 checkpoints to the right MoE runner.

+11

llms 266 shipped

Local LLMs Aug 21

DeepSeek V4 fuses shared experts, as vLLM ships a security guard

The biggest local-LLM repos are moving fast: DeepSeek V4's MoE kernel gets a massive fusion, while vLLM patches a security hole.

+12

llms 274 shipped

Local LLMs Aug 20

OLLAMA FIXES DEADLOCK THAT HUNG CHAT AND GENERATE

A mid-stream parser error in Ollama's server silently wedged chat and generate requests, leaking goroutines and hanging retries until a fix landed overnight.

+14

llms 268 shipped

Local LLMs Aug 18

LLAMA.CPP SHIPS BAILINGMOE3 SPEC-DECODING SUPPORT

llama.cpp lands full BailingMoE3 support with speculative decoding, giving developers a native path for a key emerging sparse MoE family.

+10

llms 234 shipped

Elsewhere on the wire

AI Agents Aug 23

Cline hubs now drain without dropping work

Cline's new hub drain and upgrade flow means you can restart a hub mid-session and lose nothing, thanks to durable event replay.

ai-agents 18 shipped

@CachyOS Aug 23

AIRE: ANANICY RULES FIX SESSION LAUNCHER SCHEDULING

ly-dm was being pinned to the idle scheduler, killing your login session's responsiveness.

infra 74 shipped

Elixir & Phoenix Aug 23

JOSE VALIM FIXES THE CLASUE BUG

Jose Valim shipped two refactors plus the actual fix for that `none` clause compiler issue, and a UTC fast path in DateTime could save you a TZ DB lookup.