Who contributed to Local LLMs on August 26, 2026?

18 developers shipped this update, including Jesse Gross, hoyyeva, pd95, Kyozzz, Ryan C, github-actions[bot], allozaur, and JonathanC-ARM, and 10 more.

What were the notable Local LLMs updates?

mlxrunner: add structured output support, llm: remove the unused Grammar completion-request field, and server : reject prefilled assistant messages with tool calls (#27626).

Local LLMs

Ollama, llama.cpp, vLLM, and SGLang - running and serving open models locally

Pick a date

Topics: Local LLMs AI / ML Full archive →

The Wire · Showcase

OLLAMA'S MLX RUNNER NOW SPEAKS STRUCTURED OUTPUT

By RepoJournal · Filed 07:01 UTC on August 26, 2026 · About Local LLMs

18 people shipped this

Jesse Gross @jessegross 2 cited

Kyozzz @kyo-zzz 1 cited

github-actions[bot] @github-actions[bot] 1 cited

JonathanC-ARM @JonathanC-ARM 1 cited

njhill @njhill 1 cited

shikicloud @shikicloud 1 cited

yewentao256 @yewentao256

mgoin @mgoin

gau-nernst @gau-nernst

JiataiWang @JiataiWang

michaelzhang-ai @michaelzhang-ai

ch-wan @ch-wan

KnightYao @KnightYao

cctry @cctry

Ollama closed a silent correctness gap on Apple Silicon, and the LLM inference stack shipped a wave of fixes that touch everything from tool calls to native ARM support.

Ollama's MLX runner finally enforces the `format` field, using xgrammar to mask logits so every generated token satisfies the requested JSON or JSON Schema constraint [1]. It's the fix that makes structured output trustworthy on Apple Silicon, and it lands alongside a cleanup that removes the dead Grammar completion-request field, a leftover from the CGO engine removal [2]. Meanwhile, llama.cpp closed the hole where tool calls vanished when `--prefill-assistant` was active, forcing auto-continuation to skip trailing assistant messages with tool_calls and throwing a clear error instead [3]. And SGLang removed the fusion barrier on DeepSeek-V4, routing the shared expert through the same MoE kernel to cut kernel launches and stream syncs per layer [4]. For infrastructure teams, vLLM's cudagraph memory profiling fix prevents a nasty allocator assert by running profiling in a throwaway graph pool [5], and llama.cpp v0.3.0 shipped with the KleidiAI build rework that lifts manual kernel-source and `-march` requirements [6][7].

Action items

→ Update ollama to a build with this MLX runner change to get true structured output on Apple Silicon ollama/ollama [immediate]
→ Upgrade llama.cpp to v0.3.0 or pull the prefill-assistant tool-call fix before relying on tool calling ggml-org/llama.cpp [immediate]
→ Monitor vLLM for the cudagraph fix if you hit allocator asserts on multi-GPU setups vllm-project/vllm [plan]

References

[1] mlxrunner: add structured output support ↗ ollama/ollama
[2] llm: remove the unused Grammar completion-request field ollama/ollama
[3] server : reject prefilled assistant messages with tool calls (#27626) ggml-org/llama.cpp
[4] [Deepseek-V4] Enable shared-experts fusion on the flashinfer_mxfp4 (trtllm-gen) MoE path ↗ sgl-project/sglang
[5] [Bugfix][MRV2] Run cudagraph memory profiling in a throwaway graph pool ↗ vllm-project/vllm
[6] v0.3.0 ↗ ggml-org/llama.cpp
[7] kleidiai: Rework KleidiAI Build System/Integration ↗ ggml-org/llama.cpp

Quick answers

What shipped in Local LLMs on August 26, 2026?: Ollama closed a silent correctness gap on Apple Silicon, and the LLM inference stack shipped a wave of fixes that touch everything from tool calls to native ARM support. In total, 116 commits, 113 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 26, 2026?: 18 developers shipped this update, including Jesse Gross, hoyyeva, pd95, Kyozzz, Ryan C, github-actions[bot], allozaur, and JonathanC-ARM, and 10 more.
What were the notable Local LLMs updates?: mlxrunner: add structured output support, llm: remove the unused Grammar completion-request field, and server : reject prefilled assistant messages with tool calls (#27626).

OLLAMA BOLSTERS CLAUDE DESKTOP, VLLM HARDENS AUDIO PATHS

Ollama's desktop app just made Claude Desktop integration faster and safer to manage, while vLLM closed a security gap on audio file size limits.

llms 229 shipped

Local LLMs Aug 24

VLLM'S P2P WEIGHT SYNC CUTS RL MOE TRANSFER COSTS

Forget all-gathering experts during RL rollout: vLLM's new sharded_rdt backend lets each worker pull only the slice it needs, turning a total_bytes transfer into a total_bytes divided by num_workers one.

llms 180 shipped

Local LLMs Aug 23

SGLang Fixes Pixtral 500s as vLLM Cuts TTFT 25%

The biggest news overnight: SGLang is patching a production-breaking Pixtral bug, and vLLM engineers found a way to shave up to 25% off time-to-first-token on Mamba models.

llms 134 shipped

Local LLMs Aug 22

SGLang ships v0.5.18 with 710 PRs and DeepSeek V4 FP4 auto-routing

The biggest release of the week just landed: SGLang v0.5.18 adds new models, fixes a silent disagg bug, and defaults DeepSeek V4 FP4 checkpoints to the right MoE runner.

+11

llms 266 shipped

Elsewhere on the wire

AI Agents Aug 26

CLINE OVERHAULS DESKTOP UX

Cline's desktop app got a massive consolidation pass that merges its customization pages into one hub and reworks the sidebar.

ai-agents 31 shipped

@CachyOS Aug 26

HANDHELD GAMING OVERHAUL: STEAMOS SESSION CONTROL LANDS

CachyOS is rewiring its handheld gaming stack so Steam's session management is treated as a first-class citizen, with a new steamosctl-powered switch to game mode.