Who contributed to Local LLMs on August 1, 2026?

16 developers shipped this update, including drifkin, jessegross, dhiltgen, arthw, johnkarlhill, Masato Nakasaka, fairydreaming, and MatthewBonanni, and 8 more.

What were the notable Local LLMs updates?

mlx: qwen3.5 MTP speculative decoding, cmd: suggest :cloud when a model has no default tag, and llama : load MTP tensors only if they are really used.

Local LLMs

Ollama, llama.cpp, vLLM, and SGLang - running and serving open models locally

Pick a date

Topics: Local LLMs AI / ML Full archive →

The Wire · Showcase

QWEN3.5 SPECULATIVE DECODING HITS 70% SPEEDUP ON LLAMA.CPP, OLLAMA ROUTES CLOUD MODELS

By RepoJournal · Filed 07:01 UTC on August 1, 2026 · About Local LLMs

16 people shipped this

drifkin @drifkin 1 cited

jessegross @jessegross 1 cited

fairydreaming @fairydreaming 1 cited

MatthewBonanni @MatthewBonanni 1 cited

ch-wan @ch-wan 1 cited

nanjiangwill @nanjiangwill 1 cited

dhiltgen @dhiltgen

arthw @arthw

johnkarlhill @johnkarlhill

Masato Nakasaka @rillomas

fxmarty-amd @fxmarty-amd

yewentao256 @yewentao256

NickLucche @NickLucche

kpham-sgl @kpham-sgl

mickqian @mickqian

luoroger37 @luoroger37

Qwen3.5's MTP head is now live across both Ollama and llama.cpp, delivering 24-70% inference speedup on dense models while Ollama adds smart cloud-model routing to end user confusion.

The speculative decoding win spans two major inference engines. Ollama loaded the MTP head for qwen3.5 and wired it as an automatic draft model in the MLX engine [1], pushing the 35B MoE from 131 to 162 tokens/s on code (24% faster) and the dense 27B from 31 to 53 tokens/s (70% speedup). Llama.cpp followed with its own MTP tensor loading [3], but added a load_mtp flag to skip MTP overhead when speculative decoding isn't requested. In parallel, Ollama shipped a UX fix [2] that probes for :cloud variants when a model pull fails, suggesting kimi-k3:cloud to users at the terminal instead of leaving them with a cryptic file-not-found error. On the vLLM side, the team landed a block-table alignment fix [4] that unifies width calculation across three indexer paths, preventing buffer mismatches under deep context packing. Across 4 repos, the stack logged 110 commits, 104 PRs, and 10 releases in 24 hours. SGLang is now routing Kimi Linear through paged MLA backends [5] and shipping fastsafetensors without GDS for hosts lacking the NVIDIA driver [6].

Action items

→ Pull Ollama latest and test qwen3.5 inference on M-series; measure your token/s before and after ollama/ollama [monitor]
→ Rebuild llama.cpp with MTP support and verify --spec-type draft-mtp on Qwen models in your pipeline ggml-org/llama.cpp [plan]
→ If running vLLM with deep context packing, upgrade to pick up block-table alignment fix immediately vllm-project/vllm [immediate]
→ SGLang users: test fastsafetensors no-GDS loading if your host lacks NVIDIA GPUDirect Storage driver sgl-project/sglang [monitor]

References

[1] mlx: qwen3.5 MTP speculative decoding ↗ ollama/ollama
[2] cmd: suggest :cloud when a model has no default tag ↗ ollama/ollama
[3] llama : load MTP tensors only if they are really used ↗ ggml-org/llama.cpp
[4] [Bugfix] Universally align block table width to 128 tokens ↗ vllm-project/vllm
[5] [unified-memory] Let Kimi-Linear use the paged MLA attention backends ↗ sgl-project/sglang
[6] Support fastsafetensors no-GDS loading and page-cache release ↗ sgl-project/sglang

Quick answers

What shipped in Local LLMs on August 1, 2026?: Qwen3.5's MTP head is now live across both Ollama and llama.cpp, delivering 24-70% inference speedup on dense models while Ollama adds smart cloud-model routing to end user confusion. In total, 110 commits, 104 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 1, 2026?: 16 developers shipped this update, including drifkin, jessegross, dhiltgen, arthw, johnkarlhill, Masato Nakasaka, fairydreaming, and MatthewBonanni, and 8 more.
What were the notable Local LLMs updates?: mlx: qwen3.5 MTP speculative decoding, cmd: suggest :cloud when a model has no default tag, and llama : load MTP tensors only if they are really used.

OLLAMA FIXES KV CACHE CORRUPTION, LLAMA.CPP HARDENS FILE SEARCH, VLLM KILLS MOE LEGACY CODE

Ollama patched a critical cache corruption bug that corrupts lazy snapshots during multi-request prefills, while llama.cpp hardened Windows junction traversal and vLLM removed 14 hardcoded MoE shards across quantization methods.

+10

llms 240 shipped

Local LLMs Aug 5

OLLAMA FIXES RACE CONDITIONS AS LLAMA.CPP SHIPS BREAKING TTS CHANGES

Ollama patched critical data races across its progress tracking system while llama.cpp introduces incompatible Qwen3-TTS support, and vLLM lands kernel optimizations that cut latency on specialized hardware.

+11

llms 246 shipped

Local LLMs Aug 4

OLLAMA REWIRES OPENAI COMPATIBILITY, LLAMA.CPP HARDENS SAMPLING

Ollama reworked its entire streaming protocol to match OpenAI's wire format chunk-for-chunk, unblocking drop-in replacement deployments.

+11

llms 230 shipped

Local LLMs Aug 3

DEEPSEEK V4 INFERENCE GETS 50% SPEEDUP; LLAMA.CPP AND VLLM BOTH SHIPPING OPTIMIZED PATHS

DeepSeek V4 speculative decode just went from nice-to-have to standard: llama.cpp landed MTP and DSpark support with roughly 50% speedup [ref:3], vLLM hardened its KV cache coordinator for the same models [ref:7], and Metal got native hyper-connection ops [ref:5].

llms 148 shipped

Elsewhere on the wire

AI Agents Aug 6

LANGCHAIN HARDENS ANTHROPIC TOOL CALLING AND GIT CREDENTIALS AS OPENHANDS SHIPS 1.10.0

LangChain fixed a critical credential leak in evaluation runs while Anthropic tool schemas get proper fallback handling, as three major agent frameworks push releases across the ecosystem.

ai-agents 54 shipped

@CachyOS Aug 6

WIKI TRANSLATION ROLLBACK CHAOS, MIRROR STACK HOLDING STEADY

CachyOS wiki maintainers are caught in a revert loop over German translations while the status infrastructure logs 55 commits without incident.

infra 57 shipped

Elixir & Phoenix Aug 6

ELIXIR BUMPS ZIZMOR SECURITY SCANNER TO 1.28.0

The Elixir repo updated its CI security action overnight, pulling in the latest zizmor static analysis tooling.

elixir 1 shipped

@dotnet Aug 6

DOTNET FOUNDATION PULLS iOS METRICS INTO FOCUS AS RUNTIME HARDENS HTTP PARSING

Performance team fixed iOS size reporting to measure only distributable packages, while runtime clamped down on RFC 9112 compliance for chunked encoding.

dotnet 72 shipped

Want every project, not just this one?

Follow Local LLMs