Who contributed to Local LLMs on August 6, 2026?

14 developers shipped this update, including jessegross, Daniel Hiltgen, Xuan-Son Nguyen, Niklas Wenzel, Pascal, SilenNaihin, bnellnm, and Lin-z-w, and 6 more.

What were the notable Local LLMs updates?

mlxrunner: stop cache rewind refills from corrupting later lazy snapshots, server: harden the file_glob_search directory walk, and [Bugfix] Size and iterate w13 by shard count for non-gated MoE.

Local LLMs

Ollama, llama.cpp, vLLM, and SGLang - running and serving open models locally

Pick a date

Topics: Local LLMs AI / ML Full archive →

The Wire · Showcase

OLLAMA FIXES KV CACHE CORRUPTION, LLAMA.CPP HARDENS FILE SEARCH, VLLM KILLS MOE LEGACY CODE

By RepoJournal · Filed 07:02 UTC on August 6, 2026 · About Local LLMs

14 people shipped this

jessegross @jessegross 1 cited

Pascal @ServeurpersoCom 1 cited

SilenNaihin @SilenNaihin 1 cited

aoshen02 @aoshen02 1 cited

BBuf @BBuf 1 cited

mmangkad @mmangkad 1 cited

Daniel Hiltgen @dhiltgen

Xuan-Son Nguyen @ngxson

Niklas Wenzel @nikwen

bnellnm @bnellnm

Lin-z-w @Lin-z-w

Tianyu Guo @gty111

DevashishLal-CB @DevashishLal-CB

hnyls2002 @hnyls2002

Ollama patched a critical cache corruption bug that corrupts lazy snapshots during multi-request prefills, while llama.cpp hardened Windows junction traversal and vLLM removed 14 hardcoded MoE shards across quantization methods.

Ollama's mlxrunner was silently overwriting lazy KV snapshots when later appends reached their buffer slots, corrupting cached conversations on request reuse [1]. The fix ensures snapshots are copied before any append that touches their range, not just the first one. In parallel, llama.cpp tightened file_glob_search to skip Windows junctions that std::filesystem reports as directories, preventing infinite loops when a junction points to an ancestor [2]. The patch also adds case-insensitive junk directory matching on Windows and surfaces incomplete results instead of silent failures. Over on vLLM, a major MoE refactor removes 14 quantization methods that hardcoded w13 shard counts to 2, where non-gated MoE (e.g. NemotronH) only fuses one shard [3]. Separately, vLLM fixed level-2 sleep/wake/reload for LoRA models by restructuring BaseLayerWithLoRA to preserve checkpoint semantics after memory discard [4]. SGLang pushed forward on diffusion kernels, preferring cuDNN SDPA over FlashAttention-4 on Blackwell for 1.81x speedups [5], while adding ModelOpt MXFP8 checkpoint support and unifying BaseFusedOp dispatch across backends [6]. Across 4 repos, 115 commits and 10 releases shipped this period.

Action items

→ Update Ollama immediately if you cache long conversations with short prefixes ollama/ollama [immediate]
→ Upgrade llama.cpp if serving from Windows directories with deep hierarchies ggml-org/llama.cpp [plan]
→ Patch vLLM MoE quantization methods if using non-gated MoE models vllm-project/vllm [plan]
→ Monitor SGLang B200 performance gains on your Blackwell hardware sgl-project/sglang [monitor]

References

[1] mlxrunner: stop cache rewind refills from corrupting later lazy snapshots ollama/ollama
[2] server: harden the file_glob_search directory walk ↗ ggml-org/llama.cpp
[3] [Bugfix] Size and iterate w13 by shard count for non-gated MoE ↗ vllm-project/vllm
[4] [Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True ↗ vllm-project/vllm
[5] [diffusion] Prefer cuDNN SDPA over FA4 for dense attention on sm_100 (B200) ↗ sgl-project/sglang
[6] Support ModelOpt MXFP8 checkpoints ↗ sgl-project/sglang

Quick answers

What shipped in Local LLMs on August 6, 2026?: Ollama patched a critical cache corruption bug that corrupts lazy snapshots during multi-request prefills, while llama.cpp hardened Windows junction traversal and vLLM removed 14 hardcoded MoE shards across quantization methods. In total, 115 commits, 115 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 6, 2026?: 14 developers shipped this update, including jessegross, Daniel Hiltgen, Xuan-Son Nguyen, Niklas Wenzel, Pascal, SilenNaihin, bnellnm, and Lin-z-w, and 6 more.
What were the notable Local LLMs updates?: mlxrunner: stop cache rewind refills from corrupting later lazy snapshots, server: harden the file_glob_search directory walk, and [Bugfix] Size and iterate w13 by shard count for non-gated MoE.

OLLAMA FIXES RACE CONDITIONS AS LLAMA.CPP SHIPS BREAKING TTS CHANGES

Ollama patched critical data races across its progress tracking system while llama.cpp introduces incompatible Qwen3-TTS support, and vLLM lands kernel optimizations that cut latency on specialized hardware.

+11

llms 246 shipped

Local LLMs Aug 4

OLLAMA REWIRES OPENAI COMPATIBILITY, LLAMA.CPP HARDENS SAMPLING

Ollama reworked its entire streaming protocol to match OpenAI's wire format chunk-for-chunk, unblocking drop-in replacement deployments.

+11

llms 230 shipped

Local LLMs Aug 3

DEEPSEEK V4 INFERENCE GETS 50% SPEEDUP; LLAMA.CPP AND VLLM BOTH SHIPPING OPTIMIZED PATHS

DeepSeek V4 speculative decode just went from nice-to-have to standard: llama.cpp landed MTP and DSpark support with roughly 50% speedup [ref:3], vLLM hardened its KV cache coordinator for the same models [ref:7], and Metal got native hyper-connection ops [ref:5].

llms 148 shipped

Local LLMs Aug 2

LLAMA.CPP SHIPS FOUR BUILDS IN 24 HOURS AS VLLM AND SGLANG RACE ON SEQUENCE PARALLELISM

The local LLM stack is moving fast: llama.cpp pushed builds b10219-b10223 with reasoning persistence and security updates, while vLLM landed sequence parallelism for DeepSeek V4 and sglang unified its radix cache for multi-turn agentic work.

llms 82 shipped

Elsewhere on the wire

AI Agents Aug 6

LANGCHAIN HARDENS ANTHROPIC TOOL CALLING AND GIT CREDENTIALS AS OPENHANDS SHIPS 1.10.0

LangChain fixed a critical credential leak in evaluation runs while Anthropic tool schemas get proper fallback handling, as three major agent frameworks push releases across the ecosystem.

ai-agents 54 shipped

@CachyOS Aug 6

WIKI TRANSLATION ROLLBACK CHAOS, MIRROR STACK HOLDING STEADY

CachyOS wiki maintainers are caught in a revert loop over German translations while the status infrastructure logs 55 commits without incident.

infra 57 shipped

Elixir & Phoenix Aug 6

ELIXIR BUMPS ZIZMOR SECURITY SCANNER TO 1.28.0

The Elixir repo updated its CI security action overnight, pulling in the latest zizmor static analysis tooling.

elixir 1 shipped

@dotnet Aug 6

DOTNET FOUNDATION PULLS iOS METRICS INTO FOCUS AS RUNTIME HARDENS HTTP PARSING

Performance team fixed iOS size reporting to measure only distributable packages, while runtime clamped down on RFC 9112 compliance for chunked encoding.

dotnet 72 shipped

Want every project, not just this one?

Follow Local LLMs