RepoJournal
Local LLMs

Local LLMs

Ollama, llama.cpp, vLLM, and SGLang - running and serving open models locally

Keep up with Local LLMs in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

One email a day. Unsubscribe in one click.

Pick a date

  1. Thu 23 jul
  2. Fri 24 jul
  3. Sat 25 jul
  4. Sun 26 jul
  5. Mon 27 jul
  6. Tue 28 jul
  7. Wed 29 jul
  8. Thu 30 jul
  9. Fri 31 jul
  10. Sat 1 aug
  11. Sun 2 aug
  12. Mon 3 aug
  13. Tue 4 aug
  14. Wed 5 aug
  15. Today 6 aug
Topics: Local LLMs AI / ML Full archive →

The Wire · Showcase

DEEPSEEK V4 INFERENCE GETS 50% SPEEDUP; LLAMA.CPP AND VLLM BOTH SHIPPING OPTIMIZED PATHS

By RepoJournal · Filed · About Local LLMs

am17an hnyls2002 Talha Adnan aldehir ggerganov bigPYJ1151 ivanium
12 people shipped this
am17an @am17an 2 cited hnyls2002 @hnyls2002 2 cited Talha Adnan @Blackcyan30 1 cited aldehir @aldehir 1 cited ggerganov @ggerganov 1 cited bigPYJ1151 @bigPYJ1151 1 cited ivanium @ivanium 1 cited omkar-droid @omkar-droid 1 cited alisonshao @alisonshao 1 cited jasonlizhengjian @jasonlizhengjian Itay Alroy @itayalroy sherlockwu @sherlockwu

DeepSeek V4 speculative decode just went from nice-to-have to standard: llama.cpp landed MTP and DSpark support with roughly 50% speedup [ref:3], vLLM hardened its KV cache coordinator for the same models [ref:7], and Metal got native hyper-connection ops [ref:5].

This is the biggest single inference win across the stack in months. The llama.cpp DSpark path [2] lets you export separate draft heads for speculative decoding, while the Metal implementation of DeepSeek V4's specialized hyper-connection operations [3] unlocks GPU-native inference without performance cliffs. vLLM's KV connector bugfix [7] was the linchpin: it correctly propagates EAGLE/MTP state across merged cache groups, fixing a subtle coordinator failure that would have silently tanked throughput on multi-GPU setups. On the parallel track, vLLM shipped a CPU MoE migration to modular-kernel experts [6], eliminating three legacy code paths and unifying quantized and unquantized backends. llama.cpp also added SILU backward ops for Metal [4], and Qwen3 chat parsing now handles both `</think>` and `<tool_call>` reasoning delimiters [5]. SGLang landed DSPARK sampling support in graph-folded mode [9], broadcast-optimized vision embeddings [10], and hardened its spec fixture teardown [11], while vLLM added jina-embeddings-v5 support with encoder-only dispatch [8]. Both projects shipped 69 commits and 69 PRs across 10 releases in the last 24 hours.

One email a day. Unsubscribe in one click.

Action items

References

  1. [1] DeepseekV4 MTP + DSpark ↗ ggml-org/llama.cpp
  2. [2] convert: add option to create separate dspark GGUF ↗ ggml-org/llama.cpp
  3. [3] metal: implement DeepSeek V4 hyper-connections ↗ ggml-org/llama.cpp
  4. [4] metal : add SILU_BACK (#25982) ggml-org/llama.cpp
  5. [5] chat : add qwen3 specialized parser ↗ ggml-org/llama.cpp
  6. [6] [CPU] Migrate unquantized MoE to the modular-kernel experts structure ↗ vllm-project/vllm
  7. [7] [Bugfix][KV Connector] Propagate EAGLE state across merged Mooncake store groups ↗ vllm-project/vllm
  8. [8] [Model] Support jina-embeddings-v5-text-nano (EuroBERT encoder backbone) ↗ vllm-project/vllm
  9. [9] [Spec] Support sampling in the DSPARK graph-folded draft proposal ↗ sgl-project/sglang
  10. [10] [Perf] Broadcast single-image DP vision embedding instead of pad-to-max all-gather ↗ sgl-project/sglang
  11. [11] [CI] Graceful teardown for kv_canary and EAGLE spec fixtures ↗ sgl-project/sglang

Quick answers

What shipped in Local LLMs on August 3, 2026?
DeepSeek V4 speculative decode just went from nice-to-have to standard: llama.cpp landed MTP and DSpark support with roughly 50% speedup [ref:3], vLLM hardened its KV cache coordinator for the same models [ref:7], and Metal got native hyper-connection ops [ref:5]. In total, 69 commits, 69 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 3, 2026?
12 developers shipped this update, including Talha Adnan, aldehir, am17an, ggerganov, bigPYJ1151, ivanium, omkar-droid, and jasonlizhengjian, and 4 more.
What were the notable Local LLMs updates?
DeepseekV4 MTP + DSpark, convert: add option to create separate dspark GGUF, and metal: implement DeepSeek V4 hyper-connections.

More from Local LLMs

Local LLMs Local LLMs

OLLAMA FIXES KV CACHE CORRUPTION, LLAMA.CPP HARDENS FILE SEARCH, VLLM KILLS MOE LEGACY CODE

Ollama patched a critical cache corruption bug that corrupts lazy snapshots during multi-request prefills, while llama.cpp hardened Windows junction traversal and vLLM removed 14 hardcoded MoE shards across quantization methods.

jessegross Daniel Hiltgen Xuan-Son Nguyen Niklas Wenzel +10
llms 240 shipped
Local LLMs Local LLMs

OLLAMA FIXES RACE CONDITIONS AS LLAMA.CPP SHIPS BREAKING TTS CHANGES

Ollama patched critical data races across its progress tracking system while llama.cpp introduces incompatible Qwen3-TTS support, and vLLM lands kernel optimizations that cut latency on specialized hardware.

dhiltgen github-actions[bot] Xuan-Son Nguyen Evan Huus +11
llms 246 shipped
Local LLMs Local LLMs

OLLAMA REWIRES OPENAI COMPATIBILITY, LLAMA.CPP HARDENS SAMPLING

Ollama reworked its entire streaming protocol to match OpenAI's wire format chunk-for-chunk, unblocking drop-in replacement deployments.

BruceMacD Konrad Moren Xuan-Son Nguyen Pascal +11
llms 230 shipped
Local LLMs Local LLMs

LLAMA.CPP SHIPS FOUR BUILDS IN 24 HOURS AS VLLM AND SGLANG RACE ON SEQUENCE PARALLELISM

The local LLM stack is moving fast: llama.cpp pushed builds b10219-b10223 with reasoning persistence and security updates, while vLLM landed sequence parallelism for DeepSeek V4 and sglang unified its radix cache for multi-turn agentic work.

Xuan-Son Nguyen github-actions[bot] Isotr0py WoosukKwon +6
llms 82 shipped

Daily updates, in your inbox

Follow Local LLMs

Keep up with Local LLMs in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

One email a day. Unsubscribe in one click. Read a past issue →

Elsewhere on the wire

AI Agents AI Agents

LANGCHAIN HARDENS ANTHROPIC TOOL CALLING AND GIT CREDENTIALS AS OPENHANDS SHIPS 1.10.0

LangChain fixed a critical credential leak in evaluation runs while Anthropic tool schemas get proper fallback handling, as three major agent frameworks push releases across the ecosystem.

langchain-oss-model-profiles[bot] Mason Daugherty John Kennedy ccurme +7
ai-agents 54 shipped
CachyOS @CachyOS

WIKI TRANSLATION ROLLBACK CHAOS, MIRROR STACK HOLDING STEADY

CachyOS wiki maintainers are caught in a revert loop over German translations while the status infrastructure logs 55 commits without incident.

Jinra Aarrayy buster-biopsys
infra 57 shipped
Elixir & Phoenix Elixir & Phoenix

ELIXIR BUMPS ZIZMOR SECURITY SCANNER TO 1.28.0

The Elixir repo updated its CI security action overnight, pulling in the latest zizmor static analysis tooling.

dependabot
elixir 1 shipped
.NET @dotnet

DOTNET FOUNDATION PULLS iOS METRICS INTO FOCUS AS RUNTIME HARDENS HTTP PARSING

Performance team fixed iOS size reporting to measure only distributable packages, while runtime clamped down on RFC 9112 compliance for chunked encoding.

matouskozak dotnet-maestro Tomas Weinfurt Adeel Mujahid +4
dotnet 72 shipped
Want every project, not just this one?