RepoJournal
Local LLMs

Local LLMs

Ollama, llama.cpp, vLLM, and SGLang - running and serving open models locally

Keep up with Local LLMs in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

One email a day. Unsubscribe in one click.

Pick a date

  1. Thu 23 jul
  2. Fri 24 jul
  3. Sat 25 jul
  4. Sun 26 jul
  5. Mon 27 jul
  6. Tue 28 jul
  7. Wed 29 jul
  8. Thu 30 jul
  9. Fri 31 jul
  10. Sat 1 aug
  11. Sun 2 aug
  12. Mon 3 aug
  13. Tue 4 aug
  14. Wed 5 aug
  15. Today 6 aug
Topics: Local LLMs AI / ML Full archive →

The Wire · Showcase

OLLAMA REWIRES OPENAI COMPATIBILITY, LLAMA.CPP HARDENS SAMPLING

By RepoJournal · Filed · About Local LLMs

BruceMacD Konrad Moren Xuan-Son Nguyen Pascal Juan Pérez de Algaba aoshen02 BBuf
15 people shipped this
BruceMacD @BruceMacD 1 cited Konrad Moren @kmorennv 1 cited Xuan-Son Nguyen @ngxson 1 cited Pascal @ServeurpersoCom 1 cited Juan Pérez de Algaba @jperezdealgaba 1 cited aoshen02 @aoshen02 1 cited BBuf @BBuf 1 cited Thiago Padilha @tarruda Ozymandias_EBON @johnkarlhill TQCB @TQCB Jiangyun Zhu @ZJY0516 fuscof-ibm @fuscof-ibm hnyls2002 @hnyls2002 Fridge003 @Fridge003 DarkSharpness @DarkSharpness

Ollama reworked its entire streaming protocol to match OpenAI's wire format chunk-for-chunk, unblocking drop-in replacement deployments.

Ollama's `/v1/chat/completions` endpoint now sends `finish_reason` on its own chunk with an empty delta instead of riding the last content chunk, matching real OpenAI traffic captures [1]. The move closes compatibility gaps that broke clients expecting OpenAI's exact stream behavior. Separately, llama.cpp extended CUDA penalty handling to the backend sampler layer [2], hardening frequency and presence adjustments while llama.cpp's server gained introspective tooling to report system info and OS diagnostics [3]. Model resolution testing in llama.cpp now covers shard fallback and preset loading end-to-end on synthetic repo layouts without network access [4]. On the inference side, vLLM tightened security by classifying DeepStream as a GPU backend and enforcing pixel limits [5], while routed-experts capture returned to Model Runner V2, preserving backward compatibility with MRV1 [6]. SGLang's diffusion component-accuracy harness rebuilt model-parallel groups when Ulysses/Ring decomposition changed, fixing topology reuse bugs that persisted across unchanged total SP [7]. The period saw 110 commits and 110 PRs across four repos, with 10 releases shipping.

One email a day. Unsubscribe in one click.

Action items

References

  1. [1] openai: match openai's streaming wire format for chat completions ↗ ollama/ollama
  2. [2] CUDA: Add backend sampler for penalties sampler (#25262) ggml-org/llama.cpp
  3. [3] server: add get_info tool (#26522) ggml-org/llama.cpp
  4. [4] tests: add model resolution test on synthetic repo listings (#26172) ggml-org/llama.cpp
  5. [5] fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755) vllm-project/vllm
  6. [6] [MRV2] Enable routed-experts capture ↗ vllm-project/vllm
  7. [7] [diffusion] Fix component accuracy topology reuse ↗ sgl-project/sglang

Quick answers

What shipped in Local LLMs on August 4, 2026?
Ollama reworked its entire streaming protocol to match OpenAI's wire format chunk-for-chunk, unblocking drop-in replacement deployments. In total, 110 commits, 110 pull requests, and 10 releases landed.
Who contributed to Local LLMs on August 4, 2026?
15 developers shipped this update, including BruceMacD, Konrad Moren, Xuan-Son Nguyen, Pascal, Thiago Padilha, Ozymandias_EBON, Juan Pérez de Algaba, and TQCB, and 7 more.
What were the notable Local LLMs updates?
openai: match openai's streaming wire format for chat completions, CUDA: Add backend sampler for penalties sampler (#25262), and server: add get_info tool (#26522).

More from Local LLMs

Local LLMs Local LLMs

OLLAMA FIXES KV CACHE CORRUPTION, LLAMA.CPP HARDENS FILE SEARCH, VLLM KILLS MOE LEGACY CODE

Ollama patched a critical cache corruption bug that corrupts lazy snapshots during multi-request prefills, while llama.cpp hardened Windows junction traversal and vLLM removed 14 hardcoded MoE shards across quantization methods.

jessegross Daniel Hiltgen Xuan-Son Nguyen Niklas Wenzel +10
llms 240 shipped
Local LLMs Local LLMs

OLLAMA FIXES RACE CONDITIONS AS LLAMA.CPP SHIPS BREAKING TTS CHANGES

Ollama patched critical data races across its progress tracking system while llama.cpp introduces incompatible Qwen3-TTS support, and vLLM lands kernel optimizations that cut latency on specialized hardware.

dhiltgen github-actions[bot] Xuan-Son Nguyen Evan Huus +11
llms 246 shipped
Local LLMs Local LLMs

DEEPSEEK V4 INFERENCE GETS 50% SPEEDUP; LLAMA.CPP AND VLLM BOTH SHIPPING OPTIMIZED PATHS

DeepSeek V4 speculative decode just went from nice-to-have to standard: llama.cpp landed MTP and DSpark support with roughly 50% speedup [ref:3], vLLM hardened its KV cache coordinator for the same models [ref:7], and Metal got native hyper-connection ops [ref:5].

Talha Adnan aldehir am17an ggerganov +8
llms 148 shipped
Local LLMs Local LLMs

LLAMA.CPP SHIPS FOUR BUILDS IN 24 HOURS AS VLLM AND SGLANG RACE ON SEQUENCE PARALLELISM

The local LLM stack is moving fast: llama.cpp pushed builds b10219-b10223 with reasoning persistence and security updates, while vLLM landed sequence parallelism for DeepSeek V4 and sglang unified its radix cache for multi-turn agentic work.

Xuan-Son Nguyen github-actions[bot] Isotr0py WoosukKwon +6
llms 82 shipped

Daily updates, in your inbox

Follow Local LLMs

Keep up with Local LLMs in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

One email a day. Unsubscribe in one click. Read a past issue →

Elsewhere on the wire

AI Agents AI Agents

LANGCHAIN HARDENS ANTHROPIC TOOL CALLING AND GIT CREDENTIALS AS OPENHANDS SHIPS 1.10.0

LangChain fixed a critical credential leak in evaluation runs while Anthropic tool schemas get proper fallback handling, as three major agent frameworks push releases across the ecosystem.

langchain-oss-model-profiles[bot] Mason Daugherty John Kennedy ccurme +7
ai-agents 54 shipped
CachyOS @CachyOS

WIKI TRANSLATION ROLLBACK CHAOS, MIRROR STACK HOLDING STEADY

CachyOS wiki maintainers are caught in a revert loop over German translations while the status infrastructure logs 55 commits without incident.

Jinra Aarrayy buster-biopsys
infra 57 shipped
Elixir & Phoenix Elixir & Phoenix

ELIXIR BUMPS ZIZMOR SECURITY SCANNER TO 1.28.0

The Elixir repo updated its CI security action overnight, pulling in the latest zizmor static analysis tooling.

dependabot
elixir 1 shipped
.NET @dotnet

DOTNET FOUNDATION PULLS iOS METRICS INTO FOCUS AS RUNTIME HARDENS HTTP PARSING

Performance team fixed iOS size reporting to measure only distributable packages, while runtime clamped down on RFC 9112 compliance for chunked encoding.

matouskozak dotnet-maestro Tomas Weinfurt Adeel Mujahid +4
dotnet 72 shipped
Want every project, not just this one?