$ tail -f topics/llms.log
Daily updates from the local-LLM inference stack - Ollama, llama.cpp, vLLM, and SGLang. What shipped this week in running and serving open models on your own hardware.
11 updates across 2 projects this week.
One calm review of what shipped across Local LLMs - the commits, releases, and security advisories that matter. Every Monday, with security advisories same-day. Free, unsubscribe in one click.
We'll start you on the top Local LLMs projects - refine anytime. · Read a sample issue →
Local LLMs
The biggest release of the week just landed: SGLang v0.5.18 adds new models, fixes a silent disagg bug, and defaults DeepSeek V4 FP4 checkpoints to the right MoE runner.
read --wire →
Local LLMs
The biggest local-LLM repos are moving fast: DeepSeek V4's MoE kernel gets a massive fusion, while vLLM patches a security hole.
read --wire →
Local LLMs
A mid-stream parser error in Ollama's server silently wedged chat and generate requests, leaking goroutines and hanging retries until a fix landed overnight.
read --wire →
Local LLMs
llama.cpp lands full BailingMoE3 support with speculative decoding, giving developers a native path for a key emerging sparse MoE family.
read --wire →
Local LLMs
Across 87 commits and 5 releases, the local LLM stack is grinding through the unglamorous work that makes inference faster: killing dead code, fixing kernel launches, and removing synchronization bottlenecks.
read --wire →
Local LLMs
llama.cpp deprecates mmap flags in favor of unified load-mode, while vllm fixes critical issues blocking DeepSeek-V4-Flash across all decode paths.
read --wire →
Local LLMs
Ollama shipped full Qwen3.8 support overnight, adding reasoning-effort semantics and a critical fix for coding agents that send developer instructions before user messages.
read --wire →
Local LLMs
Ollama just opened its launch ecosystem to DeepSeek Harness and Meta's Muse Code, while vLLM pivots to HuggingFace transformers as a unified backend for tail models.
read --wire →
Local LLMs
A rogue OCI registry could exploit duplicate digest collisions in Ollama's blob verification to exfiltrate data from internal endpoints.
read --wire →
Local LLMs
Ollama kills a decade of default penalty cruft that was drowning every model, while llama.cpp gains multimodal audio chops and vLLM fixes ROCm speculative decoding corruption.
read --wire →
Local LLMs
Ollama shipped v0.32.7 with a critical fix for boundary token corruption in model invocation, while full MLX support for Nemotron 3 Nano Omni brings self-draft speculation to inference without a separate draft model.
read --wire →
Local LLMs
Ollama's MLX runner now processes images natively, preserving prefix caching and speculative decoding while rejecting unsupported requests with clear errors.
read --wire →
Local LLMs
Ollama eliminated redundant manifest data ballooning config blobs on safetensors models, while llama.cpp landed docker-backed tool isolation and fused three GPU kernels into one.
read --wire →
Local LLMs
Draft models just got faster: Ollama merged DFlash block-diffusion speculative decoding that proposes entire token sequences in one forward pass, while SGLang shipped production-ready support for Kimi K3, a 2.8T-parameter multimodal LatentMoE that routes through a latent space with 896 experts.
read --wire →
Local LLMs
Vulkan submission batching and AMD ROCm CI fixes land in llama.cpp as vLLM optimizes attention kernels and sglang adds per-request MLA scheduling.
read --wire →
Local LLMs
Ollama patched a critical cache corruption bug that corrupts lazy snapshots during multi-request prefills, while llama.cpp hardened Windows junction traversal and vLLM removed 14 hardcoded MoE shards across quantization methods.
read --wire →
Local LLMs
Ollama patched critical data races across its progress tracking system while llama.cpp introduces incompatible Qwen3-TTS support, and vLLM lands kernel optimizations that cut latency on specialized hardware.
read --wire →
Local LLMs
Ollama reworked its entire streaming protocol to match OpenAI's wire format chunk-for-chunk, unblocking drop-in replacement deployments.
read --wire →
Local LLMs
DeepSeek V4 speculative decode just went from nice-to-have to standard: llama.cpp landed MTP and DSpark support with roughly 50% speedup [ref:3], vLLM hardened its KV cache coordinator for the same models [ref:7], and Metal got native hyper-connection ops [ref:5].
read --wire →
Local LLMs
The local LLM stack is moving fast: llama.cpp pushed builds b10219-b10223 with reasoning persistence and security updates, while vLLM landed sequence parallelism for DeepSeek V4 and sglang unified its radix cache for multi-turn agentic work.
read --wire →
