RepoJournal

$ cat local-llm/week/2026-08-24.log

Local LLMs

Local LLMs

the week in review · Aug 24 – Aug 30, 2026

vLLM 0.28.0 lands with P2P weight sync, llama.cpp and partners broaden model reach

♥

vLLM 0.28.0 shipped with 584 commits, headlined by a peer-to-peer weight synchronization for RL workloads, while llama.cpp, Ollama, and SGLang pushed new model support and polished integrations.

728 commits 725 PRs merged 67 releases 7 briefings covered
local-llm wire → next week →

vLLM 0.28.0 landed with 584 commits and a focus on performance and RL training. The headline feature is peer-to-peer RDT weight sync, which cuts RL MoE transfer costs. The merge request describes it as "P2P engine-to-engine weight sync" that reduces reliance on centralized parameter servers, a typical bottleneck in multi-engine RL setups. Related work added routed expert loading for gpt-oss, enabling efficient use of large MoE models in distributed inference. [1][2]

Security and robustness got attention across the stack. vLLM now rejects oversized media before fully downloading it, and a fix enforces VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths, closing a gap where some paths bypassed the limit. The project also documented UUID security implications for multimodal media. [3][4][5] llama.cpp fixed a meta tensor split state propagation bug in tensor parallel mode, which could cause incorrect weights on some devices, and addressed a rollback issue with DeepseekV4 in multi-sequence scenarios. [6][7]

Model support expanded across three projects. llama.cpp added support for Qwen3.8-Flash-Next. OpenVINO's backend brought Qwen3.5 to NPUs along with Whisper.cpp support. SGLang fused LongCat-Image QKNorm and interleaved RoPE for diffusion models, and dropped empty assistant turns for mistral_common tokenizers to avoid malformed API responses. [8][9]

Ollama's desktop app series focused on Claude Desktop integration: it added model mappings and an Auto mode setting, improved integration responsiveness, fixed regressions, and made the integrations list scrollable. These land as separate commits but form one coherent effort to stabilize Claude Desktop support. [10][11][12][13][14]

Housekeeping signaled forward momentum. vLLM removed ten deprecated model architectures and cleaned up deprecated parameters, while llama.cpp tuned Metal flash-attention vectors per device for better performance. The grammar parser now treats hyphen in character classes as a literal, fixing a long-standing edge case. [15][16][17][18]

References

  1. [1] [RL] P2P RDT weight sync ↗ vllm-project/vllm
  2. [2] Add routed expert loading for gpt-oss ↗ vllm-project/vllm
  3. [3] Reject oversized media before fully downloading it ↗ vllm-project/vllm
  4. [4] fix(security): enforce VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths (#53561) vllm-project/vllm
  5. [5] [Docs][Security] Document multimodal media UUID security implications (#53582) vllm-project/vllm
  6. [6] [Tensor parallel] Fix meta tensor split state propagation ↗ ggml-org/llama.cpp
  7. [7] DeepseekV4: fix rollback with multi-seq ↗ ggml-org/llama.cpp
  8. [8] [diffusion] Fuse LongCat-Image QKNorm and interleaved RoPE ↗ sgl-project/sglang
  9. [9] [OpenAI] Drop empty assistant turns for mistral_common tokenizers ↗ sgl-project/sglang
  10. [10] app: add Claude Desktop model mappings ↗ ollama/ollama
  11. [11] app: add Claude Desktop Auto mode setting ↗ ollama/ollama
  12. [12] app: improve desktop integration responsiveness ↗ ollama/ollama
  13. [13] app: fix desktop interaction regressions ↗ ollama/ollama
  14. [14] app: make integrations list scrollable ↗ ollama/ollama
  15. [15] metal : per-device tuned (Q, NE) for flash-attn vec (#26570) ↗ ggml-org/llama.cpp
  16. [16] grammar : parse \- in char classes as literal hyphen (#27591) ↗ ggml-org/llama.cpp
  17. [17] [Model] Remove ten deprecated model architectures (#53608) vllm-project/vllm
  18. [18] [MISC] Cleanup deprecated parameters (#53559) vllm-project/vllm

$ ls local-llm/week/ # the briefings behind this review

Mon Aug 24 VLLM'S P2P WEIGHT SYNC CUTS RL MOE TRANSFER COSTS Tue Aug 25 OLLAMA BOLSTERS CLAUDE DESKTOP, VLLM HARDENS AUDIO PATHS Wed Aug 26 OLLAMA'S MLX RUNNER NOW SPEAKS STRUCTURED OUTPUT Thu Aug 27 vLLM 0.28.0 LANDED WITH 584 COMMITS AND A KIMI-K3 PERFORMANCE PUSH Fri Aug 28 LLAMA.CPP UNLEASHES QWEN3.8-FLASH-NEXT SUPPORT Sat Aug 29 OpenVINO backend lands Qwen3.5 on NPU, Whisper.cpp support Sun Aug 30 llama.cpp Vulkan memory bloat fixed, vLLM deprecates PyAV
local-llm wire → next week →