RepoJournal
Local LLMs Local LLMs
46 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-09-02
stories 277
RSS Archive Topics

© 2026 RepoJournal Home Showcase Explore How it works Privacy

$ the-wire · showcase

Ollama honors model generation defaults; llama.cpp and vLLM ship MoE performance fixes

By RepoJournal · Filed · About Local LLMs

dhiltgen Titaniumtown philip-jingxin anujj khluu gau-nernst DarkSharpness
16 people shipped this
dhiltgen @dhiltgen 3 cited Titaniumtown @Titaniumtown 1 cited philip-jingxin @philip-jingxin 1 cited anujj @anujj 1 cited khluu @khluu 1 cited gau-nernst @gau-nernst 1 cited DarkSharpness @DarkSharpness 1 cited BBuf @BBuf 1 cited ServeurpersoCom @ServeurpersoCom lstolcman @lstolcman Nick Hill @njhill BugenZhao @BugenZhao mingg26 @mingg26 ch-wan @ch-wan JustinTong0323 @JustinTong0323 cctry @cctry

Ollama now respects model-authored sampler defaults from GGUF metadata, and both llama.cpp and vLLM land performance and correctness fixes for MoE kernels.

Ollama now parses model-authored sampler defaults from GGUF metadata and HF generation_config.json and applies them before Modelfile and request options, making the precedence: API parameters, Modelfile Parameters, GGUF KVs or MLX generation_config.json, then Ollama general default [1]. On the MLX front, a patch is no longer needed after the MLX and MLX-C version bumps [2], and CI now wires up MLX unit tests for PR runs by downloading the matching Darwin release payload [3]. llama.cpp adds CUDA fusion for the MoE weighted expert reduction, reducing launch count from two kernels to one and removing intermediate global-memory traffic [4], while a SyCL FWHT feature that was reverted due to breaking GitHub Actions [5] is now back with support for non-power-of-two embedding dimensions 384, 640, 768, and 1280 [6]. vLLM fixes a bug where padded routes in CUTLASS MoE permutations caused crashes in Qwen3-30B NVFP4 configs [7], and separates prefill and decode paths for its QSA indexer to enable specialized kernels [8]. SGLang removes its single-purpose dsv3_router_gemm kernel in favor of the unified tiny GEMM [9] and adds tuned Triton MoE configs for LFM2.5 on B300, noting that the generic Triton path was faster for medium/large token counts [10].

Action items

References

  1. [1] Honor model generation defaults ↗ ollama/ollama
  2. [2] MLX, MLX-C: version bump ↗ ollama/ollama
  3. [3] ci: wire up MLX unit tests for PR runs ↗ ollama/ollama
  4. [4] cuda: fuse MoE weighted expert reduction ↗ ggml-org/llama.cpp
  5. [5] Revert "sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280" ↗ ggml-org/llama.cpp
  6. [6] sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280 ↗ ggml-org/llama.cpp
  7. [7] [Bugfix] Handle padded routes in CUTLASS MoE permutations ↗ vllm-project/vllm
  8. [8] [Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer ↗ vllm-project/vllm
  9. [9] [Kernel] Replace dsv3_router_gemm with the unified tiny GEMM ↗ sgl-project/sglang
  10. [10] [Kernel] Add tuned LFM2.5 Triton MoE configs on B300 ↗ sgl-project/sglang

$ ls local-llm/reviews/ # the step-back reads

the week in review Vulkan top_k radix select arrives for long rows the month in review Speculative decoding consolidates across the stack as DeepSeek V4 lands
Topics: Local LLMs AI / ML Full archive →

Quick answers

What shipped in Local LLMs on September 2, 2026?
Ollama now respects model-authored sampler defaults from GGUF metadata, and both llama.cpp and vLLM land performance and correctness fixes for MoE kernels. In total, 133 commits, 133 pull requests, and 11 releases landed.
Who contributed to Local LLMs on September 2, 2026?
16 developers shipped this update, including dhiltgen, Titaniumtown, philip-jingxin, ServeurpersoCom, anujj, lstolcman, khluu, and gau-nernst, and 8 more.
What were the notable Local LLMs updates?
Honor model generation defaults, MLX, MLX-C: version bump, and ci: wire up MLX unit tests for PR runs.
♥

$ log Local LLMs --recent

Vulkan adds TQ1_0, Spark2.5 lands in llama.cpp 180 shipped vLLM closes scale-out multimodal handoff hole 86 shipped vLLM Fast Start maps weights zero-copy; GLMGA sampling capped 256 shipped Ollama enables speculative decoding under structured output, vLLM adds LoRA and fusion 259 shipped

$ log --all # elsewhere on the wire

Meta React Native adds Stable API guards to renderer attributed string FastAPI & Pydantic Pydantic fixes Mypy plugin crash, tightens datetime constraints

$ ls showcases/

Rails Django Laravel Node.js Go Vercel Vue.js Kubernetes HashiCorp Arch Linux Linux Anthropic

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.