RepoJournal
Local LLMs Local LLMs
122 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-11
stories 296

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Ollama brings parallel generation to MLX, llama.cpp adds FP8 KV-cache

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

Ollama's MLX path can now serve several generation requests at once, and llama.cpp grew an FP8 KV cache, while the serving engines spent the day on scheduler and connector plumbing rather than headline throughput.

mlx: support concurrent generation ollama/ollama

by dhiltgen

MLX models can now serve concurrent generation through OLLAMA_NUM_PARALLEL, with the default still at one request and each additional in-flight request costing its own memory. Ready requests share projection and feed-forward work while attention, caches, sampling and response streams stay independent, and the scheduler interleaves 2048-token prompt chunks with decode rounds.

Add FP8 KV-cache support ggml-org/llama.cpp

by ORippler

llama.cpp accepts OCP-FP8 KV caches, with optional run-time static per-tensor scales; scaling is applied at graph level rather than inside ggml_flash_attn_ext, matching the NVFP4/FP8 ownership model and letting the same logic serve flash and standard attention. Models with non-unit-scaled FP8 KV caches, such as nvidia/Mistral-Medium-3.5-128B-NVFP4, are what the path is built for.

ggml : add MUL_MAT src1 precision tests and gating ggml-org/llama.cpp

by ggerganov

Backends that cannot honor a requested MUL_MAT src1 precision now fall back to the CPU instead of silently computing at the wrong width. Metal rejects MUL_MAT and MUL_MAT_ID for GGML_PREC_F32 and MUL_MAT for GGML_PREC_BF16 unless the tensor API is present and src0 is not F32; CUDA rejects GGML_PREC_F32 for MUL_MAT on Ampere+ (TF32) and for MUL_MAT_ID, and test-backend-ops gains BF16/F32 src1 ca...

[Bugfix][Rust Frontend] Fix cancellation during request submission vllm-project/vllm

by bzsuni

A race in the Rust frontend could send Abort before Add when a caller cancelled mid-submission, letting the engine process a request that was already cancelled; the output stream now stays alive in the spawned Add send task until the send completes. A deterministic regression test cancels after the task is spawned and before it runs.

[Refactor] moe: Remove the triton_kernel MoE runner backend sgl-project/sglang

by Fridge003

The triton_kernel MoE runner backend is deleted, the largest step in a four-PR stack to drop the vendored triton_kernels package from the sglang-kernel wheel; --moe-runner-backend triton_kernel no longer resolves.

Quick answers

What shipped in Local LLMs on November 11, 2026?
Ollama's MLX path can now serve several generation requests at once, and llama.cpp grew an FP8 KV cache, while the serving engines spent the day on scheduler and connector plumbing rather than headline throughput. In total, 143 commits, 142 pull requests, and 11 releases landed.
Who contributed to Local LLMs on November 11, 2026?
16 developers shipped this update, including hoyyeva, dhiltgen, ngxson, allozaur, ORippler, ggerganov, Leizhen Zhang, and zixi-qi, and 8 more.
What were the notable Local LLMs updates?
mlx: support concurrent generation, Add FP8 KV-cache support, and ggml : add MUL_MAT src1 precision tests and gating.