RepoJournal
Local LLMs Local LLMs
122 wires and counting

$ follow Local LLMs

Keep up with Local LLMs in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-21
stories 340

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Ollama picks parallel slots from memory, llama.cpp fuses Gemma 4 prompt rows

By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology

The day's work is mostly about squeezing more out of the hardware you already have: Ollama's MLX runner now sizes its slot count from available memory and llama.cpp skips work it never needed to do, while vLLM closes a Rust-frontend gap that returned a hard 400.

mlxrunner: choose default parallelism from available memory ollama/ollama

by dhiltgen

When OLLAMA_NUM_PARALLEL is unset, the MLX runner now picks 1, 2, 4 or 8 slots from available memory and per-client storage estimated at the model's context limit. Explicit settings still win, and it keeps the serial default under memory pressure, for unknown cache layouts and on other backends.

gemma4: Optimized prompt processing for multi-seq queries ggml-org/llama.cpp

by Meet91721

Gemma 4 E4B and E2B can now skip logits computation for prompt rows that do not need it during multi-sequence processing, continuing the work started in an earlier PR. It targets multi-seq prompt processing specifically, so single-stream decode is unaffected.

mlx: fuse Inkling residual mixing on Metal ollama/ollama

by dhiltgen

Inkling residual mixing on Metal now computes normalized residuals, query scores and the weighted sum in a single dispatch, keeping intermediates in the threadgroup while preserving the existing reduction order and rounding. On an M5 Max with HumanEval at 2048 prompt / 256 generated tokens and a cold cache, generation went from 54.75 to 59.90 TPS (+9.4%) for one user.

[Fix] speculative: Feed every accepted token to the penalizers sgl-project/sglang

by hnyls2002

Speculative decoding was feeding the penalizers only the last output token per request per verify round, so frequency, presence and repetition penalties plus min_new_tokens counted one token per round instead of every committed token. A new Req.penalty_cumulated_len tracks what the penalizers have seen and cumulate_penalty_output_tokens() feeds each token past it with its batch row as req_indices.

[Rust Frontend] Support `additional_tools` input items in the Responses API vllm-project/vllm

by albertoperdomo2

The Rust frontend rejected additional_tools input items with a 400 unknown variant error; it now hoists their tools into the request tools before anything reads them, so they reach the model, tool_choice resolution and the echoed tools like top-level tools. Duplicates and redefinitions merge as they do in Python's _merge_add, and the Rust skip is removed from the e2e test.

Correct gradients and batching for DivMod ml-explore/mlx

by mahimn01

Elsewhere in the long tail: mx.divmod now produces real vjp and jvp for its remainder output instead of zeros (so differentiating divmod(x, y)[1] no longer silently yields 0) and its vmap rule reports the second output axis, while llama.cpp added Inkling 2 Mini to mlxrunner, wired Drama 3 voice cloning to the shared Fish DAC encoder, and vLLM published an Inkling 2 Mini recipe that serves from ...

Quick answers

What shipped in Local LLMs on November 21, 2026?
The day's work is mostly about squeezing more out of the hardware you already have: Ollama's MLX runner now sizes its slot count from available memory and llama.cpp skips work it never needed to do, while vLLM closes a Rust-frontend gap that returned a hard 400. In total, 165 commits, 165 pull requests, and 10 releases landed.
Who contributed to Local LLMs on November 21, 2026?
20 developers shipped this update, including dhiltgen, hoyyeva, Aleksander Grygier, Meet91721, lifeiteng, WoosukKwon, albertoperdomo2, and harshaljanjani, and 12 more.
What were the notable Local LLMs updates?
mlxrunner: choose default parallelism from available memory, gemma4: Optimized prompt processing for multi-seq queries, and mlx: fuse Inkling residual mixing on Metal.