$ the-wire · showcase
llama.cpp lands GDN chunked kernels and SYCL IQ3 reorder
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
20 people shipped this
hoyyeva
@hoyyeva
1 cited
dhiltgen
@dhiltgen
1 cited
am17an
@am17an
1 cited
anantshri
@anantshri
1 cited
ServeurpersoCom
@ServeurpersoCom
1 cited
Juntian777
@Juntian777
1 cited
sherlockwu
@sherlockwu
1 cited
willietran
@willietran
Aleksander Grygier
@allozaur
frozenblade1224
@frozenblade1224
simondanielsson
@simondanielsson
HangruiCao2000
@HangruiCao2000
Woosuk Kwon
@WoosukKwon
yhyang201
@yhyang201
DarkSharpness
@DarkSharpness
Oasis-Git
@Oasis-Git
XXXXRT666
@XXXXRT666
aaishwarymishra
@aaishwarymishra
Anastasiia Filippova
@nastya236
Ivan Petrukha
@petrukha-ivan
The day's gains are mostly measured in tok/s: GPU kernel work in llama.cpp, ROCm and sglang for long-context decode, and a batch of correctness fixes that unblock downstream projects.
cuda, hip: GDN chunked kernel ggml-org/llama.cpp
A chunked GDN kernel upstreamed from QVAC fabric lands for both CUDA and HIP. The PR reports 3 to 6% better prefill on dense 27B and 11 to 21% on 35B-A3B across RTX 4090/5090/PRO 6000, with 3 to 6% on Spark, all measured with -b 2048 -ub 2048 -fa 1, full GPU offload and F16 KV cache against master 296538ade.
sycl: IQ3 code reorder ggml-org/llama.cpp
Persistent reordered layouts for IQ3_S and IQ3_XXS MMVQ plus reorder-aware dequantization lift SYCL decode on Intel Arc Pro B70: 12.30 to 21.59 tok/s at 9k context and 9.76 to 14.92 tok/s at 64k, on oneAPI 2026.1.1.
[Perf][DSv4.1] Length-aware candidate block selection in the decode indexer vllm-project/vllm
DeepSeek-V4.1's decode indexer was scoring every block of the full [rows, max_model_len] buffer and running torch.topk(2048) over all of it, about 850 us per decode step at 1M regardless of real context length. Candidate selection is now length-aware, scoring only valid blocks. sglang's parallel DeepSelect work rewrites the cluster kernel for small-batch, long-context decode with bf16 index log...
openai: accept custom tool calls in Responses history ollama/ollama
Continuing a Responses conversation that contains custom_tool_call or custom_tool_call_output used to return HTTP 400 before inference; those items now go through the existing function-call path, keeping call IDs, namespaces, and freeform input as {"input": "..."}, and standalone and triggered compaction replay them too.
[Fix] sglang-processor: Follow Python's OpenAI layer through #43027, #37577, #43006, #43026, #43005 sgl-project/sglang
The Rust sglang-processor had drifted from Python's OpenAI layer after five changes merged to main past its parity fixtures, including sending "tool_calls": null where Python now omits absent tool_calls. The fix realigns the /generate path used by sgl-router and the Rust server; the same author also corrects cache-aware routing, which never matched for MTP/EAGLE on hybrid Mamba/linear-attention...
ggml: fix conv_1d result layout for batched input ggml-org/llama.cpp
ggml_conv_1d read its GEMM result as [OL, OC, N] when it is actually [OL, N, OC], and because a reshape only relabels the buffer the shape looked right while values came out interleaved between batches. Nothing in mtmd batches a vocoder today, so there is no in-tree output change; this is a ggml-level fix for downstream vocoder batching such as qwentts.cpp.
MLX: version bump ollama/ollama
An MLX version bump is the other thread to watch: the author asks that the other change be merged first and the smaller qwen 3.5 models be re-published afterward, noting that with this bump their tendency to loop in thinking has gotten worse.
