$ the-wire · showcase
Ollama exposes thinking levels, llama.cpp widens Hexagon and architecture coverage
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
14 people shipped this
ParthSareen
@ParthSareen
1 cited
ServeurpersoCom
@ServeurpersoCom
1 cited
tboinovski1
@tboinovski1
1 cited
bnellnm
@bnellnm
1 cited
draftbk
@draftbk
1 cited
pdevine
@pdevine
shaofeiqi
@shaofeiqi
frida-andersson
@frida-andersson
hmellor
@hmellor
DarkLight1337
@DarkLight1337
NickLucche
@NickLucche
Qiaolin-Yu
@Qiaolin-Yu
ch-wan
@ch-wan
ZYHowell
@ZYHowell
Ollama made per-model reasoning controls discoverable through its API, llama.cpp pushed more architectures and Hexagon kernels through CI, and vLLM added back-pressure detection to its KV cache tiering manager.
api: expose model thinking levels and defaults ollama/ollama
The /api/show endpoint and `ollama show` now report each model's supported thinking values and their defaults through an optional `thinking` object holding `values` and `default`, so clients like Codex CLI can populate reasoning-level controls from the model itself rather than hardcoding them. Older local templates pick up the metadata without a re-pull, and generic renderers preserve known lev...
[KV Offloading] Back-pressure detection and remediation vllm-project/vllm
The KV cache tiering offloading manager now watches store completion latency via an exponential moving average and stops cascading new stores to a saturated tier (disk, shared storage, or P2P) until it recovers, rather than letting unbounded job pileup degrade serving latency. Operators running multi-tier offload get a serving path that degrades instead of snowballing.
hexagon: support for HMX flash-attention head_dim not multiple of 64 ggml-org/llama.cpp
The Hexagon HMX flash-attention kernel now handles head_dim values that are not multiples of 64, such as SigLIP's head_dim=72, by internally padding DK/DV to the next multiple of 64 with zero-filled tail lanes. Those shapes previously fell back to the slower HVX/CPU path, so multimodal models on Hexagon hardware now stay on the accelerated kernel.
Model-Saver: Write the SWA pattern, 15 more architectures roundtrip ggml-org/llama.cpp
The model saver gained the SWA pattern, which pulls 15 additional architectures into the dummy-model, roundtrip, save/load-state, and fusion tests. A full local run of test-llama-archs on CPU and CUDA reports 254 roundtrips green, and the diff is mostly the same one-line substitution in a loader across 18 of its 21 touched files.
perf(sampling): avoid GPU syncs when applying custom logit processors sgl-project/sglang
Custom logit processors now cache CPU request rows and GPU indices instead of syncing per application; in one measurement on Qwen3.5-0.8B at batch size 1 on a B200 in BF16, time to first token moved from 18.60 ms to 17.59 ms and inter-token latency from 2.41 ms to 1.35 ms with identical outputs. Separately, sglang cut v0.5.20 with 713 PRs from 237 contributors and added GLM-5.3-Flash and Hy4-Pr...
