$ the-wire · showcase
GGUF offset corruption fixed, vLLM restores Dynamo KV metadata
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
18 people shipped this
pdevine
@pdevine
1 cited
Jesse Gross
@jessegross
1 cited
Yuri Khrustalev
@ykhrustalev
1 cited
Daniel Bevenius
@danbev
1 cited
drluoto
@drluoto
1 cited
WoosukKwon
@WoosukKwon
1 cited
wzhao18
@wzhao18
1 cited
Kangyan-Zhou
@Kangyan-Zhou
1 cited
dhiltgen
@dhiltgen
hoyyeva
@hoyyeva
Benjamin Babik
@benbenbenbenbenben
Kartik Gulia
@frozenblade1224
taneem-ibrahim
@taneem-ibrahim
mjkvaak-amd
@mjkvaak-amd
tlrmchlsmth
@tlrmchlsmth
BBuf
@BBuf
hnyls2002
@hnyls2002
1am9trash
@1am9trash
llama.cpp now aligns the GGUF data section relative to the GGUF start, ending silent tensor corruption for files embedded at non-aligned offsets, while vLLM and sglang both shipped infrastructure fixes that unblock external consumers and lock contention.
gguf : align the data section relative to the GGUF start, not the file (#28993) ggml-org/llama.cpp
gguf_init_from_file_ptr padded the data section from file offset 0 rather than the GGUF's own start, so a GGUF embedded at an offset that was not a multiple of the alignment loaded without error and returned wrong tensor data; the same commit adds llama_adapter_lora_init_from_file_ptr and disables mmap with a warning for embedded data.
[Bugfix] Restore KV cache metadata GET method for external event consumer vllm-project/vllm
get_kv_cache_group_metadata was deleted as dead code but is still called by external event consumers such as Dynamo, so the call is restored; the same fix reports the effective scheduler KV block size, after Kimi K3 DCP8 emitted 12,288-token MLA blocks while metadata advertised 1,536 and Dynamo rejected those events.
[Router] Shard the cache-aware KV tree by chain root sgl-project/sglang
The in-process HashTree behind --cache-prefix-provider radix_tree held one process-wide RwLock, so every BlockStored and BlockRemoved event blocked each routing match_prefix; the tree is now split into 32 shards keyed by the chain's root block hash.
mlxrunner: Release freed KV buffers during speculative decode ollama/ollama
The decode loop only released MLX's pool of freed buffers when the token count landed exactly on a multiple of 256, but speculative decoding emits several tokens per round, so most rounds stepped over the boundary and long-context KV cache growth left several GB of buffers unreclaimed.
x/transfer, server: tighten redirect handling for registry requests ollama/ollama
Registry and blob transfers now validate redirect target scheme and resolved addresses before following, re-check DNS on each redirect, and refuse to follow a redirect that switches an https session to plain http; --insecure still relaxes address checks for private registries but no longer scheme checks.
vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 512 experts (#28501) ggml-org/llama.cpp
count_experts.comp sized its shared arrays with BLOCK_SIZE (256), which switched row-id hoisting off above 256 experts and forced every mul_mat_id workgroup to rescan the whole ids tensor; the limit is now 512, covering 512-expert models like Qwen3.8-Flash-Next that had been running on the slow path.
[Bugfix][DSv4.1] Fix FlashInfer DSpark non-causal attention vllm-project/vllm
DSpark's non-causal draft window carries 133 valid keys at draft length 5 padded to 256 indices, and the DSV4.1 FlashInfer integration passed the padded width as the active sparse length, so padding entered softmax normalization: a constant-value reference expecting 1 returned 0.51953125, or 133/256. The fix uses the valid SWA length while keeping FlashInfer's 128-column minimum.
ci : add API/ABI check to make-release workflow [no ci] (#28947) ggml-org/llama.cpp
The make-release workflow gains an API/ABI compatibility check that fails the release on breaking changes between versions; it can be skipped when needed because it takes time.
Action items
