$ the-wire · showcase
Vulkan adds TQ1_0, Spark2.5 lands in llama.cpp
By RepoJournal · Filed · About Local LLMs
14 people shipped this
Aleksander Grygier
@allozaur
2 cited
Anjielon
@Anjielon
1 cited
KnightYao
@KnightYao
1 cited
JoursBleu
@JoursBleu
1 cited
Whamp
@Whamp
1 cited
Thien Tran
@gau-nernst
1 cited
wenhuach21
@wenhuach21
1 cited
billishyahao
@billishyahao
1 cited
mickqian
@mickqian
1 cited
arathi-hlab
@arathi-hlab
1 cited
froststeam
@froststeam
1 cited
zhouyou9505
@zhouyou9505
jeejeelee
@jeejeelee
Xia-Weiwen
@Xia-Weiwen
llama.cpp and its ecosystem shipped new hardware support and a model architecture, while vLLM and SGLang fixed bugs in prefill and warmup paths.
llama.cpp gained end-to-end support for the Spark2.5 model family [1] and an opt-in `--fuse-qkv` flag on `convert_hf_to_gguf.py` [2], which concatenates complete per-layer Q/K/V triples into a single `attn_qkv` tensor when enabled. Also, the Vulkan backend now supports the TQ1_0 quant format across matrix multiplication, mat-vec, dequant, and get_rows operations [3]. The included test run on gfx1151 shows "MUL_MAT 11/11, MUL_MAT_ID 6/6, GET_ROWS 4/4, unchanged."
The llama.cpp chat UI improved rendering performance through two related changes [4][5]: active conversation fields are now updated in place rather than replacing the whole `activeConversation` object, which previously changed its identity on every send and triggered a full refetch of all messages, and off-screen rows are now lazily mounted near the viewport, avoiding full tree builds for rows the garbage collector keeps walking.
vLLM fixed a chunked prefill bug in KV offload where the wrong SWA chunk count was used as the store-reachability horizon, causing chunks to be submitted that the load path never requests [6]. The fix uses the prompt's final offloadable length while prefill is active. vLLM also removed `torch.compile` from the NVIDIA implementation of the Qwen3.8-Flash-Next model [7], which may affect compile-time overhead and runtime behavior, and restored CUDA support for 2-bit and 3-bit AutoRound formats [8].
SGLang added qlen>1 support for the aiter gluon path on AMD, which previously only worked with qlen=1 [9]. The diffusion path now silences internal warmup frame searches, which were repeatedly logging Wan and causal-frame warnings for each candidate [10]. The XPU nightly dashboard was fixed so it renders data instead of empty tables [11]. Finally, SGLang added MUSA installation support with a Dockerfile and a Moore Threads GPU installation guide [12], though the PR does not change scheduler or runtime behavior.
Action items
- → Test Spark2.5 and Vulkan TQ1_0 workloads in llama.cpp if you use those features ggml-org/llama.cpp [plan]
- → Check whether removing torch.compile from Qwen3.8-Flash-Next in vLLM changes your inference performance vllm-project/vllm [monitor]
- → Verify KV offload behavior after the chunked prefill fix in vLLM if you run chunked prefill with SWA vllm-project/vllm [plan]
- → Review new SGLang AMD and XPU paths if you use those backends sgl-project/sglang [monitor]
References
- [1] [Model] Support for Spark2_5ForCausalLM implementation ↗ ggml-org/llama.cpp
- [2] convert : add `--fuse-qkv` flag to fuse Q/K/V into QKV during HF-to-GGUF conversion ↗ ggml-org/llama.cpp
- [3] vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765) ↗ ggml-org/llama.cpp
- [4] ui: Improve Chat Messages rendering performance (#28460) ↗ ggml-org/llama.cpp
- [5] ui: Improve Chat Messages rendering performance ↗ ggml-org/llama.cpp
- [6] [Bugfix][KV Offload] Fix SWA store reachability during chunked prefill ↗ vllm-project/vllm
- [7] [Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272) ↗ vllm-project/vllm
- [8] add 2/3/5/6/7 CUDA support in AutoRound format ↗ vllm-project/vllm
- [9] [AMD] support qlen>1 for aiter gluon path for Kimi K3 ↗ sgl-project/sglang
- [10] [diffusion] fix: quiet internal warmup frame searches ↗ sgl-project/sglang
- [11] [XPU][CI] Fix empty nightly dashboard ↗ sgl-project/sglang
- [12] [MUSA] Add installation guide and Dockerfile ↗ sgl-project/sglang
