$ the-wire · showcase
llama.cpp lands SYCL DPAS flash attention, Ollama repairs Responses history
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
18 people shipped this
hoyyeva
@hoyyeva
1 cited
Jesse Symons
@Torchit1
1 cited
ServeurpersoCom
@ServeurpersoCom
1 cited
vMaroon
@vMaroon
1 cited
FaizPalwala
@FaizPalwala
1 cited
Daniel Hiltgen
@dhiltgen
Anant Shrivastava
@anantshri
tdakhran
@tdakhran
aoshen02
@aoshen02
hmellor
@hmellor
njhill
@njhill
khluu
@khluu
weireweire
@weireweire
lishuoshuo-amd
@lishuoshuo-amd
hnyls2002
@hnyls2002
metamergebot
@metamergebot
aaishwarymishra
@aaishwarymishra
turing.huang
@turingH
llama.cpp's SYCL backend now runs flash attention directly on quantized KV blocks, Ollama stops silently dropping reasoning and images resent through /v1/responses, and vLLM fixes KV corruption on HiSparse disaggregated decoders.
[SYCL] add DPAS flash attention kernel for quantized KV cache (#31125) ggml-org/llama.cpp
Decode and small batches (1 to 31 query tokens) now run flash attention straight on q4_0 and q8_0 KV blocks, instead of scalar dequant or converting the whole cache to f16 on every call. Q.K uses s8 x s4 and s8 x s8 DPAS on XMX with V.P left in f16, covering head sizes 64 to 512 and GQA up to 32.
openai: preserve readable Responses input content ollama/ollama
Readable reasoning, assistant refusals, and inline images in agent messages are now preserved when an app resends chat history through /v1/responses, and a previous refusal no longer fails the request. Images carrying only a file_id are replaced with an explicit notice in the model's input rather than skipped silently, so the rest of the conversation continues.
[Bugfix][Core] Defer async KV loads on HiSparse decoders until in-flight steps finish vllm-project/vllm
A NIXL load could overwrite a GPU page while the previous step was still reading it, corrupting the KV cache on HiSparse P/D decoders. Every async load is now deferred until the in-flight step has been processed; the reviewer-requested revision drops the earlier reuse-detection scheme, so the GLM-5.3 numbers were measured on that prior version.
ui: build the reasoning menu from the template effort levels ggml-org/llama.cpp
The Low/Medium/High/Max menu was hardwired to thinking_budget_tokens, so on Qwen3.8 picking Medium still sent xhigh instructions in the prompt and cut the think block at 2048 tokens. When chat_template_caps reports reasoning_effort_levels the menu now offers Default, Off, and those levels and sends reasoning_effort with no budget; templates that report none keep the token-budget menu.
Fix FP8 NaN preservation in to_fp8 and E8M0 scale dequantization ml-explore/mlx
The saturation check in ToFP8 and fp8_e4m3 assigned 0x7E unconditionally, and since float32 NaN has f_bits > 0x7F800000 > fp8_max, every NaN saturated to the finite 448.0. NaNs now map to the standard E4M3fn bit pattern. The same day also brought fixes for left/right shift overflow, gguf tensor to f16 size overflow, and E8M0 scale dequantization.
