$ the-wire · showcase
SYCL memory reporting fixed, sparse-attention kernels land across vLLM and SGLang
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
13 people shipped this
Łukasz Ślusarczyk
@lslusarczyk
1 cited
arthw
@arthw
1 cited
ZJY0516
@ZJY0516
1 cited
zhewenl
@zhewenl
1 cited
mickqian
@mickqian
1 cited
ShangmingCai
@ShangmingCai
1 cited
Sigbjørn Skjæret
@CISC
gty111
@gty111
LopezCastroRoberto
@LopezCastroRoberto
Fangzhou-Ai
@Fangzhou-Ai
Jiminator
@Jiminator
huangtingwei9988
@huangtingwei9988
Hritik003
@Hritik003
Two recurring threads run through today's local-LLM work: making accelerator memory reporting trustworthy on the SYCL path, and pushing sparse-attention and caching infrastructure into the serving stacks.
sycl : fix oneDNN scratchpad breaking the pool free order (#28704) ggml-org/llama.cpp
oneDNN's scratchpad allocation was being freed out of order relative to the pool, which is exactly the kind of thing that shows up as intermittent SYCL failures rather than a clean crash. Fixed in llama.cpp's SYCL backend.
[SYCL] Fix get mem error ggml-org/llama.cpp
Memory-size queries on Intel GPUs went through the level zero Sysman API, which has no equivalent on WSL; the code now falls back to the SYCL API when Sysman is unavailable, and the build stops with a clear message if the level zero dev package is missing.
[Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer vllm-project/vllm
vLLM now vendors DeepSeek's DeepSelect TopK kernel as an optional extension and makes every decode top-k implementation on the DSA sparse indexer path selectable, with an auto heuristic on top. The extension compiles only for sm_100a/sm_103a with nvcc >= 12.9, so it is opt-in hardware rather than a default.
[PD][OpenAI] Gate /v1/responses persistence behind --enable-response-store, default off sgl-project/sglang
The Responses path kept two unbounded in-memory dicts with no TTL or eviction, and in PD deployments a prefill node's bookkeeping was unreachable by construction because every retrieval call routes to decode. Persistence is now gated behind --enable-response-store and defaults off.
[Bugfix][KV Connector] MooncakeStore: exclude non-prefix-cacheable (QSA ring) groups; fix align-mode check vllm-project/vllm
MooncakeStore built its hash/store/lookup layout from every enabled group, including Qwen3.8-Flash-Next's QSA ring scratch cache, which reports prefix_cacheable as False and has a capacity of 4 (8 with MTP=3) against block size 800 for the aligned groups. Those non-prefix-cacheable groups are now excluded and the align-mode check is corrected.
[Diffusion] Make the SP sequence gather pass contiguous shards sgl-project/sglang
In USPAttention, slicing a replicated prefix or suffix along dim 1 preserves the row stride, so the shard is contiguous only when the batch dim is 1. Batched passes now gather contiguous shards instead.
