$ the-wire · showcase
sglang v0.5.22 lands, MLX fixes M1/M2 SDPA
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
17 people shipped this
max-krasnyansky
@max-krasnyansky
1 cited
ch-wan
@ch-wan
1 cited
mmangkad
@mmangkad
1 cited
aleroot
@aleroot
1 cited
jmmarkiewicz
@jmmarkiewicz
1 cited
Georgi Gerganov
@ggerganov
Andrea Rossetti
@rssndr
Paul Mairhofer
@invaliddayta
Isotr0py
@Isotr0py
Fangzhou-Ai
@Fangzhou-Ai
aoshen02
@aoshen02
jeejeelee
@jeejeelee
dilberx
@dilberx
ankitdhall
@ankitdhall
jain-ria
@jain-ria
Oasis-Git
@Oasis-Git
Daniel Hiltgen
@dhiltgen
sglang shipped v0.5.22 with 950 PRs from 245 contributors, while llama.cpp and MLX both spent the day on attention-kernel and buffer-handling fixes that mostly matter on specific hardware.
v0.5.22 sgl-project/sglang
The release pulls in clef / Clef Flash, PPLX-Decider v1 and v1.1, EmbeddingGemma 2, and LLaDA2.2 among its new models, and the release notes state it carries 950 PRs from 245 contributors.
hexagon: FLASH_ATTN_EXT improvements for longer context and SWA models ggml-org/llama.cpp
Hexagon FLASH_ATTN_EXT now runs gemma-4's SWA layers on HVX instead of HMX after the crossover point was retuned, since HVX measured 2-3x faster for those layers; masked block skipping cuts the extra DMA that large 1K to 2K ubatches were pulling in for padding.
[Perf] dsa: Route the k-pool indexer top-k through the top-k v2 kernel sgl-project/sglang
The GLM-5.3-Flash k-pool indexer's top-k selection moves to the top-k v2 kernel, which splits long rows across thread-block clusters instead of letting one block scan the whole pooled history per decode step. The PR puts that cost at about 84 us per DSA layer per decode step at 1M context.
Add float16 head_dim 256 TK=1 SDPA vector kernel for M1/M2 ml-explore/mlx
fp16 SDPA at head_dim 256 on M1/M2 hit "Maximum threads per threadgroup is 896 but requested 960" when the 2-pass vector kernel ran; the new TK=1 kernel covers that shape, which surfaced in MTP verify on an fp16 copy of Qwen3.8-27B.
Fix unaligned scalar reads and oversized contiguous buffer views ml-explore/mlx
Two buffer-handling bugs are fixed: unaligned scalar reads now go through memcpy instead of a typed pointer, and as_strided no longer flattens its input, which broke views of very large buffers. The MLX Metal threadgroup fix for f16/f32 d512 SDPA also landed alongside.
