$ the-wire · showcase
sglang locks down admin endpoints, DFlash spec decoding splits into four methods
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
sglang made its weight, LoRA, and profiling endpoints require an admin key by default, and llama.cpp pulled the four DFlash speculative decoding variants apart into separate methods, the two changes most likely to touch a deployment this week.
[Feat] entrypoints: Require --admin-api-key for sensitive admin endpoints sgl-project/sglang
Endpoints that update weights, load or unload LoRA adapters, profile, control generation, or change runtime config are now ADMIN_FORCE and return 403 unless the server was started with --admin-api-key or SGLANG_ADMIN_API_KEY, and they accept only that key. Any script or control plane hitting those routes against a server without an admin key starts failing.
spec + model: refactor DFlash based speculative decoding methods ggml-org/llama.cpp
dflash, dflash2, dspark, and lilicorr shared one DFlash backbone but all ran under the single --spec-type draft-dflash option; they are now separate methods, draft-dflash and draft-dflash2 among them. The PR calls the old layout increasingly difficult to follow, so expect the generic flag to stop selecting all four.
[ROCm] Minimax M3.1 attn indexer XCD balancing vllm-project/vllm
At 9 to 48 requests the Minimax M3.1 decode attention indexer read and dequantized each index-K page twice for 8-row requests and left XCDs imbalanced. Measured at 128K context, per-layer time at 17 to 24 requests drops from 29.6-31.1 µs to 22.1-22.8 µs and at 33 to 40 requests from 47.1-49.4 µs to 34.5 µs on gfx942 and gfx950.
[Perf] diffusion: Walk one token's heads per warp in the two-rows QK-norm+RoPE kernel sgl-project/sglang
The head_dim-128 QK-norm+RoPE kernel gave each warp a pair of rows and redid the token, head, position, and cos/sin loads per row; it now walks one token's heads per warp. It is one of five bit-exact cuts to the small kernels around FA4 and the GEMMs, which the MiniMax-H3 T2VA profile shows consuming about 93% of a 119 s request on one B300.
[Perf] hicache: Let the PP-sync channel run the pipeline's depth ahead, not one round (#45351) sgl-project/sglang
The PP-sync channel posted one gloo isend per _all_reduce and waited on the previous round's sends at the start of every check_hicache_events, which blocked on a receiver that had not posted yet; it now runs the pipeline's depth ahead instead of one round. The fix closes one hang report but the PR states it does not fix a rank that never posts its recv.
jinja : reduce template rendering overhead ggml-org/llama.cpp
The rest of the day is kernel-level cleanup: llama.cpp's jinja renderer drops accidental builtin-table copies and temporary allocations without touching loop execution or scope ownership, mtmd exposes mtmd_helper_video_decoder so downstream builds can swap ffmpeg for AVFoundation or NDK MediaCodec, vLLM adds slots=True to its internal dataclasses and removes OpenPangu-VL, and sglang moved its A...
Action items
- → Set --admin-api-key (or SGLANG_ADMIN_API_KEY) on any sglang server whose clients call weight-update, LoRA, profiling,... sgl-project/sglang [immediate]
- → Audit --spec-type draft-dflash usage in llama.cpp scripts and switch to the new per-method spec types ggml-org/llama.cpp [plan]