$ the-wire · showcase
MoE expert masking lands across every llama.cpp backend
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
14 people shipped this
ngxson
@ngxson
1 cited
Aman Gupta
@am17an
1 cited
ggerganov
@ggerganov
1 cited
xaguilar-amd
@xaguilar-amd
1 cited
vvagaytsev
@vvagaytsev
1 cited
dhiltgen
@dhiltgen
danbev
@danbev
allozaur
@allozaur
xiao-llm
@xiao-llm
charxwu
@charxwu
sragrawal
@sragrawal
ukannika
@ukannika
ispobock
@ispobock
mickqian
@mickqian
llama.cpp taught mul_mat_id to read a -1 index as "skip this computation" in one unfenceable sweep across all backends, opening the door to the expert-mask MoE pattern and to DeepSeek-V4.1-Flash support in the same day.
ggml : allow MUL_MAT_ID -1 index to skip the computation ggml-org/llama.cpp
A -1 index passed to mul_mat_id now means "skip the computation," which enables an expert mask like the PyTorch MoE implementation. ngxson notes the change could not be gated behind a graph build-time condition, so every backend was touched at once, and states plainly that "code is 100% AI-generated." The motivating models are ones with "small" and "big" expert tiers.
model: DSV4.1 flash (#31086) ggml-org/llama.cpp
llama.cpp gains DeepSeek-V4.1-Flash (deepseek41): MXFP8 is removed, the Engram table converts to BF16, compressed-KV read layers are computed once in the cache, and Engram rows are gathered in-graph rather than on the host. The Engram hash constants now live in hparams.ple_*, one module per Engram layer.
metal : remove mul_mm_id amax rescaling ggml-org/llama.cpp
The Metal mul_mm_id kernel drops the amax rescaling of src1 added to prevent inf/NaN when MoE activations overflow f16. Since the MoE down projection is rescaled per expert-token in the compute graph, activations reach MUL_MAT_ID already in f16 range and the in-kernel rescale is redundant; ggerganov also notes the removed path was gated on GGML_PREC_F32 for src1, which nothing in the graph sets.
[Fix] [AMD] custom-allreduce: Stop registering oversized inputs in the deterministic all-reduce sgl-project/sglang
On AMD GPUs, deterministic all-reduce (enabled by --enable-deterministic-inference, --rl-on-policy-target, or SGLANG_USE_1STAGE_ALLREDUCE=1) produced each GPU's own input instead of the sum whenever a single all-reduce exceeded the 16 MiB working buffer, which a normal prefill on a large model triggers. Each oversized call also permanently consumed an entry in a fixed-size table until the proce...
[ROCm][MLA][DCP] Drop VLLM_ROCM_AITER_MLA_DCP_VERIFY vllm-project/vllm
The remainder is housekeeping and narrow tuning: vLLM removes VLLM_ROCM_AITER_MLA_DCP_VERIFY so the DCP spec-decode verify kernel is picked heuristically with unchanged defaults, adds an opt-in MXFP4 int8-dot GEMV for gfx11 decode and indexer context parallelism for MiniMax-M3 behind --attention-config '{"indexer_cp": true}', while sglang moves the Marlin GPT-OSS SwiGLU onto a fused Triton kern...
