$ the-wire · showcase
Metal GROUP_NORM fix, Ripple preview lands in Ollama's MLX runner
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
13 people shipped this
pdevine
@pdevine
1 cited
ggerganov
@ggerganov
1 cited
am17an
@am17an
1 cited
ngxson
@ngxson
1 cited
subhashpolisetti
@subhashpolisetti
1 cited
wenkaidu
@wenkaidu
1 cited
max-krasnyansky
@max-krasnyansky
AndreasKaratzas
@AndreasKaratzas
fadara01
@fadara01
yewentao256
@yewentao256
BBuf
@BBuf
kevin-mii
@kevin-mii
metamergebot
@metamergebot
llama.cpp corrected a Metal kernel that silently truncated output for batched inputs, which matters more than the day's usual backend refactors because it produced wrong numbers rather than slow ones.
metal : fix GROUP_NORM for ne03 > 1 and non-contiguous inputs ggml-org/llama.cpp
The Metal GROUP_NORM kernel ignored the batch dimension ne03 and assumed contiguous inputs, leaving most of the output unwritten for ne03 > 1 and producing garbage on padded inputs. The new GROUP_NORM(type=f32,ne=[3,1,1,65537],num_groups=1) test now passes on Metal, where it previously returned ERR = 4.0.
Add ripple preview support to the MLX runner ollama/ollama
The MLX runner gains the Ripple model plus its renderer, parser and import registration, with per-layer NoPE support in DFlash drafts and fused MLP activation, value blending and attention gating, backed by reference fixtures and model/runner regression tests. Preview weights ship as pdevine/ripple-preview:65b-mlx-bf16, 65b-mxfp8 and 65b-nvfp4.
[Feat] [AMD] custom-all-reduce: Run carv2 on MI450 with FABRIC handles and a gfx1250 table sgl-project/sglang
carv2 now runs on MI450 (gfx1250) with FABRIC handles and a launch table tuned for that part; the MI350X table left it 21% behind RCCL on DeepSeek-V4-Flash TP8 at C64, at 1,628 versus 2,051 to 2,105 tok/s. SGLang reports that a C64 trace showed 1shot_push on a 96-block grid spending most of each call in a wait, which is the tail the new table targets.
llama_moe_cache: run missed experts on CPU, add tuning script ggml-org/llama.cpp
The MoE cache can now execute missed experts on CPU according to --moe-cache-cpu-share, which costs decode speed proportional to how fast the CPU can read experts. Because --moe-cache-mib and --moe-cache-cpu-share no longer carry a fixed penalty, the PR adds a tuning script you prepend to your llama-server command to find the right pair for your machine.
mtmd: move log-mel computation to cgraph ggml-org/llama.cpp
Audio preprocessing no longer spawns threads inside mtmd_audio_preprocessing, a pattern that could create N threads times M requests and did not work in wasm builds at all. The log-mel computation moved to cgraph, with DFT matmul implemented via ggml_custom and run on CPU because the Metal version deviated too far.
[Bugfix][Qwen2.5-Omni] Size the video encoder budget from the processor's video limits vllm-project/vllm
Qwen2.5-Omni sized its video encoder budget and profiling dummy from image limits (3584x3584 frames) while its HF processor caps video frames at 768 * 28 * 28 pixels, the same gap previously fixed for Qwen3-Omni. The visible cost was a budget that stops short of max_model_len when that value is not a multiple of the two-frame step, and startup profiling running the vision encoder on a roughly 1...
