$ the-wire · showcase
vLLM 0.29.0 makes Model Runner V2 the default, llama.cpp deprecates --mmap|mlock|dio
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
16 people shipped this
ParthSareen
@ParthSareen
2 cited
pwilkin
@pwilkin
2 cited
Aaron Teo
@taronaeo
1 cited
khluu
@khluu
1 cited
Juan Pérez de Algaba
@jperezdealgaba
1 cited
sfeng33
@sfeng33
1 cited
BugenZhao
@BugenZhao
1 cited
ykamiset
@ykamiset
1 cited
xiezhq-hermann
@xiezhq-hermann
1 cited
nvpohanh
@nvpohanh
1 cited
Eva H
@hoyyeva
Todor Boinovski
@tboinovski1
Ruben Ortlam
@0cc4m
jianuo-huang
@jianuo-huang
willhu-jpg
@willhu-jpg
akhilg-nv
@akhilg-nv
vLLM 0.29.0 lands 594 commits from 277 contributors and switches Model Runner V2 on for all models, while llama.cpp formally deprecates its --mmap, --mlock, and --dio flags.
vLLM tagged v0.29.0, and the highlight is that Model Runner V2 is now the default for all models, completing a rollout that started with pooling models [1]. The same release adds CUDA graph memory profiling for KV cache auto-sizing and batch-sharded sampling that cuts per-step logits memory by 1/TP [1]. This is a breaking change: code and workflows pinned to the old default model runner will now execute on MRV2 unless configured otherwise. Separately, the Rust frontend normalizes HTTP method labels in metrics, a change flagged as security [2].
llama.cpp officially deprecates the `--mmap`, `--mlock`, and `--dio` command-line flags [3]. Anyone passing these flags in scripts or service units should treat them as deprecated today. The CUDA backend also replaces `GGML_FA_ALL_QUANTS` with `GGML_CUDA_FA_QUANTS`, letting you name the exact KV quant combinations built in X-Y form instead of compiling an oversized package of combinations, some of which the PR notes are "totally unviable" and others never selected in practice [4]. Uncompiled combinations now fall back at runtime with a warning [5].
On the vLLM parser side, the DeepSeek-V4-Flash tool-call parser now tolerates a misspelled DSML opener: DeepSeek-V4-Flash intermittently emits `<｜DSML｜toolcalls>` or `<｜DSML｜tool>` instead of `<｜DSML｜tool_calls>`, and the literal matcher previously matched nothing, dropping the tool call [6]. The Rust frontend also resolves unified and split parser names consistently, checking the unified registry first so names like Kimi K3's `kimi_k3` are not shadowed by the split registry's broader `kimi` pattern [7]. On ROCm, aiter indexer scoring and top-k kernels are now integrated into the MiniMax-M3 sparse attention path [8].
In SGLang, HiCache replaces `skip_lock_node_ids` with a segment lock protocol; the old per-component tombstone set had to be built, carried, and replayed at every acquire/release pair and grew with the tree [9]. The Rust frontend gates `/health` and `/health_generate` at 503 until startup warmup completes, matching Python frontend behavior, and `--skip-server-warmup` still starts ready [10]. Ollama fixed Codex compaction by accepting standalone `function_call_output` items that carry a tool `name` and optional `namespace` but no `call_id`, which previously caused GLM sessions to fail when Codex triggered compaction after a handoff [11]. Its proxy also strips escalation-only arguments from namespaced `exec_command` tools in Full Access mode, avoiding escalation requests Codex rejects under its Never approval policy [12].
Action items
- → Audit vLLM deployments for Model Runner V2 default behavior before upgrading to 0.29.0 vllm-project/vllm [immediate]
- → Remove --mmap, --mlock, and --dio from llama.cpp launch scripts and service files ggml-org/llama.cpp [plan]
- → Switch CUDA builds from -DGGML_CUDA_FA_ALL_QUANTS to -DGGML_CUDA_FA_QUANTS listing only needed X-Y quant combinations ggml-org/llama.cpp [plan]
- → Update Rust frontend health-check probes to tolerate 503 until warmup completes sgl-project/sglang [monitor]
References
- [1] v0.29.0 ↗ vllm-project/vllm
- [2] [Security][Rust Frontend] Normalize HTTP method labels in metrics (#56058) ↗ vllm-project/vllm
- [3] args: officially deprecate --mmap|mlock|dio (#28334) ↗ ggml-org/llama.cpp
- [4] CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled ↗ ggml-org/llama.cpp
- [5] CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079) ↗ ggml-org/llama.cpp
- [6] [Bugfix] Tolerate misspelled DSML tool_calls wrapper ↗ vllm-project/vllm
- [7] [Rust Frontend] Resolve unified and split parser selections consistently ↗ vllm-project/vllm
- [8] [Performance][ROCm] Integrate aiter indexer scoring and top-k kernels into MiniMax-M3 sparse attention path ↗ vllm-project/vllm
- [9] [HiCache] Replace skip_lock_node_ids with a segment lock protocol ↗ sgl-project/sglang
- [10] [Rust] Gate health on startup warmup completion ↗ sgl-project/sglang
- [11] openai: support standalone named function outputs ↗ ollama/ollama
- [12] proxy: normalize namespaced commands in Full Access ↗ ollama/ollama
