$ the-wire · showcase
OLLAMA REWRITES TUI PERFORMANCE LAYER AS LLAMA.CPP ARMS MEMORY TIERS
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
16 people shipped this
ParthSareen
@ParthSareen
3 cited
Aaron Teo
@taronaeo
1 cited
ColinZ22
@ColinZ22
1 cited
microslaw
@microslaw
1 cited
netanel-haber
@netanel-haber
1 cited
JustinTong0323
@JustinTong0323
1 cited
yuan-luo
@yuan-luo
1 cited
dhiltgen
@dhiltgen
Georgi Gerganov
@ggerganov
zql
@zqlcode
shalinib-ibm
@shalinib-ibm
ronensc
@ronensc
afriedri
@afriedri
zijiexia
@zijiexia
longxin9715
@longxin9715
Fridge003
@Fridge003
Ollama shipped three performance wins for the terminal interface while llama.cpp redefines mlock semantics and vLLM closes quant bugs across ROCm and transformer backends.
Ollama's chat TUI got a performance overhaul across three commits. The `/prompt` scrolling cache [1] now snapshots formatted lines at terminal width instead of rebuilding on every frame, eliminating visible lag for large prompts. File mention completion [2] now accepts highlighted references with Enter, keeping cursor position and text flow intact, fixing a root cause where Enter only checked slash completions. The markdown table parser [3] tightened its separator logic to require hyphens in each cell, keeping pipe-delimited prose as text instead of mangling it into tables. Over in llama.cpp, memory management got clearer semantics: the new `-lm mlock` flag [4] specifies memory-lock without mmap, while `-lm mmap+mlock` pairs them together, fixing a call path that was lost during the #20834 refactor. vLLM closed critical bugs blocking inference: per-channel FP8 dequant for GLM-5.2 on ROCm [5], MQA tensor-parallel replication [6], and non-gated MoE support for Nemotron models [7]. Meanwhile, sglang added standalone Qwen3.5 text-only model support [8] and cut spec-v2 host overhead in hybrid-linear MTP decode [9].
Action items
- → Test ollama TUI file completions and table rendering after upgrade ollama/ollama [plan]
- → Verify GLM-5.2 quant checkpoints on ROCm with latest vLLM vllm-project/vllm [plan]
- → Monitor llama.cpp mlock behavior in production after semantics shift ggml-org/llama.cpp [monitor]
References
- [1] Improve /prompt rendering performance ↗ ollama/ollama
- [2] cmd/tui: accept file mentions with Enter in the agent TUI ↗ ollama/ollama
- [3] cmd/tui: keep pipe-delimited prose out of tables ↗ ollama/ollama
- [4] arg: add `-lm mlock` where it mlocks but doesnt mmap ↗ ggml-org/llama.cpp
- [5] [ROCm] [BugFix] Fix Quark GLM-5.2 Checkpoint inference: indexer wk per-channel FP8 dequant + missing sparse-MLA metadata fields ↗ vllm-project/vllm
- [6] Fix MQA with tensor parallelism on transformers modeling backend ↗ vllm-project/vllm
- [7] Fix Humming non-gated MoE ↗ vllm-project/vllm
- [8] [Model] Support standalone text-only Qwen3.5 checkpoints ↗ sgl-project/sglang
- [9] [MTP] Cut spec-v2 host-seam overhead in hybrid-linear MTP decode ↗ sgl-project/sglang
