$ the-wire · showcase
Strix Halo loads, llama.cpp tightens its exit codes, vLLM patches a torch.load gap
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
Ollama fixed the Windows iGPU memory accounting that made large models unloadable on Strix Halo, vLLM gates embedding archive size before torch.load, and llama.cpp turned silent test passes into hard failures.
llm: keep weights out of pinned host memory on Windows iGPUs (#19636) ollama/ollama
On Windows iGPUs, pinned host memory is charged against the same WDDM shared-memory budget as device buffers, and llama-server's CPU-side weights in that pool were invisible to --fit, so a partially offloaded model overflowed and the KV cache allocation failed. The commit passes --no-host for Windows iGPUs so CPU-side weights stay in pageable memory; models larger than roughly half of system RA...
[Bugfix][Security] Check the uncompressed size of embedding archives before torch.load (#67316) vllm-project/vllm
vLLM now checks the uncompressed size of embedding archives before calling torch.load. Anyone loading embedding weights from an archive they did not build themselves should take the fix.
server: answer reasoning_effort levels the template rejects with 400 (#32332) ggml-org/llama.cpp
The server's reasoning effort probe rendered every candidate level but discarded the ones the template raises on, so a request carrying such a level (the report names max on Qwen3.8) reached the template and returned a 500 on /v1/chat/completions, /v1/responses and /v1/messages. Rejected levels are now kept in reasoning_effort_rejected and returned as invalid_argument, which the server answers ...
server : fix output_config.effort being dropped in /v1/messages (#32321) ggml-org/llama.cpp
The Anthropic-compatible /v1/messages endpoint was silently dropping output_config.effort, so effort settings sent through that route had no effect on the request.
Fail `test-backend-ops` and exit with error status 1 when missing and/or unsupported types and/or unsupported backends are provided in the command line ggml-org/llama.cpp
Running test-backend-ops with a bogus type used to report 0/0 tests passed and exit 0, for example test -b MTL0 -p 'no-such-type'; it now fails hard and exits with status 1, including when no tests ran at all, which turns a green CI run that exercised nothing into a red one.
[Perf][MiniMax-M3.1] MiniMax-M3.1-Flash Specialized Decode: CuTe DSL Optimized GB300 (up to 1.45x) vllm-project/vllm
On the server side, vLLM merged an opt-in NVIDIA-specialized decode for MiniMax-M3.1-Flash on GB300 targeting small agentic decode batches (up to 64 rows, no speculation or DSpark with up to 7 draft tokens), which SGLang's Blackwell/Rubin runtime work and hybrid megakernel binding views target from the other direction.
[CI] rust: Cache the Rust Ext cargo target dir and narrow the sgl-router cache key sgl-project/sglang
The long tail: SGLang cached the Rust Ext cargo target dir so unchanged lock files rebuild only the sglang crates instead of roughly 800 units, added Atlas 800I A3 deployment scenarios to the GLM-5.1 cookbook, and extended --language-model-only to Gemma 4 so the Rust server can start.
Action items