The Wire · Showcase
LLAMA.CPP LOCKS DOWN WASM, ADDS MCP STDIO, HEXAGON GETS IM2COL ACCELERATION
By RepoJournal · Filed · About Local LLMs
llama.cpp disabled OpenMP for Emscripten builds to fix WASM compilation, while shipping full MCP stdio support and unlocking 1.5x to 3.5x latency gains on Hexagon vision models.
The WASM build now runs in isolation after disabling OpenMP for Emscripten [1], a direct response to linking issues with 32-bit binaries that threatened the entire browser inference pipeline. Simultaneously, llama.cpp shipped production-ready MCP stdio transport [2], letting the server spawn and manage subprocesses that expose language model tools through standardized JSON protocols. On the acceleration front, Hexagon backend added partial im2col support [3] for patch-embed operations, delivering measurable 1.5x to 3.5x per-op latency uplift across Qwen3 VL and SigLIP vision encoders. The stack also bumped cpp-httplib to 0.51.0 [4], a vendor dependency update carrying standard maintenance and bug fixes. Across vLLM, the Transformers backend now gates audio model support behind Transformers compatibility [5], while a new import enforcement rule [6] forbids bare hugging_face imports to route all Hub requests through controlled API surfaces. A critical tool parser fix [7] stopped vLLM from dropping streaming arguments in Jamba and InternLM2 when JSON re-serialization didn't preserve raw delta text verbatim. In SGLang, Marlin MoE got occupancy-aware kernel specialization [8] that removes per-block branches from hot loops, while Mamba support landed in the Unified Radix Cache [9] and speculative decoding gained both KDA target verification [10] and grammar-constrained DFLASH support [11]. A scheduler livelock fix [12] resolved SWA admission spinning on cached-prefix resumes.
One email a day. Unsubscribe in one click.
Keep up with Local LLMs in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
One email a day. Unsubscribe in one click. Read a past issue →
Action items
- → Test WASM builds against Emscripten toolchain before next release ggml-org/llama.cpp [plan]
- → Review MCP stdio subprocess isolation and error handling in production deployments ggml-org/llama.cpp [plan]
- → Verify vLLM audio model gating against Transformers PR 45326 merge status vllm-project/vllm [monitor]
- → Deploy SGLang scheduler fix to prevent SWA admission livelock in hybrid setups sgl-project/sglang [immediate]
References
- [1] ggml-webgpu: Fix WASM compilation with OpenMP ↗ ggml-org/llama.cpp
- [2] server: support MCP stdio ↗ ggml-org/llama.cpp
- [3] hexagon: partial im2col support ↗ ggml-org/llama.cpp
- [4] vendor : update cpp-httplib to 0.51.0 ↗ ggml-org/llama.cpp
- [5] feat[vLLM × v5]: Add audio support for the Transformers backend ↗ vllm-project/vllm
- [6] Make bare `hugging_face` imports forbidden ↗ vllm-project/vllm
- [7] [Bugfix][Tool Parser] Fix dropped streaming arguments in Jamba and InternLM2 parsers ↗ vllm-project/vllm
- [8] [Performance] Speed up Marlin MoE with occupancy-aware launch specialization ↗ sgl-project/sglang
- [9] [Hicache][1/2]Support Mamba branching in Unified Radix Cache with HiCache ↗ sgl-project/sglang
- [10] [KDA] Add target_verify support for speculative decoding ↗ sgl-project/sglang
- [11] [DFLASH] Support grammar-constrained decoding in speculative verify ↗ sgl-project/sglang
- [12] Fix SWA admission livelock on cached-prefix resumes ↗ sgl-project/sglang