$ the-wire · showcase
Five vLLM crash and forgery paths, realtime transcription lands in llama.cpp
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
vLLM's default configuration took five separate single-request kill paths to the engine in one disclosure batch, while llama.cpp shipped a realtime transcription endpoint and mlx found a 2.07x short-input win on M5.
Permissive default CORS origins allow cross-origin requests to the vLLM HTTP server vllm-project/vllm
vLLM's defaults ship with permissive CORS origins, so arbitrary web pages can reach the HTTP server cross-origin. Treat any internet-reachable vLLM endpoint as unauthenticated surface until you pin an explicit allowlist.
Public multimodal SHM handle forgery reaches worker-side `pickle.loads()` with insecure serialization disabled vllm-project/vllm
A forged public multimodal SHM handle still reaches worker-side pickle.loads() even with insecure serialization disabled, so the deserialization guard is not the boundary you may have assumed.
DeepSeek-OCR family: an image whose normalized pixels sum to exactly 0 makes embed_multimodal return None, tripping the encoder-output sanity assert — one unauthenticated request kills the whole vLLM vllm-project/vllm
In the DeepSeek-OCR family, an image whose normalized pixels sum to exactly 0 makes embed_multimodal return None, tripping the encoder-output sanity assert. One unauthenticated request kills the whole engine under the default config, and the Phi-4 audio encoder has a comparable path.
[Metal] Add staged affine_qmm_t_nax kernels with split-k ml-explore/mlx
A split-K NAX route plus a 16-row simdgroup schedule for transposed affine quantized_matmul puts an 85-token forward at 14.5 ms instead of 30.1 ms on upstream main, 2.07x with a 95% CI of [2.05, 2.09] on an M5 Max. The gain narrows as inputs get longer.
[Chore] attention: Remove the tokenspeed_mla backend and dependency sgl-project/sglang
sglang drops the tokenspeed_mla backend and the tokenspeed_mla==0.1.8 pin that every pip install pulled in, since cutedsl_mla already covers FP8 or BF16 KV, page size 32/64, decode context parallelism and speculative verify of any width on Blackwell. Elsewhere, llama.cpp added a realtime transcription API with API-key middleware answering 401 at the websocket handshake, and Ollama made runnerRe...
Action items
- → Set an explicit CORS origin allowlist on vLLM HTTP servers instead of the permissive default vllm-project/vllm [immediate]
- → Pin vLLM to a release without the multimodal SHM forgery and DeepSeek-OCR None-output crash paths before exposing the... vllm-project/vllm [plan]
