$ the-wire · showcase
vLLM patches ZMQ socket exhaustion and CORS defaults, Ollama moves decision models into llama.cpp
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
14 people shipped this
Juan Pérez de Algaba
@jperezdealgaba
2 cited
jmorganca
@jmorganca
1 cited
stu-cao
@stu-cao
1 cited
niehen6174
@niehen6174
1 cited
Prudctual
@Prudctual
1 cited
Georgi Gerganov
@ggerganov
kurquhar
@kurquhar
ngxson
@ngxson
jeffbolznv
@jeffbolznv
bri-prism
@bri-prism
hubert-marek
@hubert-marek
aoshen02
@aoshen02
hnyls2002
@hnyls2002
SleepAviator
@SleepAviator
Two vLLM security fixes tighten defaults that shipped open, while Ollama hands decision-model prompting to llama.cpp b11521 and drops its own scorer.
[Security] Fix P2P ZMQ socket exhaustion via unbounded peer sessions (#51504) vllm-project/vllm
vLLM bound the peer sessions behind its P2P ZMQ transport, which previously let unbounded sessions exhaust sockets. If you run P2P serving, this is the fix to take.
[Security] Restrict default CORS origins (#61009) vllm-project/vllm
The default CORS origins are now restricted rather than wide open, so browser clients calling the server from an origin outside the new allowlist will be blocked until you configure it explicitly. This is a breaking change to default behavior.
llama: update llama.cpp to b11521 for decision models and EmbeddingGemma 2 ollama/ollama
Ollama updated llama.cpp to b11521, so GGUF decision models now send /v1/systemone requests straight to llama.cpp, which builds the prompt itself; Ollama's own llama-server scorer and the Clef patch are gone. Questions within a request run in parallel, and the scheduler sizes the batch to context for Clef, Laya and LFM2-D1-Omni, which read the whole prompt in one batch.
[Bugfix][Mooncake] Use the EAGLE replay boundary when retaining sliding-window tails vllm-project/vllm
Under EAGLE, MooncakeStoreCoordinator.store_mask masked at the non-EAGLE replay boundary, one scheduler block past where the core's replay_boundaries places it, so the store kept different sliding-window tails than the local prefix cache. The store now uses the EAGLE boundary.
fix metal conv bounds checks and integer overflows ml-explore/mlx
Three bounds and overflow fixes in MLX's Metal conv path: a widened diff that wrapped negative past 32767 output channels per group and left upper channels unwritten on GPU, plus bounded weight loads in loader_channel_n.h and loader_channel_l.h. Apple-silicon conv workloads get correct output where they previously silently lost channels.
[diffusion] perf: layerwise offload for the LTX-2.5 Gemma-4 text encoder sgl-project/sglang
Elsewhere: SGLang added layerwise offload for LTX-2.5's Gemma-4-Unified text encoder, a 23.9 GB bf16 module (21.8 GB of decoder layers, 2.0 GB vocab) that previously raised ComponentResidencyError under --layerwise-offload-components text_encoder, and llama.cpp turned GGML_RPC_DEBUG into a 0-3 verbosity level behind a shared log.h.
Action items
- → Set explicit CORS allowed origins for vLLM servers before upgrading; the default allowlist is now restricted vllm-project/vllm [plan]
