$ the-wire · showcase
vLLM fixes spec-decode logprobs and tool references; MLX takes partial RoPE
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
14 people shipped this
anencore94
@anencore94
1 cited
SCcagg5
@SCcagg5
1 cited
vMaroon
@vMaroon
1 cited
yihong-w
@yihong-w
1 cited
shadyuwugurl
@shadyuwugurl
1 cited
Sigbjørn Skjæret
@CISC
DongjiGao
@DongjiGao
esmeetu
@esmeetu
CaoE
@CaoE
xiaofei-zheng
@xiaofei-zheng
Codoge
@Codoge
mickqian
@mickqian
XXXXRT666
@XXXXRT666
mycomputercrashsalot
@mycomputercrashsalot
vLLM closed three correctness gaps that bit real traffic (dropped logprobs under speculative decoding, 400s on DeepSeek V4.1 tool references, malformed Responses API history) while MLX moved partial rotary encoding out of user Python and into nn.RoPE.
[Bugfix][Spec Decode] V2 rejection sampler: return logprob_token_ids scores vllm-project/vllm
With the V2 model runner and speculative decoding, a request setting `logprob_token_ids` (and no `logprobs`) got nothing back whenever another request in the same batch carried draft tokens, because `RejectionSampler` only consulted `max_num_logprobs`. It now uses the same `get_logprobs_dims` path the regular V2 `Sampler` already had, so batched scoring calls stop silently losing their scores.
[Bugfix][Frontend] Fix tool_reference results on DeepSeek V4.1 and /v1/messages vllm-project/vllm
The DeepSeek V4.1 encoder rejected `tool_reference` content parts with a 400, and `/v1/messages` split one `tool_result` carrying references into two `tool` messages per call. Both are fixed, so Claude Code's tool search now runs against V4.1 and renders one result per tool call.
Add freqs and partial_rotary_factor support to RoPE ml-explore/mlx
`mlx.nn.RoPE` forwarded only `dims`, `traditional`, `base` and `scale`, so any rotary encoding that was not a plain power law had to be reimplemented in Python on top of MLX. It now exposes `freqs` (already accepted by `mx.fast.rope`) plus a `partial_rotary_factor` convenience that builds the wavelengths for a proportional rotary encoding.
[Responses] Normalize OpenResponses input items to match openai-python SDK requirements vllm-project/vllm
Responses API requests with incomplete assistant or reasoning history no longer lose turns or crash on empty assistant content: missing fields are filled through the existing `input_item_parsing` validator, the message discriminator is preserved so typeless assistant turns keep their content, and empty tool-only turns are handled. Related to openai/codex#12669, so agent clients round-tripping p...
[Fix] mem_cache: Skip Mamba checkpoint donation when slots are exhausted sgl-project/sglang
Optional Mamba checkpoint donation could exhaust the state pool when every slot was held by active requests or locked prefixes and terminate the scheduler; unfinished-request donation is now skipped when allocation stays unavailable after eviction, while slots needed by active state or load-back still fail strictly. Also this cycle: XPU full prefill graph capture and replay landed in SGLang, ai...
