$ the-wire · showcase
Ollama stops dropping tool schema keywords, vLLM patches hybrid KV cache paging
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
Ollama now round-trips every keyword in a tool's JSON Schema instead of only the seven it modeled, while vLLM fixed KV cache page selection for speculative decoding on hybrid attention targets and removed a serving-time recompile stall in DiffusionGemma.
api: preserve unmodeled tool schema keywords ollama/ollama
Ollama decoded tool parameter schemas into api.ToolProperty, which has fields for only type, description, enum, items, properties, required and anyOf; Go's JSON decoder silently discarded every other key, so default and minimum were gone by the time the schema was written back out. Schemas now preserve the unmodeled keywords, which matters if your client round-trips tool definitions and relies ...
[Bugfix] Fix KV cache pages for hybrid models with draft layers vllm-project/vllm
Fixes KV cache page selection when a model mixes layer types, the case speculative decoding with a draft model on a hybrid attention-plus-linear-attention target hits: FlashInfer now chooses kernel pages per layer, padded manager pages can split into kernel pages, and hybrid block alignment counts the drafter's KV.
[Bugfix][Model] DiffusionGemma: stop sampler recompiles during serving vllm-project/vllm
DiffusionGemma's sampler ran its post-sample step, softcap and two helpers through @torch.compile(dynamic=True) outside vLLM's compile pipeline, so each new request shape compiled during serving and stalled every request on the replica. The revision drops torch.compile from the sampler and makes the step and softcap Triton kernels that register their warmup inputs with vLLM's JIT warmup registry.
[Bugfix][HiSparse] Fix GLM-5.3-Flash startup: derive resident/hot specs without grouping on workers vllm-project/vllm
GLM-5.3-Flash with HiSparse failed at startup on main with ValueError: KV cache layout has not been resolved yet, because workers called resolve_hisparse_specs before the engine core resolved the layout. Resident and hot specs are now derived without grouping on workers.
[Bugfix] Apply `WeightsMapper` stacked rules last, and fix the `--mm-encoder-only` shared root check vllm-project/vllm
WeightsMapper now applies orig_to_new_stacked after every other rule; previously stacked rules renamed .k_proj.output_scale to .qkv_proj.output_scale before the Fp8 and Quark suffix rules could match it, breaking KV cache scale names on affected checkpoints. The --mm-encoder-only shared root guard now checks the checkpoint's keys.
[Feat] disaggregation: Add opt-in Mooncake TENT transfer engine backend for PD sgl-project/sglang
Elsewhere, sglang added Mooncake's TENT transfer engine as an opt-in PD connector backend, reporting lower KV transfer latency under NIC congestion on one node, with classic TE staying the default; llama.cpp moved moe-cache code out of llama-graph, optimized grammar matching via left factoring and pruned old files from ./examples; and Ollama's schema fix landed on main.