$ the-wire · showcase
verl salts external KV caches, and a k3 config proposal lands as breaking
By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology
verl's rollout engines can stop leaking stale old-policy KV into trainers, while its distillation estimators now get validated in a way that will reject existing k3+policy-gradient configs.
[rollout, vllm, sglang, trtllm, cfg] feat: salt external KV caches by weight version instead of resetting them verl-project/verl
Rollout engines that keep prefix KV outside GPU memory (vLLM's LMCacheMPConnector and HF3FSKVConnector in 0.29 and 0.31, SGLang via flush_cache, and HiCache tiers) never actually saw verl's reset_connector=True hard reset, so they kept serving old-policy KV after weight updates. Those caches are now salted by weight version instead of reset, which changes what a stale-cache bug looks like: chec...
[BREAKING][algo, cfg] fix: validate distillation estimator update modes verl-project/verl
The PR says a detached k3 reward does not give the reverse-KL policy gradient, even at an exactly on-policy update with no clipping, and can ascend that KL. Validation now rejects k3 with low_var_kl when use_policy_gradient=True, so if your config combines those, it will fail to start; the proposal is pending maintainer agreement on intended k3+PG semantics, and the loss kernels and PPO impleme...
[trainer] fix: skip rows without valid tokens in the IS and RS sequence metrics verl-project/verl
Rows whose response_mask is all zero were skipped by some per-sequence off-policy metrics after the earlier fix, but rollout_is_seq_*, the sequence-mode rollout_is_max and rollout_is_min, rollout_is_ratio_fraction_*, and rollout_rs_seq_masked_fraction still reduced over every row. Those metrics now skip rows with no valid tokens, which means your IS/RS curve values and thresholds shift on datas...
[hardware] fix: install TransferQueue 0.1.11 in the TPU GRPO image (#8627) verl-project/verl
examples/tpu/grpo/Dockerfile pinned TransferQueue==0.1.8 while the verl-core extra requires TransferQueue>=0.1.11, and the image now installs 0.1.11. If you built TPU GRPO V1 async runs from the old image, your first checkpoint failed with an AttributeError because the checkpoint API only arrived in 0.1.9; rebuild from the updated Dockerfile.
MILES: Small B300 recipes and terminal refresh cleanup allenai/open-instruct
Measured Olmo 3.5 Small (A4B, 72.58B total parameters) B300 recipes arrive under configs/miles/examples/small/ for two-, four-, and eight-node layouts, with SGLang fused-MoE tiles for E=256,N=3072, plus a terminal refresh cleanup so completed runs cancel unused generation before closing weight transport. No Small-specific model adapter or weight-transfer implementation is added, so you still su...
Action items