RepoJournal
Alignment & Post-training Alignment & Post-training
47 wires and counting

$ follow Alignment & Post-training

Keep up with Alignment & Post-training in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-19
stories 18

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

verl salts external KV caches, and a k3 config proposal lands as breaking

By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology

verl's rollout engines can stop leaking stale old-policy KV into trainers, while its distillation estimators now get validated in a way that will reject existing k3+policy-gradient configs.

[rollout, vllm, sglang, trtllm, cfg] feat: salt external KV caches by weight version instead of resetting them verl-project/verl

by daegyu94

Rollout engines that keep prefix KV outside GPU memory (vLLM's LMCacheMPConnector and HF3FSKVConnector in 0.29 and 0.31, SGLang via flush_cache, and HiCache tiers) never actually saw verl's reset_connector=True hard reset, so they kept serving old-policy KV after weight updates. Those caches are now salted by weight version instead of reset, which changes what a stale-cache bug looks like: chec...

[BREAKING][algo, cfg] fix: validate distillation estimator update modes verl-project/verl

by Eppiehu

The PR says a detached k3 reward does not give the reverse-KL policy gradient, even at an exactly on-policy update with no clipping, and can ascend that KL. Validation now rejects k3 with low_var_kl when use_policy_gradient=True, so if your config combines those, it will fail to start; the proposal is pending maintainer agreement on intended k3+PG semantics, and the loss kernels and PPO impleme...

[trainer] fix: skip rows without valid tokens in the IS and RS sequence metrics verl-project/verl

by ethannnnnn

Rows whose response_mask is all zero were skipped by some per-sequence off-policy metrics after the earlier fix, but rollout_is_seq_*, the sequence-mode rollout_is_max and rollout_is_min, rollout_is_ratio_fraction_*, and rollout_rs_seq_masked_fraction still reduced over every row. Those metrics now skip rows with no valid tokens, which means your IS/RS curve values and thresholds shift on datas...

[hardware] fix: install TransferQueue 0.1.11 in the TPU GRPO image (#8627) verl-project/verl

by hx

examples/tpu/grpo/Dockerfile pinned TransferQueue==0.1.8 while the verl-core extra requires TransferQueue>=0.1.11, and the image now installs 0.1.11. If you built TPU GRPO V1 async runs from the old image, your first checkpoint failed with an AttributeError because the checkpoint API only arrived in 0.1.9; rebuild from the updated Dockerfile.

MILES: Small B300 recipes and terminal refresh cleanup allenai/open-instruct

by farhatkevin

Measured Olmo 3.5 Small (A4B, 72.58B total parameters) B300 recipes arrive under configs/miles/examples/small/ for two-, four-, and eight-node layouts, with SGLang fused-MoE tiles for E=256,N=3072, plus a terminal refresh cleanup so completed runs cancel unused generation before closing weight transport. No Small-specific model adapter or weight-transfer implementation is added, so you still su...

Quick answers

What shipped in Alignment & Post-training on November 19, 2026?
verl's rollout engines can stop leaking stale old-policy KV into trainers, while its distillation estimators now get validated in a way that will reject existing k3+policy-gradient configs. In total, 9 commits and 9 pull requests landed.
Who contributed to Alignment & Post-training on November 19, 2026?
5 developers shipped this update, including ethannnnnn, Eppiehu, daegyu94, farhatkevin, and Robert Berry.
What were the notable Alignment & Post-training updates?
[rollout, vllm, sglang, trtllm, cfg] feat: salt external KV caches by weight version instead of resetting them, [BREAKING][algo, cfg] fix: validate distillation estimator update modes, and [trainer] fix: skip rows without valid tokens in the IS and RS sequence metrics.