$ cat alignment/week/2026-11-16.log
the week in review · Nov 16 – Nov 22, 2026
verl removes V0 trainer, V1 becomes only PPO path
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
The same release train broke DataProto auto padding and shard dispatch: batches must now divide evenly by world size.
[BREAKING][trainer, recipe, ci, doc] refactor: remove V0 trainer and migrate legacy async trainers to recipes verl-project/verl
V1 is now the only PPO trainer in the main tree. The legacy fully async and one-step-off implementations live in verl-recipe, where they pin verl v0.9.1, so anyone still running those flows either moves onto V1 or tracks a frozen dependency.
[BREAKING][algo, cfg] fix: validate distillation estimator update modes verl-project/verl
The PR proposes explicit validation of k1/k3 estimator and update-mode combinations, because a detached k3 reward does not give the reverse-KL policy gradient even at an exactly on-policy update with no clipping. If your distillation config pairs those modes today, expect it to be rejected rather than silently trained wrong.
[BREAKING][single_controller, data, doc] refactor: remove DataProto auto padding verl-project/verl
DP_COMPUTE_PROTO now splits with _split_args_kwargs_data_proto and requires len(batch) % world_size == 0; explicit DataProto.padding, pad_dataproto_to_divisor, and unpad_dataproto stay. Auto padding was duplicating samples, so check that your per-rank batch sizes divide evenly before you upgrade.
[BREAKING][single_controller, worker, trainer] feat: dispatch shards only to the model-parallel source rank (#8708) verl-project/verl
An optional path sends each data-parallel shard only to the model-parallel source rank instead of copying it to every rank in the group, changing the dispatch contract for custom workers and trainers. Flagged breaking, so review any worker code that assumes every rank receives the shard.
fix(agent_loop): honor data.truncation policy in _cap_text_prompt_length verl-project/verl
The shared prompt cap in AgentLoopBase unconditionally left-truncated overlong text prompts, ignoring the documented data.truncation policy of error, left, right, or middle. With error (the default) configured, overlong prompts now fail loudly instead of quietly losing their prefix.
[rollout, vllm, sglang, trtllm, cfg] feat: salt external KV caches by weight version instead of resetting them verl-project/verl
On every weight update verl dropped old-policy KV with a hard reset, which does not reliably reach KV connectors, offload tiers, or HiCache storage. Salting by weight version keeps those external caches coherent, which matters if your rollout stack has prefix KV living outside GPU memory.
[hardware] fix: add B300 and GB300 peak FLOPS (#8772) verl-project/verl
B300 and GB300 were missing from _DEVICE_FLOPS, so get_device_flops() returned inf on them and perf/mfu/actor and perf/mfu/actor_infer always reported 0. Those two metrics now produce real numbers on Blackwell hardware.
MILES: commit recovery checkpoints before evaluation and extend Small heartbeats allenai/open-instruct
A run reached optimizer update 100 and lost its allocation during the native recovery save, because the evaluation hook had already exported weights inside the optimizer loop. Checkpoints are now committed before evaluation, so exporting weights is off the critical path to recoverable state. Alongside: MILES Tiny templates moved to 2,048 responses per update with LR 6e-6, bounded recovery saves...
$ ls alignment/week/ # the briefings behind this review
Keep up with Alignment & Post-training in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.