RepoJournal

$ cat alignment/week/2026-11-16.log

Alignment & Post-training

Alignment & Post-training

the week in review · Nov 16 – Nov 22, 2026

verl removes V0 trainer, V1 becomes only PPO path

By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology

♥

The same release train broke DataProto auto padding and shard dispatch: batches must now divide evenly by world size.

39 commits 39 PRs merged 7 briefings covered

all alignment reviews →

[BREAKING][trainer, recipe, ci, doc] refactor: remove V0 trainer and migrate legacy async trainers to recipes verl-project/verl

by Begunner

V1 is now the only PPO trainer in the main tree. The legacy fully async and one-step-off implementations live in verl-recipe, where they pin verl v0.9.1, so anyone still running those flows either moves onto V1 or tracks a frozen dependency.

[BREAKING][algo, cfg] fix: validate distillation estimator update modes verl-project/verl

by Eppiehu

The PR proposes explicit validation of k1/k3 estimator and update-mode combinations, because a detached k3 reward does not give the reverse-KL policy gradient even at an exactly on-policy update with no clipping. If your distillation config pairs those modes today, expect it to be rejected rather than silently trained wrong.

[BREAKING][single_controller, data, doc] refactor: remove DataProto auto padding verl-project/verl

by vermouth1992

DP_COMPUTE_PROTO now splits with _split_args_kwargs_data_proto and requires len(batch) % world_size == 0; explicit DataProto.padding, pad_dataproto_to_divisor, and unpad_dataproto stay. Auto padding was duplicating samples, so check that your per-rank batch sizes divide evenly before you upgrade.

[BREAKING][single_controller, worker, trainer] feat: dispatch shards only to the model-parallel source rank (#8708) verl-project/verl

by Chi Zhang

An optional path sends each data-parallel shard only to the model-parallel source rank instead of copying it to every rank in the group, changing the dispatch contract for custom workers and trainers. Flagged breaking, so review any worker code that assumes every rank receives the shard.

fix(agent_loop): honor data.truncation policy in _cap_text_prompt_length verl-project/verl

by Jackie2049

The shared prompt cap in AgentLoopBase unconditionally left-truncated overlong text prompts, ignoring the documented data.truncation policy of error, left, right, or middle. With error (the default) configured, overlong prompts now fail loudly instead of quietly losing their prefix.

[rollout, vllm, sglang, trtllm, cfg] feat: salt external KV caches by weight version instead of resetting them verl-project/verl

by daegyu94

On every weight update verl dropped old-policy KV with a hard reset, which does not reliably reach KV connectors, offload tiers, or HiCache storage. Salting by weight version keeps those external caches coherent, which matters if your rollout stack has prefix KV living outside GPU memory.

[hardware] fix: add B300 and GB300 peak FLOPS (#8772) verl-project/verl

by hxw12219

B300 and GB300 were missing from _DEVICE_FLOPS, so get_device_flops() returned inf on them and perf/mfu/actor and perf/mfu/actor_infer always reported 0. Those two metrics now produce real numbers on Blackwell hardware.

MILES: commit recovery checkpoints before evaluation and extend Small heartbeats allenai/open-instruct

by no0p

A run reached optimizer update 100 and lost its allocation during the native recovery save, because the evaluation hook had already exported weights inside the optimizer loop. Checkpoints are now committed before evaluation, so exporting weights is off the critical path to recoverable state. Alongside: MILES Tiny templates moved to 2,048 responses per update with LR 6e-6, bounded recovery saves...

$ ls alignment/week/ # the briefings behind this review

Keep up with Alignment & Post-training in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

all alignment reviews →