$ cat alignment/week/2026-09-28.log
the week in review · Sep 28 – Oct 4, 2026
verl adds score centering to bypass-mode REINFORCE
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
The RFC #8009 implementation lands on the REINFORCE loss path, and verl flags it as breaking for existing training configs.
[rollout, vllm, algo, fsdp, trainer, doc] feat: score centering for bypass-mode REINFORCE verl-project/verl
Score centering is now part of the bypass-mode REINFORCE loss, implementing RFC #8009. The motivation is concrete: under training-inference mismatch the policy gradient carries a drift term, E_q[R] * E_q[∇log p], that distills the trainer toward the rollout model, and centering removes it. If your runs sit on the REINFORCE bypass path, the losses your dashboards have been showing will not match...
[rollout, vllm, algo, fsdp, trainer, doc] feat: score centering for bypass-mode REINFORCE (#8010) verl-project/verl
The same change ships as a breaking item in the module's release stream, so this is not an opt-in experiment you can leave off. Treat it as an upgrade you schedule deliberately: confirm the version you pin, then re-run a short job before turning it loose on a long one.
$ ls alignment/week/ # the briefings behind this review
Keep up with Alignment & Post-training in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.