RepoJournal
Alignment & Post-training Alignment & Post-training
11 wires and counting

$ follow Alignment & Post-training

Keep up with Alignment & Post-training in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-10-02
stories 2
RSS Archive Reviews

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Score centering lands for bypass-mode REINFORCE in verl

By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology

vOv
1 person shipped this
vOv @cr-gao 2 cited

verl added a hyperparameter-free correction that neutralizes the sampler drift term in bypass-mode REINFORCE, so trainers stop distilling toward the quantized or stale sampler.

[rollout, vllm, algo, fsdp, trainer, doc] feat: score centering for bypass-mode REINFORCE verl-project/verl

by cr-gao

Bypass-mode REINFORCE now subtracts the sampler's expected score from each token's score, removing the drift term that pulled the trainer toward a quantized or stale sampler under training-inference mismatch. The correction is additive and hyperparameter-free, and it is a no-op when you are running on-policy, so operators can enable it without retuning existing runs.

[rollout, vllm, algo, fsdp, trainer, doc] feat: score centering for bypass-mode REINFORCE (#8010) verl-project/verl

by vOv

The change implements RFC #8009 and follows Marek & Ryabinin, 2026, touching rollout, vLLM, algo, FSDP, trainer, and doc paths. If you run bypass-mode REINFORCE with a separate inference engine, this is the knob to pick up on your next environment rebuild.

$ ls alignment/reviews/ # the step-back reads

the week in review verl adds TPU SFT via TorchTitan and ETP checkpoint conversion
Full archive → Weekly reviews →

Quick answers

What shipped in Alignment & Post-training on October 2, 2026?
verl added a hyperparameter-free correction that neutralizes the sampler drift term in bypass-mode REINFORCE, so trainers stop distilling toward the quantized or stale sampler. In total, 1 commits and 1 pull requests landed.
Who contributed to Alignment & Post-training on October 2, 2026?
1 developer shipped this update, including vOv.
What were the notable Alignment & Post-training updates?
[rollout, vllm, algo, fsdp, trainer, doc] feat: score centering for bypass-mode REINFORCE and [rollout, vllm, algo, fsdp, trainer, doc] feat: score centering for bypass-mode REINFORCE (#8010).
♥

$ log Alignment & Post-training --recent

verl rejects conflicting vLLM engine_kwargs, open-instruct adds MILES OPD behind a Megatron learner 8 shipped FSDP2 CPU offload was silently corrupting parameters, and MILES gets FlashAttention 8 shipped Fused cross-entropy lands on Ascend NPUs and Megatron, checkpoint offload stops assuming CUDA 26 shipped DeepSeek MTP support fixed for vLLM and Megatron backends 10 shipped

$ log --all # elsewhere on the wire

Interpretability nnsight 0.8.0 cuts import time from 7.6s to 1.6s Agent Security Approval digests get RFC 8785 float serialization; sandbox-runtime 0.0.80 kills a 300 ms path

$ ls showcases/

Safety Evals AI Red Teaming Agent Security Interpretability

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.