RepoJournal

$ cat alignment/week/2026-11-09.log

Alignment & Post-training

Alignment & Post-training

the week in review · Nov 9 – Nov 15, 2026

MILES adopts anchored Math-Verify grading and fixes reward bugs

By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology

♥

verl shipped a batch-invariant actor forward, an MXFP4 MoE recipe and a TileLang fused cross-entropy head over the same week.

57 commits 57 PRs merged 7 briefings covered

all alignment reviews →

Use anchored Math-Verify grading by default in MILES allenai/open-instruct

by no0p

MILES math grading now routes through pinned Hugging Face Math-Verify 0.9.0 with anchored prediction extraction, replacing a scalar-oriented verifier that could reward an incomplete root list and reject equivalent structured answers. Scores you collected under the old verifier are not comparable, so re-check any math run you are using as a baseline.

Put ToolRL rewards on the unit scale and gate truncation at the verifier floor allenai/open-instruct

by no0p

MILES' truncation gate assumed zero was the floor score, which fails once ToolRL's negative rewards enter the mix: training could credit a response for hitting the length limit rather than completing a wrong answer. ToolRL rewards now sit on the unit scale, so mixed-verifier reward totals shift if you tuned thresholds against the old scale.

Guard unsafe Math-Verify equations with cached legacy comparison fallback allenai/open-instruct

by no0p

A reference like f(1)=1 was accepting f(1)=0 or 3+4=12, because Math-Verify 0.9.0's equation-solving fallback treats two empty solution lists as equivalent. The new guard detects unsafe equation references and falls back to a cached legacy comparison, which changes scores on equation-heavy math sets.

Strip selected terminal special tokens before MILES grading allenai/open-instruct

by no0p

The reward adapter passed rollout text to graders with generation stop tokens still attached, so a correct answer ending in <|endoftext|> failed an instruction to end with a given phrase. Quotation, last-word and JSON checks were all affected; if your graders test final-string formatting, re-run them after this lands.

[fsdp, trainer, cfg, doc, tests] feat: batch-invariant actor forward and sampler log-prob formula verl-project/verl

by cr-gao

A new opt-in FSDP engine option, batch_invariant (default off), installs vLLM's matmul, softmax, mean and RMSNorm kernel overrides plus the cuBLAS and TF32 settings, and computes log-probs with the sampler formula. Turn it on when training and rollout numerics must match; leave it off to keep the default throughput.

[perf, fsdp, megatron, veomni] feat: add TileLang fused linear cross entropy verl-project/verl

by wuxibin89

A TileLang fused linear cross-entropy output head now works across FSDP, Megatron and VeOmni, and supports multi-label and top-k log probabilities with repeated labels so distillation and score-centering losses consume precomputed values. It is the head-level memory saving rather than a new training mode, so existing recipes pick it up by config.

[megatron, doc] feat: MXFP4 MoE recipe and Qwen3-30B-A3B example verl-project/verl

by 0z5a

The MXFP4 counterpart to the MXFP8 recipe targets the routed experts of MoE models, shipping with a Qwen3-30B-A3B example: GRPO on DAPO-Math-17k across one 4-GPU Blackwell node. Operators on Blackwell gain the low-precision option without changing the rest of the recipe.

[hardware, recipe] feat: unify dense Qwen3 (0.6B to 32B) TPU GRPO recipes into run_qwen3_torchtitan.sh verl-project/verl

by wenjung2007

The dense Qwen3 TPU GRPO scripts in examples/tpu/grpo/ collapse into a single run_qwen3_torchtitan.sh, selected by MODEL_SIZE=0.6B, 1.7B, 4B, 8B, 14B or 32B, replacing the per-size launchers you currently invoke. Alongside: verl also landed score centering on the Megatron actor, a Qwen3-30B-A3B TPU GRPO recipe, and a Minerva fix that stops bold Answer: labels from staining the extracted answer.

$ ls alignment/week/ # the briefings behind this review

Keep up with Alignment & Post-training in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

all alignment reviews →