$ the-wire · showcase
MILES reward fixes land: truncation gate, stop tokens, ToolRL scale
By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology
Three reward-path fixes in open-instruct remove ways a correct or completed response could be scored wrong, and one of them made hitting the length limit better than answering incorrectly.
Put ToolRL rewards on the unit scale and gate truncation at the verifier floor allenai/open-instruct
MILES' truncation gate assumed zero was the floor, but ToolRL scores call rows from -3 to 4, so a truncated response outranked every completed wrong answer in its prompt group. The MILES binding now maps the reward onto [0, 1] by default and gated responses get their verifier's lowest score, which changes the effective reward scaling you trained against if you ran mixed-verifier ToolRL before.
Align prepared tool data with checkpoint templates and XML grading allenai/open-instruct
Prepared Tagliere tool data could select an older Pythonic template or a Pythonic-only structured-answer grader while the Olmo 3.5 checkpoint answers in XML, so correct native tool calls scored zero reward. This supplies the runtime half of the fix, enforces recorded template hashes on resumed prepared caches, and leaves offline rendering and the existing task comparison intact.
Strip selected terminal special tokens before MILES grading allenai/open-instruct
MILES keeps generation stop tokens in rollout text and handed that text straight to graders, so an answer ending correctly still failed an instruction to end with that phrase, and quotation, last-word and JSON checks were affected too. A new response_text module strips selected terminal markers from grading text only, leaving the original rollout and training tokens untouched.
[reward] fix: parse bold Answer labels in math_dapo Minerva scoring verl-project/verl
In verl's math_dapo Minerva scoring, bolds like `**Answer:** 16` and `**Answer**: 16` had their closing `**` left in the extracted answer instead of consumed by the label. The fix is a one-line answer_pattern change with no post-extraction stripping, which matters if you compared reward curves across the two approaches.