$ the-wire · showcase
open-instruct exposes FP32 vocabulary-head output in Small MILES examples
By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology
One change lands in allenai/open-instruct today: the Small MILES examples can now emit the vocabulary head in FP32, a switch that matters only to operators running those examples on hardware where the logits need to stay full precision.
Enable FP32 vocabulary-head output in Small MILES examples (#2309) allenai/open-instruct
The Small MILES examples now support FP32 output from the vocabulary head, so logits are no longer forced down to reduced precision on that path. If your runs hit instability or numeric drift at the head, this is the flag to reach for before you start hand-patching the loss.