$ the-wire · showcase
TRL fixes strided fused logprob backward, AsyncGRPO mitigates FP8 sampler collapse
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
13 people shipped this
winglian
@winglian
1 cited
AmineDiro
@AmineDiro
1 cited
qgallouedec
@qgallouedec
1 cited
Ziyu Lin
@LinZiyuu
1 cited
gorkadabo1
@gorkadabo1
1 cited
yiyixuxu
@yiyixuxu
sywangyi
@sywangyi
adarshxs
@adarshxs
Kashif Rasul
@kashif
danieldk
@danieldk
NikodemBartnik
@NikodemBartnik
Maximellerbach
@Maximellerbach
Caroline Pascal
@CarolinePascal
Correctness work dominates the day: TRL repairs a backward pass that read wrong rows under activation offloading and re-centers AsyncGRPO's old log-probs when the sampler runs lower precision than the trainer, while diffusers closes an import vector that let a model repository execute code.
Merge commit from fork huggingface/diffusers
Library names from model_index.json and modular_model_index.json no longer reach importlib.import_module, and modular_config.json requirements no longer reach importlib.util.find_spec. A repo sitting on a sys.path root (a git clone or hf download --local-dir into the working directory) can no longer get modules imported just by being loaded.
Fix/fused logprob entropy strided huggingface/trl
_LogProbEntropyFunction cached the strides of logits, index and row_mask during forward, but the offloader restores the shifted views SFT and DPO trainers pass (logits[:, :-1], input_ids[:, 1:]) as contiguous copies. Backward then read every row after the first batch at the wrong offset and the last batch past the end of the buffer. This is the fix for fused selective_log_softmax under OffloadA...
Asyncgrpo score centering huggingface/trl
AsyncGRPO's old log-probs come from the vLLM sampler, so its ratio bundles staleness with server-trainer numerical mismatch. On the sanity bench with an FP8 sampler and a bf16 trainer, reward fell from 0.60 to 0.34, or 0.10 with an FP8 KV cache too, by step 300, against 0.71 with a bf16 sampler; capping or masking the ratio alone did not prevent it.
Fix shared LoRA component weight broadcasting huggingface/diffusers
When set_adapters() was handed a single component-weight dict for several LoRA adapters, only the first got the requested weight and the rest silently used the default 1.0. Relying on a one-key dict to broadcast no longer drops your weights.
Remove the experimental TPO trainer huggingface/trl
trl.experimental.tpo is gone: TPOTrainer, TPOConfig, its test, its doc page and examples/tpo_ultrafeedback/ are all removed, after nine trainings in the last 30 days and no model tagged. The losses are the ones CPOTrainer already has. Also in this pass: diffusers now treats an InputParam that is both required=True and carries a default as optional and warns instead of dropping the default, matc...
