$ the-wire · showcase
TRL removes PPOTrainer, vendors Liger-Kernel losses
By RepoJournal · Filed · About Hugging Face
TRL removed PPOTrainer and vendored its fused linear losses from Liger-Kernel, while Transformers stopped synchronizing the accelerator on each decode step and Diffusers added a modular blockset for Wan 2.2 VACE.
TRL dropped PPOTrainer [1], retiring the classic RLHF trainer after its long deprecation. The library also moved all fused linear losses (DPO, KTO, GRPO, JSD) out of `liger-kernel` into `trl.losses` using vendored code, since upstream reviews had stalled and most of the variants were written for TRL [2]. Alongside that, `DistillationTrainer` now always uses the existing token-chunked JSD path, removing the duplicated fused-JSD branch [3]. and hardens `_ChunkedLogProbFunction.backward` with LM-head bias support and weight re-gathering [4].
Transformers streamlines generation: `generate` no longer synchronizes the accelerator on every decode step, and streamed tokens now ride the same pipeline to avoid host reads per step, improving throughput for chat clients [5]. The Mamba2 mixer in Nemotron-H, Falcon-H1, and Mamba2 now forwards `kwargs` to the linear-attention branch, so kernel kwargs like `seq_idx` reach the mixer as they already did in Bamba and Zamba2 [6]. Sliding-window mask `layer_idx` handling in Gemma3/Gemma4 vision models is fixed [7].
Action items
- → Update TRL code that references PPOTrainer to a supported trainer or migrate off it huggingface/trl [immediate]
- → Review TRL imports of liger_kernel.chunked_loss; the code now lives in trl.losses huggingface/trl [plan]
- → Test generation performance with streamers after the decode-step pipeline change huggingface/transformers [monitor]
References
- [1] Remove PPOTrainer (#7020) ↗ huggingface/trl
- [2] Vendor the fused linear losses from Liger-Kernel into `trl.losses` ↗ huggingface/trl
- [3] Drop fused JSD from DistillationTrainer ↗ huggingface/trl
- [4] Harden chunked log probabilities ↗ huggingface/trl
- [5] [generate] stop synchronizing the accelerator on every decode step (#47975) ↗ huggingface/transformers
- [6] Pass kwargs to the Mamba2 mixer in Nemotron-H, Falcon-H1 and Mamba2 (#48490) ↗ huggingface/transformers
- [7] Fix sliding-window mask `layer_idx` in Gemma3/Gemma4 `create_masks_for_vision_model` ↗ huggingface/transformers
