$ the-wire ยท showcase
TRL defaults padding to "auto" on Neuron, drops vLLM 0.21.0
By RepoJournal ยท Filed ยท About Hugging Face ยท Composed from the cited sources ยท methodology
TRL's Trainium push lands as one coordinated change across padding, kernels and trainers, while transformers removes deprecated cache utils and a PEFT save path that hangs sharded LoRA checkpoints.
Default `pad_to_multiple_of` to `"auto"`: pad to a multiple of 128 on Neuron huggingface/trl
SFT, DPO, KTO, Reward, GRPO, RLOO and Distillation now default pad_to_multiple_of to "auto": 128 on Neuron, None elsewhere, so GPU behavior is untouched and an explicit value is kept as is. In the PR's own measurement on Trainium2 with Qwen3-0.6B SFT over 20 steps, the default finished in 130 s against 260 s with pad_to_multiple_of=None, because Neuron otherwise recompiles on every new sequence...
Save the PEFT adapter of a load-time sharded model from its own parameters huggingface/transformers
With a model sharded at load time via DistributedConfig and a LoRA adapter, the checkpoint save at save_steps hangs on PEFT 0.21.2; the fix saves the adapter from its own parameters rather than the main process only. Anyone training sharded LoRA checkpoints should watch for this PR.
Run the fused LM head and distillation kernels on devices without Triton huggingface/trl
The fused LM head and distillation kernels have required Triton since #7466, which broke SFT, DPO, KTO, GRPO, RLOO and Distillation on accelerators without it. Each per-tile Triton kernel now has a torch counterpart with the same signature, selected when _use_triton(device) is false, with tiling and recompute unchanged and logits still never materialized, though every position is scored on thos...
Use mlp_layer_types for AXK1 so the MoE layer layout honours moe_layer_freq (#49518) huggingface/transformers
AXK1Config now declares moe_layer_freq, scoring_func and topk_method with defaults matching the official checkpoint and validates the supported values, and the MoE layer layout honors moe_layer_freq through mlp_layer_types. Configs that omitted those fields previously fell back to a layout that did not match the checkpoint.
๐จ Remove deprecated cache utils methods huggingface/transformers
Transformers removed deprecated checks and methods from the cache utils, a breaking change if you call those helpers directly; the same day's CI work moved the NVIDIA and AMD DeepSpeed GPU images off Ubuntu 22.04's system Python 3.10 to a Python taken from uv at PYTHON_VERSION in a venv on PATH, leaving the AMD PyTorch image behind because AMD only publishes Python 3.10 flash-attention wheels. ...
Action items
- โ Pin TRL to a version before vLLM 0.21.0 support was dropped if you run against vLLM 0.21.0 huggingface/trl [plan]
- โ Stop calling the deprecated cache utils methods in transformers before upgrading past this removal huggingface/transformers [plan]
