$ the-wire · showcase
TRANSFORMERS DROPS TORCH 2.4 SUPPORT; DIFFUSERS FIXES LORA COMPILATION BUGS
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
13 people shipped this
qgallouedec
@qgallouedec
2 cited
jiqing-feng
@jiqing-feng
1 cited
Cyrilvallez
@Cyrilvallez
1 cited
daviswer
@daviswer
1 cited
Harry Mellor
@hmellor
1 cited
Hameed Ibrahim
@hameedibrh
1 cited
ydshieh
@ydshieh
1 cited
pjdurden
@pjdurden
1 cited
delmalih
@delmalih
stevhliu
@stevhliu
akshan-main
@akshan-main
thomwolf
@thomwolf
lvwerra
@lvwerra
Hugging Face is narrowing its PyTorch version floor while shipping critical fixes across diffusers, transformers, candle, and trl that address tensor allocation, batching, and generation pipeline bugs.
Transformers is ending torch 2.4 support [1], a housekeeping move that signals the team is moving past legacy PyTorch versions. More urgent: a CUDA Graph fix now makes Gemma 4 compilable by solving host-to-device copy issues from scalar tensor allocation [2], and a new vectorized NoRepeatNGramLogitsProcessor removes a performance-blocking host sync point [3]. Meanwhile, diffusers fixed a critical LoRA hot-swapping bug where models with `different_shapes_for_compilation` failed on recompile with tensor size mismatches [4]. Candle patched Qwen3 batching: causal masks were built batch-independently but reshaped as if fully batch-aware, causing batch rows past the first to read garbage and return wrong output [5]. On the feature side, transformers now supports Granite-swa and Granitemoe-swa models with sliding window attention and learnable per-head sinks [6], and a major Rotary module simplification removes redundant device/dtype casting [7]. TRL's DistillationTrainer refactor is in motion: generation switched to GRPO's stack with a deletion-heavy 193/537 PR [8], and a new `_get_last_hidden_state` method lands unwired but ready for chunked JSD loss [9].
Action items
- → Patch diffusers immediately if you use LoRA hot-swapping with different_shapes_for_compilation huggingface/diffusers [immediate]
- → Recompile Candle Qwen3 models after the batch mask fix if you rely on batch inference huggingface/candle [immediate]
- → Review Rotary module simplification in transformers; breaking change removes device/seq_len arguments huggingface/transformers [plan]
- → Watch TRL DistillationTrainer refactor stacking; generation cutover now live huggingface/trl [monitor]
References
- [1] byebye torch 2.4 ↗ huggingface/transformers
- [2] Fix CUDA Graph breaking host to device copy from scalar tensor allocation (#47547) ↗ huggingface/transformers
- [3] Vectorize NoRepeatNGramLogitsProcessor and remove its host sync (#47571) ↗ huggingface/transformers
- [4] Fix LoRA hot-swapping recompilation with `different_shapes_for_compilation` (#14297) ↗ huggingface/diffusers
- [5] fix(qwen3): build causal mask batch-independently (#3582) ↗ huggingface/candle
- [6] Add Granite-swa and Granitemoe-swa model support ↗ huggingface/transformers
- [7] Simplify all Rotary modules ↗ huggingface/transformers
- [8] [DistillationTrainer refactor] Switch generation to GRPO's stack; delete the buffer ↗ huggingface/trl
- [9] [DistillationTrainer refactor] Add `_get_last_hidden_state` (unwired) ↗ huggingface/trl
