The Wire · Showcase
TRL CI BURNS, BITSANDBYTES PINNED BELOW 0.50.0
By RepoJournal · Filed · About Hugging Face
Hugging Face's TRL training library is firefighting two separate CI failures overnight, forcing temporary pins on bitsandbytes and xfailing critical tests against a broken transformers dev build.
The NemotronH parametrization of both GRPO and RLOO trainers started failing when CI installs the dev dependency stack with kernels [1], marking those tests as expected failures (xfail, strict=True) until the upstream transformers issue resolves [3]. In parallel, a bitsandbytes 0.50.0 incompatibility surfaced in the quantization and dev dependency groups, forcing a temporary pin to versions below 0.50.0 across both dependency sets [2] [4]. These aren't production breaks yet, but they're blocking CI validation for anyone running against the dev branch. Over in pytorch-image-models, Ross Wightman merged a model factory path handler fix that also corrects filename and extension priority bugs [5] [6], addressing issue #2725. The repo also cleaned up snapshot download logic and GitHub CI OIDC configuration [7]. All three TRL fixes came from albertvillanova's rapid response to cascading upstream issues.
One email a day. Unsubscribe in one click.
Keep up with Hugging Face in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
One email a day. Unsubscribe in one click. Read a past issue →
Action items
- → Monitor transformers dev branch for kernel compatibility fix, then remove TRL xfail markers huggingface/trl [plan]
- → Track bitsandbytes release cycle and prepare to remove version pin once 0.50.0+ is stable huggingface/trl [monitor]
- → Test TRL against latest pytorch-image-models factory path changes if you load models from disk huggingface/pytorch-image-models [plan]
References
- [1] Hotfix CI: Xfail NemotronH GRPO/RLOO tests against a broken transformers dev build ↗ huggingface/trl
- [2] Hotfix CI: Temporarily pin bitsandbytes < 0.50.0 ↗ huggingface/trl
- [3] Hotfix CI: Xfail NemotronH GRPO/RLOO tests against a broken transformers dev build (#6542) huggingface/trl
- [4] Hotfix CI: Temporarily pin bitsandbytes < 0.50.0 (#6544) huggingface/trl
- [5] Improved model factory path handling. Fix #2725 ↗ huggingface/pytorch-image-models
- [6] Improved model factory path handling. Fix #2725 ... also fix a few other filename/extension priority bugs. huggingface/pytorch-image-models
- [7] GitHub CI OIDC, snapshot dl cleanup huggingface/pytorch-image-models