The Wire · Showcase
TRL UNBLOCKS MOE LORA AS PEFT PATCHES AUTOCAST
By RepoJournal · Filed · About Hugging Face
The MoE LoRA CI failures that have been plaguing TRL for days are gone, and the fix came from upstream.
TRL reverted its xfail hotfix for the seven MoE LoRA training tests now that peft#3602 fixed the autocast dtype bug that was crashing CI with "ValueError: Registering a parametrization may not change the dtype of the tensor" [1]. The temporary exclusion comment is also gone, signaling the team trusts the upstream resolution [2]. Meanwhile, huggingface_hub shipped v1.26.1, a patch release on the heels of the new bucket visibility update that adds a `--private`/`--public` CLI switch for changing bucket settings after creation [3][4]. That bucket change is client-side plumbing only, so this is a feature to adopt, not a security emergency. Transformers is quieter but notable: a fix for CpmAnt ensures the model's lm_head is sized to the actual vocab_size, preventing a mismatch with the released checkpoint's 30720-row head, a subtle loading bug that could corrupt inference [5]. The pipeline parallel inference PR is still cooking, and the DeepFloyd IF test refactors across four pipelines are a sign of housekeeping discipline [6]. The headline verdict: patch TRL's CI, and update huggingface_hub for the new CLI capability.
One email a day. Unsubscribe in one click.
Keep up with Hugging Face in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
One email a day. Unsubscribe in one click. Read a past issue →
Action items
- → Update trl to include the revert of MoE LoRA xfails once merged, to restore full test coverage huggingface/trl [immediate]
- → Upgrade huggingface_hub to v1.26.1 to get bucket visibility updates and the CLI command huggingface/huggingface_hub [plan]
- → Watch transformers for the CpmAnt fix to land, especially if you load cpm-ant-10b checkpoints huggingface/transformers [monitor]
References
- [1] Revert xfail for MoE LoRA tests now that peft#3602 fixed the autocast dtype bug ↗ huggingface/trl
- [2] Remove the stale detector exclusion comment (#6926) huggingface/trl
- [3] v1.26.1 ↗ huggingface/huggingface_hub
- [4] [Buckets] Add support for updating bucket visibility ↗ huggingface/huggingface_hub
- [5] Cpmant fix use cache (#48013) huggingface/transformers
- [6] refactor deepfloyd if img2img superresolution pipeline tests to the new mixin structure ↗ huggingface/diffusers