$ the-wire · showcase
Ernie4.5-VL-MoE experts shard under TP and EP
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
Transformers fixed two silent correctness bugs in expert-parallel training, and OpenEnv's day belonged to dependabot.
Ernie4.5-VL-MoE: shard the text and vision experts under TP and EP huggingface/transformers
The sharding plans named `layers.*.mlp.experts`, which never matched the experts under `mlp.text_moe` / `mlp.vision_moe`, so every expert stayed replicated; the plans now name both paths.
Fix zero expert gradients in bfloat16 with expert parallelism and ep_size < fsdp_size huggingface/transformers
With token dispatch and `tp_size < ep_size < fsdp_size * tp_size`, the experts' gradient divide factor is not their FSDP group size, so FSDP2 reduced them with NCCL `PREMUL_SUM` and produced zero expert gradients in bfloat16. The fix landed as a commit and its follow-up rewrites the reduction comments and compares loss and every gradient in the bf16 expert-parallel test.
Declare the input modalities of the VLMs that report text only huggingface/transformers
Eight VLMs report `input_modalities == "text"` even though their `forward` takes `pixel_values`: Exaone 4.5, GLM-5 Next, Muse Glimmer, Muse Spark, Qwen3.5, Qwen3.5-MoE, Qwen4-Exp and Step3p7. Their `*Model` and `*ForConditionalGeneration` classes now declare the missing modality.
Document how to avoid the Metaspace prefix after added tokens (#45822) huggingface/transformers
A docs commit explains how to avoid the Metaspace prefix that shows up after added tokens, which matters if you build tokenizers with added tokens and then compare encoded output.
Hotfix CI: Xfail GOLD SmolVLM cross-arch test against transformers 5.20.1 (#7791) huggingface/trl
The long tail: TRL xfailed the GOLD SmolVLM cross-arch test against transformers 5.20.1 as a CI hotfix, and OpenEnv took five dependabot bumps, including pyjwt 2.15.0, actions/upload-artifact 7.0.2 and setup-uv 10.3.0.
