RepoJournal
Hugging Face @huggingface
139 wires and counting

$ follow Hugging Face

Keep up with Hugging Face in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-08-25
stories 62
GitHub RSS Archive Reviews

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Transformers MoE loss cut 99.7% as TRL gains tool calling

By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology

qgallouedec Sayak Paul Yuekai Zhang jiqing-feng sergereview[bot] mowoe danieldk
9 people shipped this
qgallouedec @qgallouedec 4 cited Sayak Paul @sayakpaul 2 cited Yuekai Zhang @yuekaizhang 1 cited jiqing-feng @jiqing-feng sergereview[bot] @sergereview[bot] mowoe @mowoe danieldk @danieldk Wang, Yi @sywangyi albertvillanova @albertvillanova

A single PR eliminates a 128k-token memory bomb in MoE training while TRL's distillation trainer learns to call tools.

Transformers slashed MoE load-balancing loss memory by 99.7% at 128k tokens by computing per-layer instead of materializing a giant one-hot tensor [1]. That fix lands alongside a nemotron_h save_pretrained correction that restores the plural embedding key for all published checkpoints [2]. TRL's DistillationTrainer now supports tool calling, mirroring GRPO with masked tool-result tokens and a configurable iteration cap [3]. The same author reorganized TRL examples into per-task folders, untangling scripts from notebooks [4]. Kernels-community ported SageAttention3 Blackwell FP4 attention, tested on RTX-PRO-6000, bringing microscaling NVFP4 to sm_120 [5]. But watch the kernels telemetry doc fix: a typo in the title hides a real improvement in env var documentation [6]. The SFT collator now emits position_ids for sequence parallelism, fixing a crash on the first batch in ALST/Ulysses setups [7]. That's the one to test before you trust it at scale.

Action items

References

  1. [1] Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k ↗ huggingface/transformers
  2. [2] Fix nemotron_h save_pretrained emitting singular backbone.embedding.weight (#48075) ↗ huggingface/transformers
  3. [3] [DistillationTrainer refactor] Support tool calling ↗ huggingface/trl
  4. [4] Reorganize examples into self-contained per-example folders ↗ huggingface/trl
  5. [5] sage-blackwell: port SageAttention3 Blackwell FP4 attention ↗ huggingface/kernels-community
  6. [6] chore: document telemetry env var beter. ↗ huggingface/kernels
  7. [7] Emit position_ids from the SFT collator when sequence parallelism is enabled ↗ huggingface/trl

$ ls huggingface/reviews/ # the step-back reads

the week in review Diffusers 0.41.0 ships Qwen-Image 2.1, TRL 1.15 goes breaking the month in review TRL ships 1.13 and 1.14, removing trl.losses
Full archive → Weekly reviews →

Quick answers

What shipped in Hugging Face on August 25, 2026?
A single PR eliminates a 128k-token memory bomb in MoE training while TRL's distillation trainer learns to call tools. In total, 31 commits and 31 pull requests landed.
Who contributed to Hugging Face on August 25, 2026?
9 developers shipped this update, including qgallouedec, Yuekai Zhang, jiqing-feng, sergereview[bot], mowoe, danieldk, Sayak Paul, and Wang, Yi, and 1 more.
What were the notable Hugging Face updates?
Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k, Fix nemotron_h save_pretrained emitting singular backbone.embedding.weight (#48075), and [DistillationTrainer refactor] Support tool calling.
♥

$ log @huggingface --recent

MoE expert sharding gets a test, lerobot swaps vendor SDKs for rustypot 59 shipped Transformers 5.20 ships Muse Spark Open, TRL adds AWS Trainium 97 shipped TRL defaults padding to "auto" on Neuron, drops vLLM 0.21.0 164 shipped AsyncGRPO packs rollouts as prefix trees, TRL types tool arguments by schema 80 shipped

$ log --all # elsewhere on the wire

Alignment & Post-training verl rejects conflicting vLLM engine_kwargs, open-instruct adds MILES OPD behind a Megatron learner Interpretability nnsight 0.8.0 cuts import time from 7.6s to 1.6s

$ ls showcases/

Safety Evals AI Red Teaming Agent Security Interpretability Alignment & Post-training

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.