$ the-wire · showcase
TRL DROPS ASYNC DISTILLATION, DIFFUSERS GO TENSOR PARALLEL
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
HuggingFace's training and inference stacks moved in lockstep overnight.
TRL's new AsyncDistillationTrainer brings on-policy distillation online, with an architecture that mirrors AsyncGRPOTrainer: a background worker generates student completions and scores them against a teacher served over HTTP, so generation and training overlap instead of alternating [1]. The trainer also supports multi-teacher on-policy distillation, routing each sample to a specific teacher via a `teacher_id` column [1]. Meanwhile, AsyncGRPOTrainer got a simplification pass that drops redundant metric guards and merges train-begin callbacks, tightening the core loop [2]. Diffusers stepped up for production inference: tensor parallelism (TP) is now supported for model inference on CUDA and AWS Neuron, exposed through `model.enable_parallelism(config=TensorParallelConfig(...))`, the same API used for context parallelism [3]. TP shards from a flat `_tp_plan` and is validated on three pipelines on trn2 with TP=8 in both eager and `torch.compile` mode [4]. For those running kernels, the tiled MLP backward now accumulates each weight gradient once per pass instead of once per shard, fixing DDP's `Expected to mark a variable ready only once` failure [5]. Kernels also added an autotuning example, closing issue #733 [6]. Liger-kernel gets a bump to 0.8.2, required in the TRL dependency, and the SAPO warning filter is dropped [7]. Over the period, 11 commits and 11 PRs moved across the four repos.
Action items
- → Review AsyncDistillationTrainer for your distillation pipeline huggingface/trl [plan]
- → Test tensor parallelism on your Neuron or CUDA inference setup huggingface/diffusers [plan]
- → Upgrade liger-kernel to 0.8.2 to match TRL requirement huggingface/trl [immediate]
References
- [1] Add AsyncDistillationTrainer ↗ huggingface/trl
- [2] Simplify AsyncGRPOTrainer: drop redundant metric guards, merge train-begin callbacks (#6378) ↗ huggingface/trl
- [3] [core] Support tensor parallelism for model inference (CUDA, Neuron) ↗ huggingface/diffusers
- [4] [core] Support tensor parallelism for model inference (CUDA, Neuron) (#13718) ↗ huggingface/diffusers
- [5] Accumulate tiled MLP weight grads once per backward ↗ huggingface/kernels-community
- [6] feat: add an example on using autotuning. ↗ huggingface/kernels
- [7] Require liger-kernel 0.8.2 and drop the SAPO warning filter (#6768) ↗ huggingface/trl
