$ the-wire · showcase
TRL documents 1M-token training, diffusers fixes group offloading, kernels fix Intel heuristics
By RepoJournal · Filed · About Hugging Face
14 people shipped this
qgallouedec
@qgallouedec
2 cited
yiyixuxu
@yiyixuxu
1 cited
dg845
@dg845
1 cited
jiqing-feng
@jiqing-feng
1 cited
m96-chan
@m96-chan
1 cited
Altana Natyrova
@anatyrova
rkazants
@rkazants
kashif
@kashif
Albert Villanova del Moral
@albertvillanova
Zhiqi Zhang
@shoutoutuoadi325
sayakpaul
@sayakpaul
danieldk
@danieldk
nico-martin
@nico-martin
Jeremy Schoemaker
@shoemoney
TRL ships a runnable million-token SFT example, diffusers repairs a group-offloading device mismatch, and triton-kernels corrects Intel autotuning constants that cost roughly 10x on Battlemage.
TRL's long-context guide now centers on a runnable example that trains a book-length sequence per step on one 8xH100 node, with [1] documenting the configurations and levers in the order you hit them. The example lives in examples/sft_qwen3_8b_1m_context and uses the default loss_type="chunked_nll", so anyone training past the usual few thousand tokens has a verified starting point. The guide's README and docs index swap the What's New blurb from DistillationTrainer to the long context guide [2].
diffusers changes ModularPipeline.save_pretrained so overwrite_modular_index defaults to True. Previously, saving loaded components left modular_model_index.json pointing at the original repo_id; now the saved directory is self-contained after a save_pretrained call [3]. Also in diffusers, PriorTransformer held clip_mean and clip_std as parameters on the model itself, so group offloading moved them back to the offload device after forward, and post_process_latents then hit a device mismatch. This affected Stable unCLIP pipelines; the fix [4] keeps those parameters onloaded when they are needed.
In kernels-community, the Intel opt-flag heuristics in triton-kernels copied two NVIDIA constants that are wrong for Xe hardware: block_k was hardcoded to min(128, ...) instead of cacheline-derived, and num_warps assumed a 32-lane warp. Together they cost roughly 10x on Battlemage (Arc Pro B60); the fix derives both values from actual hardware properties [5]. The same PR makes matmul_ogs_torch usable off CUDA.
transformers.js fixes a KV cache leak in ChatterboxModel.generate, where a full cache of GPU buffers was left undisposed on every call [6]. The fix [6] ensures the cache is disposed after generation when it is not returned to the caller, so GPU memory no longer grows without bound in repeated synthesis.
Action items
- → Review TRL's new sft_qwen3_8b_1m_context example if you train on long sequences huggingface/trl [plan]
- → Test ModularPipeline.save_pretrained behavior change: overwrite_modular_index now defaults to True huggingface/diffusers [plan]
- → Update triton-kernels to get the corrected Intel autotuning constants before running on Xe hardware huggingface/kernels-community [plan]
- → Upgrade transformers.js if you use Chatterbox generation in production to stop the GPU memory leak huggingface/transformers.js [immediate]
References
- [1] Document 1M-token training and add a context-parallelism example ↗ huggingface/trl
- [2] Rewrite the long context guide ↗ huggingface/trl
- [3] ModularPipeline.save_pretrained: change the default `overwrite_modular_index` to be True ↗ huggingface/diffusers
- [4] Fix PriorTransformer Group Offloading Bug ↗ huggingface/diffusers
- [5] triton-kernels: fix Intel opt-flag heuristics and make matmul_ogs_torch usable off CUDA ↗ huggingface/kernels-community
- [6] fix: dispose KV cache after Chatterbox generation (#1734) ↗ huggingface/transformers.js
