$ the-wire · showcase
TRANSFORMERS PATCHES TORCH.COMPILE CRASH; KERNELS SHIP LAYER REGISTRY UPDATES
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
13 people shipped this
David-Wu1119
@David-Wu1119
1 cited
Harry Mellor
@hmellor
1 cited
sywangyi
@sywangyi
1 cited
Abdennacer-Badaoui
@Abdennacer-Badaoui
1 cited
vasqu
@vasqu
1 cited
stevhliu
@stevhliu
1 cited
yzhautouskay
@yzhautouskay
1 cited
Arek Borucki
@arekborucki
1 cited
remi-or
@remi-or
kaixuanliu
@kaixuanliu
dacorvo
@dacorvo
menglcai
@menglcai
YiYi Xu
@yiyixuxu
Transformers just fixed a half-precision torch.compile crash that broke DETR models in v5.9.0, while the kernel ecosystem shipped three layer-repo updates that unify the custom kernel interface across rotary, liger, and rope implementations.
The torch.compile regression [1] hit DETR position embeddings when a recent rewrite of sine embeddings triggered Inductor's pointless_cumsum_replacement pattern, causing dtype mismatches that crashed encoder attention under bf16. This is a direct regression from v5.8.0 and needs immediate patches for anyone compiling DETR variants. Meanwhile, kernels-community shipped three coordinated PRs [2] [3] that add nn.Module layer versions for apply_rotary_transformers, remaining liger functions, and rotary embeddings respectively, mirroring the layer-repo pattern in transformers#46883. These standardize how custom kernels expose both functional and module interfaces, making them compatible with transformers' layer-registry loading path. On the vLLM front, transformers fixed GPTBigCode and GPTNeoX for the new modelling backend [4], and the kernels team landed a smart XPU autotune improvement [5] that prunes high-register-pressure configs before compilation to skip tens of thousands of spilled instructions and wasted compile cycles. Diffusers added torch.dtype aliasing for Hub snippet compatibility [6] and shipped Cosmos3 transfer controls [7] for structure-conditioned video generation. Dataset-viewer merged IRSA support [8] for prod datasets-server, moving away from static S3 keys.
Action items
- → Apply transformers torch.compile fix [ref:2] if you're compiling DETR models in bf16 huggingface/transformers [immediate]
- → Review and integrate the three kernel layer-repo updates [ref:10] [ref:11] if you maintain custom kernel wrappers huggingface/kernels-community [plan]
- → Monitor vLLM GPTBigCode/GPTNeoX serving [ref:3] after the modelling backend fix lands huggingface/transformers [monitor]
References
- [1] Fix half-precision torch.compile crash in DETR-family sine position embeddings ↗ huggingface/transformers
- [2] aiter-rope: Add layer repo for apply_rotary_transformers ↗ huggingface/kernels-community
- [3] liger-kernels: Add layer repos for remaining functions ↗ huggingface/kernels-community
- [4] :rotating_light: Fix GPTBigCode and GPTNeoX for the Transformers modelling backend for vLLM (#47198) ↗ huggingface/transformers
- [5] finegrained-fp8: add GRF register-pressure pruner for XPU autotune ↗ huggingface/kernels-community
- [6] torch.dtype alias ↗ huggingface/diffusers
- [7] Transfer support for Cosmos3 ModularPipeline ↗ huggingface/diffusers
- [8] Add IRSA to datasets-server (#3359) ↗ huggingface/dataset-viewer
