The Wire · Showcase
TRANSFORMERS ADDS MTP DECODING, TRL FIXES DEEPSEEP ZERO-3 CRASH
By RepoJournal · Filed · About Hugging Face
Transformers finally ships proper multi-token prediction support [ref:3], while TRL patches a critical DeepSpeed ZeRO-3 incompatibility that breaks Liger kernel fusion in production training.
MTP decoding lands in Transformers [1], letting models that support multi-token prediction generate sequences faster without API changes. This is the feature teams have been waiting for in speculative decoding pipelines. Meanwhile, TRL's Liger kernel optimization was silently breaking under DeepSpeed ZeRO-3 [2] because the fused DPO loss tries to read `lm_head.weight` directly instead of through the model's forward hook, where ZeRO-3 re-materializes sharded parameters. That fix mirrors a similar GRPO bug from weeks ago. On the visualization front, LeRobot's dataset viewer now renders grayscale and depth camera feeds with the viridis colormap [3], making it actually possible to inspect robot vision data without custom scripts. Three smaller fixes ship across the stack: Qwen Omni's batched text postprocessing now handles full batches [4], `get_json_schema` stops crashing on non-string enum choices [5], and SFT training now drops fully-masked examples after truncation [6] to prevent silent training bugs.
Action items
- → Review MTP support in Transformers if you use speculative decoding or multi-token prediction models huggingface/transformers [plan]
- → Patch DPO/KTO training immediately if running with use_liger_kernel=True and DeepSpeed ZeRO-3 huggingface/trl [immediate]
- → Update TRL to get SFT truncation fix and avoid silently truncated training data huggingface/trl [plan]
- → Pull Qwen Omni fix if you have batched text-only pipelines returning truncated outputs huggingface/transformers [plan]
References
- [1] [generate] Add proper MTP support (#46229) huggingface/transformers
- [2] Fix DPO/KTO `use_liger_kernel` under DeepSpeed ZeRO-3 ↗ huggingface/trl
- [3] feat(video): viridis colormap for grayscale/depth feeds ↗ huggingface/lerobot-dataset-visualizer
- [4] Fix Qwen Omni batched text postprocessing ↗ huggingface/transformers
- [5] Fix get_json_schema crash on non-string docstring choices (#47072) huggingface/transformers
- [6] Drop fully-masked examples after truncation in SFT (#6320) huggingface/trl