RepoJournal
Hugging Face

@huggingface

Transformers, Datasets, and the open AI-model layer

Pick a date

Topics: Python AI / ML Full archive →

The Wire · Showcase

TRANSFORMERS ADDS MTP DECODING, TRL FIXES DEEPSEEP ZERO-3 CRASH

By RepoJournal · Filed · About Hugging Face

Transformers finally ships proper multi-token prediction support [ref:3], while TRL patches a critical DeepSpeed ZeRO-3 incompatibility that breaks Liger kernel fusion in production training.

MTP decoding lands in Transformers [1], letting models that support multi-token prediction generate sequences faster without API changes. This is the feature teams have been waiting for in speculative decoding pipelines. Meanwhile, TRL's Liger kernel optimization was silently breaking under DeepSpeed ZeRO-3 [2] because the fused DPO loss tries to read `lm_head.weight` directly instead of through the model's forward hook, where ZeRO-3 re-materializes sharded parameters. That fix mirrors a similar GRPO bug from weeks ago. On the visualization front, LeRobot's dataset viewer now renders grayscale and depth camera feeds with the viridis colormap [3], making it actually possible to inspect robot vision data without custom scripts. Three smaller fixes ship across the stack: Qwen Omni's batched text postprocessing now handles full batches [4], `get_json_schema` stops crashing on non-string enum choices [5], and SFT training now drops fully-masked examples after truncation [6] to prevent silent training bugs.

Action items

References

  1. [1] [generate] Add proper MTP support (#46229) huggingface/transformers
  2. [2] Fix DPO/KTO `use_liger_kernel` under DeepSpeed ZeRO-3 ↗ huggingface/trl
  3. [3] feat(video): viridis colormap for grayscale/depth feeds ↗ huggingface/lerobot-dataset-visualizer
  4. [4] Fix Qwen Omni batched text postprocessing ↗ huggingface/transformers
  5. [5] Fix get_json_schema crash on non-string docstring choices (#47072) huggingface/transformers
  6. [6] Drop fully-masked examples after truncation in SFT (#6320) huggingface/trl

Quick answers

What shipped in Hugging Face on July 14, 2026?
Transformers finally ships proper multi-token prediction support [ref:3], while TRL patches a critical DeepSpeed ZeRO-3 incompatibility that breaks Liger kernel fusion in production training. In total, 23 commits and 23 pull requests landed.
Who contributed to Hugging Face on July 14, 2026?
8 developers shipped this update, including CarolinePascal, Cyril Vallez, Sunt-ing, Sreekant Baheti, drbh, Albert Villanova del Moral, Sergio Paniego Blanco, and Quentin Gallouédec.
What were the notable Hugging Face updates?
[generate] Add proper MTP support (#46229), Fix DPO/KTO `use_liger_kernel` under DeepSpeed ZeRO-3, and feat(video): viridis colormap for grayscale/depth feeds.

More from @huggingface

Daily updates, in your inbox

Follow Hugging Face

Transformers, Datasets, and the open AI-model layer We'll email you a link to confirm first.

Free. Confirm via email. Unsubscribe in one click.

— or follow the whole beat:

Elsewhere on the wire