The Wire · Showcase
TRANSFORMERS FIXES MULTIMODAL INFERENCE PIPELINE, DIFFUSERS ADDS SDNQ QUANTIZATION
By RepoJournal · Filed · About Hugging Face
Transformers dropped native multimodal input handling in prepare_inputs_for_generation while diffusers shipped core support for SDNQ quantization, the first major quantization framework addition in months.
The transformers team merged three critical fixes across the inference stack. A patch [1] now drops multimodal inputs natively in prepare_inputs_for_generation if they're not in the prefill stage, fixing a class of edge cases where vision inputs would contaminate token generation. The same effort [2] removed the Rotary deprecation warning and [4] cleaned up Gemma 4 warnings that were firing on every model load. Separately, [5] improved torch distributed availability checking to handle binaries compiled with USE_DISTRIBUTED=0, unblocking users on constrained deployments. Chat parsing [3] got smarter about tool-call arguments, now typing inline arguments directly from the calling tool's JSON schema rather than applying generic value parsers downstream. Over in diffusers, [6] landed SDNQ quantization core loading, letting you load prequantized checkpoints with a normal from_pretrained call or quantize on the fly by passing SDNQConfig, with full save/reload support inside pipelines. TRL [7] added a regression test for KTO with pre-wrapped PEFT models using target_parameters, matching equivalent coverage in GRPO, DPO and RLOO. LeRobot fixed [8] SmolVLM layer freezing that was silently leaving the final VLM layer trainable, [9] corrected frame errors being treated as shard exhaustion in streaming datasets, and [10] removed stereo fallback on RealSense RGB-only camera color parameter setting.
One email a day. Unsubscribe in one click.
Keep up with Hugging Face in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
One email a day. Unsubscribe in one click. Read a past issue →
Action items
- → Upgrade transformers to pick up multimodal inference fix and warning removal huggingface/transformers [plan]
- → Review SDNQ quantization workflow if shipping diffusion models to production huggingface/diffusers [monitor]
- → Verify SmolVLM fine-tuning layer freezing behavior if using train_expert_only=False huggingface/lerobot [plan]
References
- [1] Drop multimodal inputs natively in prepare_inputs_for_generation if not in prefill (#47622) huggingface/transformers
- [2] Remove Rotary warning (#47642) huggingface/transformers
- [3] [Chat Parsing] Type inline tool-call arguments from the calling tool's JSON Schema (#47529) huggingface/transformers
- [4] Remove gemma4 warnings (#47664) huggingface/transformers
- [5] better guarding to handle torch compiled with USE_DISTRIBUTED=0 (#47619) huggingface/transformers
- [6] [quantization] SDNQ core loading ↗ huggingface/diffusers
- [7] Add `test_train_moe_peft_model` for KTO (#6589) huggingface/trl
- [8] fix(smolvla): freeze the intended VLM layers when train_expert_only=False (#4019) huggingface/lerobot
- [9] fix(datasets): stop frame errors being treated as shard exhaustion in StreamingLeRobotDataset (#4237) huggingface/lerobot
- [10] fix(RGB only): remove the stereo module fallback when setting colors parameters on RealSense cameras (#4225) huggingface/lerobot