$ the-wire · showcase
kernels gains minver checks, diffusers deprecates ONNX
By RepoJournal · Filed · About Hugging Face
huggingface/kernels now validates kernel metadata against a minimum version, and diffusers starts deprecating its ONNX export path.
huggingface/kernels introduces a `minver` field to kernel metadata so client libraries can verify the installed version meets requirements before use [1]. It also implements a capability check in `main` [2] and cleans up failed imports by removing partially initialized modules from `sys.modules` before re-raising the exception with kernel context [3]. In huggingface/kernels-community, tests now validate that `get_kernel()` calls use the repo id and version from each kernel's `build.toml` [4], and a JSON decoding error in hub upload comments is fixed by aligning the expected upload.json path [5]. For diffusers, the core is deprecating the ONNX export path [6], a change that will eventually remove ONNX support from the library. In transformers, the `capture_outputs` hook now supports `output_hidden_states_layers` so callers can retain only specific hidden-state layers, keeping non-requested positions as `None` [7]. A new BatchRebalanceSampler balances sequence-length cost across DP ranks in distributed training [8]. Sparse TikToken tokenizers no longer fail silently; the fix addresses non-contiguous tensors and potential `None` values [9]. Deprecated mask functions are removed from transformers [10].
Action items
- → Update kernels client to handle the new minver metadata huggingface/kernels [plan]
- → Review kernels-community tests to adopt build.toml validation utilities huggingface/kernels-community [plan]
- → Check transformers capture_outputs callers for output_hidden_states_layers huggingface/transformers [monitor]
- → Plan migration off diffusers ONNX before removal huggingface/diffusers [monitor]
References
- [1] feat: support and check minver ↗ huggingface/kernels
- [2] feat: implement capability check in main ↗ huggingface/kernels
- [3] fix: cleanup failed imports ↗ huggingface/kernels
- [4] feat: implement validation of kernel versions in the tests ↗ huggingface/kernels-community
- [5] fix json decoding error in the hub comment. ↗ huggingface/kernels-community
- [6] [core] deprecate onnx ↗ huggingface/diffusers
- [7] Allow capturing only necessary hidden_states with capture_outputs (#48081) ↗ huggingface/transformers
- [8] Batch Rebalance Data Sampler (#47340) ↗ huggingface/transformers
- [9] [Fix] Sparse TikToken tokenizers silently fail (#48446) ↗ huggingface/transformers
- [10] Remove deprecated mask functions (#48476) ↗ huggingface/transformers