$ the-wire · showcase
Transformers adds Kimi linear support
By RepoJournal · Filed · About Hugging Face
3 people shipped this
Transformers contributors added a new modular implementation for Kimi linear models, and TRL trimmed hidden-state output overhead.
Hugging Face Transformers is adding support for Kimi linear models in a large new PR [1] that includes config, modeling code, experts, attention, KDA initialization, decoder and pretrained weights, conversion mapping, tokenizer mapping, tests, and docs. The change lands as a modular implementation built from draft modeling code through auto fixes and bug fixes, so users who rely on new Transformers builds will be able to load and run Kimi linear checkpoints once it merges. The same PR carries CI badges for both CPU CI and GPU run-slow test suites [2].
In TRL, a performance cleanup now avoids calling output_hidden_states when only last_hidden_state is needed [3]. The change keeps the output path from materializing hidden states that callers discard, which cuts memory and compute in training and evaluation loops that request only the final layer state. It is co-authored by maintaner Quentin Gallouédec, and should be safe to adopt because the output tensor returned to users is unchanged.
Separately, tau_ai added a provider-neutral runtime model-catalog discovery capability for authenticated ChatGPT Codex subscriptions [4][5]. It parses and caches the Codex /models response alongside runtime limits, publishes the account-specific Codex inventory as a process-local tau_coding overlay, and refreshes Codex models at session startup and in the /model background refresh, including while another provider is active. The checked-in Codex catalog is retained on offline, empty, malformed, unauthorized, or unavailable responses, and the change also preserves model selections across startup, refresh, and resume.
Action items
- → Review the Kimi linear integration in Transformers when it lands, and plan for checkpoint conversion if you serve Kimi linear models huggingface/transformers [plan]
- → Update TRL to the commit avoiding output_hidden_states when only last_hidden_state is needed huggingface/trl [plan]
- → Monitor tau_ai for the merged Codex catalog discovery behavior, especially session refresh and /model background refresh behavior huggingface/tau [monitor]
References
- [1] Kimi linear (#48250) ↗ huggingface/transformers
- [2] Kimi linear ↗ huggingface/transformers
- [3] perf: avoid output_hidden_states when only last_hidden_state is used (#4755) ↗ huggingface/trl
- [4] Discover Codex subscription model catalogs ↗ huggingface/tau
- [5] Discover Codex subscription model catalogs (#681) ↗ huggingface/tau
