$ the-wire · showcase
BLT gets its sliding windows back, and a NaN panic in TEI goes away
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
Transformers repairs a silent regression that inflated BLT's entropy scores and halved its patch rate, while text-embeddings-inference closes a panic that could take down a whole replica under panic = "abort".
Restore BLT's 512-byte sliding windows (entropy patcher + local encoder/decoder) (#49188) huggingface/transformers
BltPatcher scored the entropy model with an unbounded causal mask instead of BLT's specified local block-causal bias over a 512-token sliding window, so predicted entropies came out systematically inflated, more bytes cleared patching_threshold, and the patch rate collapsed on inputs longer than the window. BLT users get the specified window back in the entropy patcher and in the local encoder/...
fix: panic on NaN scores in predict softmax (infer.rs) huggingface/text-embeddings-inference
A NaN classification logit could panic inside Infer::predict before the HTTP or gRPC NaN guards ran, and with panic = "abort" that is one request terminating a replica; the previous softmax also picked the largest absolute logit rather than the largest logit, underflowing on large mixed-sign or negative scores. The fix rejects non-finite logits (NaN and either infinity) before computing the mul...
Keep image inputs on the first chunk of a chunked prefill (#49753) huggingface/transformers
Chunked prefill called prepare_inputs_for_generation without is_first_iteration, so pixel_values and mm_encoder_outputs were dropped from every chunk and vision-language models generated without the image at all. The first chunk now gets the flag the way unchunked prefill does, and a Llava test asserts chunked and unchunked prefill produce the same output.
[compressed-tensors] better path through conversion mapping (#50238) huggingface/transformers
compressed-tensors checkpoints now dequantize through the conversion mapping at load time, with dequantize=True pinned to drop w8a8 activation quantization and the user's flag choosing the load op; uncompressed checkpoints no longer force it, and the no-op after-load hook is gone. If you load quantized checkpoints through this path, expect the resolved load op to follow the checkpoint format ra...
Reduce and simplify CI overhead huggingface/text-embeddings-inference
TEI's CI now runs matching CPU model tests for non-draft PRs, keeps image builds maintainer-triggered, scopes builds to relevant changes, and lets maintainers request tests or builds by PR number alone; external contributors use the existing workflow approval button and PR jobs run without publishing credentials. Elsewhere in transformers-mic, nightly Sentence Transformers scenarios for Embeddi...
Action items
- → Upgrade text-embeddings-inference to a build containing the non-finite logit guard in Infer::predict if you compile w... huggingface/text-embeddings-inference [plan]
