$ the-wire · showcase
PYTORCH-IMAGE-MODELS FIXES POSITIONAL EMBEDDING RESAMPLING BUG
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
1 person shipped this
A shortcut in positional embedding resampling was silently skipping necessary transformations when token counts matched but grid shapes differed.
The `resample_abs_pos_embed` function in pytorch-image-models had a critical logic flaw [1]. When source and target grids held the same number of patches, the function returned early without checking if the grid dimensions themselves had changed. That means a rectangular source grid like (2, 8) being resampled to a square (4, 4) target would skip the resampling entirely, breaking model behavior on architectures that use explicit grid specifications [2]. The fix adds proper dimension checking and includes a regression test to catch this case in the future. This matters because vision transformers increasingly use non-square patch grids for efficiency, and silent failures on grid reshaping can corrupt embeddings silently. Ship this before you use pytorch-image-models with any rectangular source grids.
Action items
- → Pull pytorch-image-models fix if you use explicit old_size parameters in resample_abs_pos_embed huggingface/pytorch-image-models [immediate]
- → Test vision models with non-square patch grids if you've deployed since this bug was introduced huggingface/pytorch-image-models [plan]
References
- [1] Fix positional embedding resampling for equal-token grid changes ↗ huggingface/pytorch-image-models
- [2] Fix equal-token positional embedding resampling ↗ huggingface/pytorch-image-models
