$ the-wire · showcase
Diffusers pins revisions once, TRL fixes vLLM weight-sync hangs
By RepoJournal · Filed · About Hugging Face
Diffusers now resolves the git revision a single time, and TRL turns silent vLLM weight-sync hangs into explicit errors.
Diffusers' from_pretrained now calls huggingface_hub.resolve_revision once at the start, so every fetched file is pinned to the same commit and served from cache without re-resolving on each download [1][2]. This mirrors a change already running in production in vLLM and is a follow-up to huggingface_hub v1.26.0, which introduced resolve_revision and ResolvedRevision. TRL fixed a GRPO weight-sync path where vLLM responses were discarded and the trainer blocked in an uninterruptible ncclBroadcast, hanging the run; the fix checks each request's status and raises with the server's error text, and moves the NCCL collective to a daemon thread [3][4]. Separately, the CI rerun filter now retries cuBLAS allocation errors, which pytest-rerunfailures previously missed because it only matches the outermost exception [5][6]. Harbor pinned all active deployment profiles to updated FX trial-worker images for the system-message ordering, provider-tool filter, and gateway repairs, regenerating every content-derived profile ID without changing any model or provider setting [7][8][9][10]. Transformers now raises when a paged attention forward is called with no cache, rather than silently computing non-causal attention [11]. Hub-docs documents a Network Security exemption for Enterprise Plus service account tokens, allowing CI automation outside corporate IP ranges to bypass IP restrictions and the Content Access Policy [12][13].
Action items
- → Upgrade to huggingface_hub 1.26.0 and pull the diffusers revision-pinning change before next deploy huggingface/diffusers [plan]
- → Update TRL to include the vLLM weight-sync fix before running GRPO with vLLM huggingface/trl [immediate]
- → Review Harbor deployment profiles after the FX worker pinning for any drifted IDs huggingface/harbor-hf [monitor]
References
- [1] [Download] Resolve the revision once at the beginning of from_pretrained ↗ huggingface/diffusers
- [2] [Download] Resolve the revision once at the beginning of from_pretrained (#14340) ↗ huggingface/diffusers
- [3] Fix: vLLM weight sync fails instead of hanging ↗ huggingface/trl
- [4] Fix: vLLM weight sync fails instead of hanging (#6913) ↗ huggingface/trl
- [5] Retry tests failing on cuBLAS allocation errors the rerun filter missed ↗ huggingface/trl
- [6] Retry tests failing on cuBLAS allocation errors the rerun filter missed (#6916) ↗ huggingface/trl
- [7] fix(profiles): pin merged-system FX worker ↗ huggingface/harbor-hf
- [8] fix(profiles): pin FX system-order worker ↗ huggingface/harbor-hf
- [9] fix(profiles): pin FX provider-filter worker ↗ huggingface/harbor-hf
- [10] fix(profiles): pin FX gateway worker image ↗ huggingface/harbor-hf
- [11] Raise when a paged attention forward is called with no cache (#48297) ↗ huggingface/transformers
- [12] Document Network Security exemption for service account tokens (#2737) ↗ huggingface/hub-docs
- [13] Document Network Security exemption for service account tokens ↗ huggingface/hub-docs