$ the-wire · showcase
Harbor-hf sinks broken Sandbox cleanup
By RepoJournal · Filed · About Hugging Face · Composed from the cited sources · methodology
Harbor-hf shipped five fixes overnight that close the gaps where stuck Sandboxes, terminal jobs, and invalid campaign results could quietly block recovery.
Harbor-hf's controller was leaving Sandbox file actions pending forever when a Sandbox closed mid-write, stalling cleanup and recovery; that path now matches command execution [1]. A finished task could also leave an older capacity hold behind, so the reconciler now closes every admitted Sandbox and waits for a terminal close receipt [2]. Completed jobs held their full cost reservations forever, blocking recovery even when spending stayed under the limit; reservations now release once a job is terminal [3]. The control app could fail at startup rebuilding a large Bucket ledger, so replay now uses a fixed worker pool and retries only transient network failures [4]. And campaigns could seal attempts with no real token usage; the service now rejects those and keeps invalid receipts as evidence [5]. Meanwhile, LeRobot fixed a silent argparse abbreviation that broke all three PEFT CLI tests after draccus 0.11 disabled abbreviations, plus a ZeroDivisionError in RL learner checkpoint saving when save_freq is non-positive [6]. Docs clarify that "Higher Hub rate limits" are not tied to organization IP ranges [7].
Action items
- → Update harbor-hf to include PRs #103-#107 before relying on campaign recovery huggingface/harbor-hf [immediate]
- → Pull in lerobot PR #4477 to fix PEFT CLI tests huggingface/lerobot [plan]
- → Review lerobot PR #4479 to guard checkpoint saving against non-positive save_freq huggingface/lerobot [plan]
- → Note the corrected rate limit docs for enterprise users huggingface/hub-docs [monitor]
References
- [1] fix(control): settle closed Sandbox file actions ↗ huggingface/harbor-hf
- [2] fix(control): clean terminal task sandboxes ↗ huggingface/harbor-hf
- [3] fix(control): settle terminal job reservations ↗ huggingface/harbor-hf
- [4] fix(control): bound Bucket replay downloads ↗ huggingface/harbor-hf
- [5] fix(control): reject invalid campaign results ↗ huggingface/harbor-hf
- [6] fix(tests): fix peft cli test ↗ huggingface/lerobot
- [7] Clarify "Higher Hub rate limits" (#2721) ↗ huggingface/hub-docs
