$ the-wire · showcase
PYTORCH CORE FIXES CUDA MEMORY LEAKS AND CHECKPOINT STORAGE SHARING
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
10 people shipped this
PyTorch shipped critical fixes for fake tensor leaks and checkpoint deserialization on meta devices, while the RL stack tackled sparse masked modes and routing tie-breaking across three repos.
The biggest story overnight: fake tensor tracking was leaking CUDA memory after compile, contaminating subsequent runs until a clear was added [1]. Simultaneously, checkpoint loading on meta and fake devices was silently dropping storage sharing because null data pointers bypassed the deserialization cache, forcing every tensor reference to spawn its own storage [2]. Both hit production workloads. On the core side, FunctionalTensor was burning cycles on redundant symbolic arithmetic to compute storage sizes it already knew [7], and SymNode boolean constants were hitting the symbol replacement lookup on every access instead of short-circuiting [8]. The distributed training stack shed a deprecated API requirement: is_symm_mem_enabled_for_group() no longer depends on the deprecated enable_symm_mem_for_group() call to populate internal state [6]. In RL, sparse masked modes were returning offsets into compacted action lists instead of the original action indices, breaking alignment with sample() [3], while routing strategies were locking sessions to whichever candidate won a load tie on first request rather than cycling [4]. TorchTitan's MoE validation was leaking routing counts into the next training step's bias updates [5], and the Arm backend expanded its pre-push op verification list [9]. Activity across 5 repos: 24 commits, 7 PRs.
Action items
- → Pull PyTorch fake tensor leak fix [ref:7] into any 2.x branch running Dynamo compilation with CUDA pytorch/pytorch [immediate]
- → If loading checkpoints on meta/fake devices, apply storage sharing preservation [ref:8] before next checkpoint load pytorch/pytorch [immediate]
- → Review sparse masked categorical usage in RL policies and pick up the mode fix [ref:1] pytorch/rl [plan]
- → Monitor TorchTitan MoE validation runs for routing count overflow [ref:15] if using expert bias pytorch/torchtitan [monitor]
References
- [1] [BugFix] release tracked fake tensors (#193015) ↗ pytorch/pytorch
- [2] Preserve storage sharing when loading checkpoints on meta/fake devices (#193566) ↗ pytorch/pytorch
- [3] [BugFix] Map sparse masked modes to original actions ↗ pytorch/rl
- [4] [rl] Cycle least-loaded routing between tied candidates (#4109) ↗ pytorch/torchtitan
- [5] Do not count MoE tokens during evaluation (#4148) ↗ pytorch/torchtitan
- [6] Make is_symm_mem_enabled_for_group() not require deprecated enable_symm_mem_for_group() (#193115) ↗ pytorch/pytorch
- [7] [functionalization] Hand _make_wrapper_subclass the storage size it already knows (#193416) ↗ pytorch/pytorch
- [8] [fx] Skip replace() for a SymNode holding a boolean constant (#193159) pytorch/pytorch ↗
- [9] Arm backend: Add to pre-push automatic list of supported ops - VGF ↗ pytorch/executorch
