$ the-wire · showcase
Activation offloading lands in torchtitan, Dynamo closes a C slot gap
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
torchtitan gains a memory-for-compute lever on top of rematerialization and the Kimi K3 stack keeps hardening its graph replay, while PyTorch fixes a Dynamo miscompile in unbound slot wrapper dispatch.
Activation offloading on torch_remat saved-tensor hooks pytorch/torchtitan
torchtitan adds an activation offloading API that sits on top of remat: pack/unpack hooks are entered before remat's checkpoint runs, so the offloader sees exactly what remat stashes for backward. Users pick between block inputs, tensor outputs in remat regions, and tensors saved for backward, with the design in torchtitan/distributed/ACTIVATION_OFFLOAD.md. The summary calls it a perf-memory tr...
[dynamo] Run the C slot for unbound slot wrappers and method descriptors (#203085) pytorch/pytorch
Calling an unbound slot wrapper like object.__eq__(a, b) or int.__add__(sub_instance, 1) used to route through owner.call_method, an MRO lookup on the instance that could pick up a Python override on a subclass; CPython's wrapperdescr_call runs the C slot of the descriptor's own type instead. WrapperDescriptorVariable.call_function now matches CPython, so Dynamo-compiled code and eager code agr...
[dsv4] Build sparse-attention geometry once per variant pytorch/torchtitan
Compressed pool offsets, compressor group indices, and the heavily compressed variant's causal mask are now computed once per variant in build_attention_metadata rather than rebuilt inside every layer on every forward. Layer and indexer interfaces take attention_metadata instead of cu_seqlens, so an overriding InnerAttention can pair its own metadata builder with its consumers.
[attention] 7/n Keep linear-attention graph metadata replayable pytorch/torchtitan
Part 7 of the Kimi K3 / Dist-MoE stack makes KDA/GDN attention metadata CUDA-graph replayable by registering it with pytree and fixing CP routing shapes per load balancer. It applies directly to main and does not depend on the earlier parts. The constraint it works around is real: whole-step graphs copy tensor leaves into captured buffers, but any non-tensor leaf must still equal its captured v...
[c10d] Move legacy ProcessGroupNCCL to c10d/fb/ (internal only) (#205374) pytorch/pytorch
Legacy c10d::ProcessGroupNCCL is superseded by nccl2 but was still built in OSS; it moves under torch/csrc/distributed/c10d/fb/ so ShipIt strips it from OSS while fbcode keeps it via -DUSE_C10D_NCCL_LEGACY. Shared NCCL options and helpers move to NCCLCommon.{hpp,cpp}, and nccl2 no longer includes the legacy header. Elsewhere in the tail: pytorch/rl fixed NaN actions from CEMPlanner at top_k=1 (...