$ the-wire · showcase
PyTorch fixes MPS complex std/var and Sequential clone naming
By RepoJournal · Filed · About PyTorch
10 people shipped this
Nikita Shulga
@malfet
2 cited
Egemen Tuncarslan
@egeboy35
1 cited
bsprenger
@bsprenger
1 cited
wwwjn
@wwwjn
1 cited
alanhuangyoo
@alanhuangyoo
1 cited
cherry77-cloud
@taking-lying-flat
1 cited
Yuanyuan Chen
@cyyever
Aaron Gokaslan
@Skylion007
coder-jayp
@coder-jayp
francesco-bertolotti
@francesco-bertolotti
PyTorch core landed fixes for a wrong MPS complex variance result and a silent state_dict key rewrite on Sequential clone, plus torchtitan extended its CI to B200 for Kimi K3.
PyTorch fixed an MPS bug [1] where `std`/`var` on complex inputs returned a wrong complex tensor instead of the real variance; the output is now allocated as real and MPSGraph computes the conjugated form. `torch::nn::Sequential::clone()` previously rebuilt submodules with positional names, silently turning `encoder.weight` into `0.weight` and breaking save/load round trips; it now preserves registered names and keys [2]. The MPS consistency harness no longer sets `requires_grad` on ops that don't support autograd, avoiding spurious "derivative is not implemented" failures [3]. In pytorch/rl, the Trainer optimizer was silently omitted from checkpoints in several paths (direct optimizer without default stepper, default optimizer hook cases); checkpointing now includes the optimizer's state dict correctly [4]. Torchtitan added a dedicated B200 CI lane running the Kimi K3 multimodal FSDP test on SM100/SM103, and made `ciflow/rl/*` and `ciflow/h100.8/*` tags trigger workflows directly [5]. Torchtitan's FLOPs estimate for flux now counts both attention matmuls, aligning it with the shared quadratic attention formula [6]. The repo also fixed a flaky SwiGLU checkpoint test and corrected end-to-end timing to use the actual step window instead of `log_freq` [7].
Action items
- → Update pytorch/pytorch to include the MPS std/var fix before relying on complex variance on MPS pytorch/pytorch [plan]
- → Verify save/load round trips after applying the Sequential clone fix if you rely on named submodule keys pytorch/pytorch [plan]
- → Check Trainer checkpointing in pytorch/rl if you pass an optimizer directly without a default stepper pytorch/rl [immediate]
- → Pull the torchtitan B200 CI lane if you run Kimi K3 multimodal FSDP on B200 pytorch/torchtitan [monitor]
References
- [1] [MPS] Fix complex std/var to return a real result (#195344) ↗ pytorch/pytorch
- [2] Preserve submodule names in torch::nn::Sequential::clone() (#195107) ↗ pytorch/pytorch
- [3] [BE][CI] Don't require grad on non-autograd ops in test_consistency (#195395) ↗ pytorch/pytorch
- [4] [BugFix] Fix Trainer optimizer checkpointing ↗ pytorch/rl
- [5] [ci] Add B200 Kimi K3 lane and fix RL CIFlow tag triggers ↗ pytorch/torchtitan
- [6] [flux] Count both attention matmuls in the FLOPs estimate (#4366) ↗ pytorch/torchtitan
- [7] [Bug] Use actual step window for end-to-end timing (#4298) ↗ pytorch/torchtitan
