$ the-wire · showcase
PyTorch trims CuTeDSL known-good versions, fixes lerp on MPS
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
PyTorch dropped nvidia-cutlass-dsl 4.4.1 through 4.5.2 from the torch/_native known-good list, because the re-vendored QuACK 0.6.4 imports cutlass APIs that only exist in nvidia-cutlass-dsl >= 4.6.
The native CuTeDSL story is the day's consequential one. A forward fix for the QuACK 0.6.4 re-vendoring removed 4.4.1, 4.4.2, and 4.5.2 from the torch/_native known-good versions [1]. Those builds would have imported cutlass.utils.block_copy, cutlass.cute.experimental.iket, and cutlass.pipeline.alloc_reserved_mbarrier, which only exist in nvidia-cutlass-dsl >= 4.6 (upstream QuACK pins ==4.6.2), so the known-good list was advertising combinations the code cannot load. The internal default is 4.4.2, which is now correctly outside the supported set.
On the correctness side, PyTorch made sparse tensor validation state thread-local by moving the deferred sparse-validation collector onto the existing _SerializationLocal state [2]. Concurrent torch.load calls used to be able to validate or clear one another's tensors; now they cannot. The change addresses the issue described in #194750, and it ships with a deterministic integration test that pauses a malformed weights-only load at validation. Separately on MPS, lerp with a mixed-dtype scalar weight now mirrors CUDA's lerp_tensor_kernel: read the CPU scalar weight on the host, drop it from the iterator with remove_operand, and let the scalar-weight path cast it to the compute dtype [3]. Metal LERP still does not support mixed dtypes in general, so an error check was added for other incompatible combinations.
The test suite also shrank where coverage was already duplicated. Five forward-only tests were removed from test/test_mps.py: test_log, test_log_ten, test_log_two, test_logsumexp, and test_gelu_tanh, all already covered by the MPS OpInfo consistency tests, which additionally check backward [4]. A second batch removed test_eq, test_eq_int64, test_logical_not, test_logical_and, test_logical_or, and test_logical_xor, plus the _wrap_tensor helper that only the logical tests used [5].
In the RL and TensorDict stacks, YeonwooSung stopped Minari mission stacks from being shared between observation and ("next", "observation") [6] and fixed RandomTruncationTransform to resolve a nested step_count key rather than only reading root "step_count" and writing root "truncated"/"done" [7]. vmoens added shared buffers for ParallelEnv workers that report their own metadata under metadata_from_workers=True [8]. torchtitan moved quantization and lora into dedicated homes, with lora becoming a ModelConfigTransform [9], relocated logging.py, profiler.py, and metrics.py into torchtitan/observability with each module logging under its own namespace [10], and deprecated minimal_async_ep as unmaintained, noting that the HybridEP backend will be deprecated once DistMoE is available [11].
Action items
- → Pin nvidia-cutlass-dsl to >= 4.6 (or == 4.6.2) if you build torch/_native CuTeDSL paths; 4.4.x and 4.5.2 are no longer known-good pytorch/pytorch [immediate]
- → Drop minimal_async_ep from torchtitan training configs before your next run pytorch/torchtitan [plan]
- → Re-check MPS lerp behavior with a scalar weight after upgrading; mixed-dtype scalar weights now cast to the compute dtype and other mixed dtypes raise pytorch/pytorch [monitor]
References
- [1] [native] Drop CuTeDSL <= 4.5.x from known-good versions (#196834) ↗ pytorch/pytorch
- [2] Make sparse tensor validation state thread-local (#194751) ↗ pytorch/pytorch
- [3] [MPS] Fix lerp with a mixed-dtype scalar weight (#196662) ↗ pytorch/pytorch
- [4] [BE] Remove tests from test_mps for log, log10, log2, logsumexp and gelu (tanh) (#196877) ↗ pytorch/pytorch
- [5] [BE] Remove tests from test_mps for eq, logical_not, logical_and, logical_or and logical_xor (#196878) ↗ pytorch/pytorch
- [6] [BugFix] Stop sharing Minari mission stacks between observation and next ↗ pytorch/rl
- [7] [BugFix] Honor nested step_count in RandomTruncationTransform ↗ pytorch/rl
- [8] [Feature] Shared buffers for ParallelEnv workers that report their own metadata ↗ pytorch/rl
- [9] move quantization and lora to new home ↗ pytorch/torchtitan
- [10] improve logging folder and module organization ↗ pytorch/torchtitan
- [11] deprecate minimal_async_ep ↗ pytorch/torchtitan