$ the-wire · showcase
TensorDict pointwise fixes and deprecations, torch.sort past INT_MAX, ExecuTorch CUDA batching on by default
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
13 people shipped this
peterdsharpe
@peterdsharpe
3 cited
Gasoonjia
@Gasoonjia
1 cited
Jeff Daily
@jeffdaily
1 cited
Tristan Rice
@d4l3k
1 cited
Ziming Wang
@ZenAlexa
1 cited
wwwjn
@wwwjn
1 cited
0xDELUXA
@0xDELUXA
Lazar Stojiljkovic
@lstojilj-amd
PyTorch MergeBot
@pytorchmergebot
tianyu-l
@tianyu-l
Jessica Zhong
@acisseJZhong
bsprenger
@bsprenger
Huy Do
@huydhn
TensorDict lands two silent-wrong-answer fixes alongside three FutureWarnings aimed at 0.17, while PyTorch lifts sort/topk's INT_MAX dimension limit and a c10d coalescing bug is closed.
[BugFix] Pointwise ops with a batch-shaped tensor give the right values for a nested lazy stack pytorch/tensordict
A batch-shaped tensor operand (a per-sample weight, say) used to line up with the feature dims of any lazy stack nested below the root: it raised when sizes differed and gave wrong values silently when they matched. add, mul, maximum and the other pointwise ops now align that operand with the batch dims of the stacked leaves.
[BugFix] In-place pointwise ops on a sub-tensordict write into a nested lazy stack pytorch/tensordict
In-place pointwise ops on a `_SubTensorDict` over a lazy stack, including add_, mul_, clamp_min_, lerp_ and +=, used to modify a copy and drop it without error, even as zero_, copy_ and set_ wrote through to the source. Those ops now write into the stack members, making sub-tensordicts over dense in-memory sources behave consistently. Both TensorDict bugfixes are worth pulling if TorchRL writes...
[Deprecation] Warn when ^, | or an ordering comparison gets an operand that is not a tensordict, a tensor or a number pytorch/tensordict
`td ^ "x"` returns True, `td | None` returns False and `td < "x"` returns False, so a typo in a mask expression yields a bool instead of an error; the PR keeps those results and emits a FutureWarning, with TensorDict 0.17 returning NotImplemented so Python raises TypeError. Two companion deprecations warn that map and map_iter will seed workers from the global torch RNG in 0.17, and that from_a...
Fix sort and topk for dimensions longer than INT_MAX (#197470) pytorch/pytorch
torch.sort and torch.topk rejected dimensions longer than INT_MAX on CUDA and ROCm; on ROCm that also broke fp16/bf16 topk above 2^32. The PR adds tests and benchmark results and is marked non-BC-breaking, so long-dimension sorts no longer need to be chunked by hand.
[c10d][nccl2] Make endCoalescing cover all ops in the window (#199585) pytorch/pytorch
ProcessGroupNCCL2's startCoalescing/endCoalescing batched only send/recv, so other collectives issued in the window, such as the alltoall_base calls pt2_sparse makes between _start_coalescing and _end_coalescing, ran immediately and their work was dropped. endCoalescing then returned a CompletedWork whose wait() was a no-op while callers could read incomplete outputs; the window now covers all ...
[rl] Support context parallelism in the RL trainer (#5424) pytorch/torchtitan
Under context parallelism, preprocess_inputs sharded labels along the packed token dimension while advantages, generator logprobs, loss mask and temperature reached the loss unsharded, so CP > 1 could not train. A new shard_per_token_tensors helper in torchtitan/distributed/context_parallel.py keeps each CP rank's contiguous token shard, and the RL trainer now supports CP.
perf: keep chunked loss gradient buffers in their native dtype (#5408) pytorch/torchtitan
The chunked-loss hidden-gradient buffers stay in the hidden-state dtype and go straight to decoder backward instead of being copied by GradAccumulator.add(); on an H100 the change reports 128 MiB less peak allocated memory for the 16K-token cases and 1,043.983 MiB for the 128K-token case.
[executorch][cuda] Test CudaExecutor on every step width up to 65 pytorch/executorch
In the long tail: ExecuTorch exercises CudaExecutor at every step width up to 65, serves Muse Glimmer batching on CUDA from the launcher, hashes prepared image identity without CommonCrypto, PyTorch fixes hipify path handling on Windows for JIT-built HIP extensions and adds W7900 device info to Inductor, the dynamo repr cycle detection change is reverted, ReplayBuffer contents can be exported t...
