$ the-wire · showcase
Inductor's nextafter lowering, cuDNN RNN strides, and a memory-snapshot refcount fix
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
11 people shipped this
PyTorch's compiler and library paths got a batch of correctness fixes today while torchtitan reworked its CUDA graph and attention paths for memory, and TensorRT closed an operator-check gap.
Fix #187340: [torch.compile][Inductor][CUDA] torch.nextafter is incorrectly compiled as no-op (#196578) pytorch/pytorch
Inductor lowered torch.nextafter as a no-op on CUDA, so compiled results silently diverged from eager. The fix adds a numerically accurate lowering that matches eager semantics for fp16, bf16, fp32, and fp64, including exact non-NaN bit patterns, non-contiguous inputs, and signaling-NaN quieting.
Fix cuDNN/MIOpen RNN backward reading the saved output with assumed strides (#198773) pytorch/pytorch
_cudnn_rnn_backward_input and _cudnn_rnn_backward_weight passed the saved forward output to cuDNN/MIOpen as-is, assuming (T, B, H) contiguity after the batch_first transpose that eager always satisfies but compiled graphs need not. The other inputs were already normalized with .contiguous() or TORCH_CHECKed; this brings the saved output in line.
[CUDA] Fix borrowed reference handling in memory snapshot (#199044) pytorch/pytorch
THCPModule_memorySnapshot wrapped the result of PyTuple_GetItem in an owning THPObjectPtr even though PyTuple_GetItem returns a borrowed reference, dropping a reference it never owned on each element of the (id1, id2[, include_traces]) argument tuple. The fix restores explicit refcount handling with different GET_ITEM functions.
[cudagraph] Capture gradient accumulation in one CUDA graph pytorch/torchtitan
The stack adds parallelism.fsdp_defer_gradient_reduction to reduce gradients after the final micro-batch, letting the whole gradient accumulation window run inside a single CUDA graph capture. It depends on two upstream PyTorch PRs and the author flags a conflict with a separate torchtitan change that still needs resolving.
perf(executorch): opt-in shared per-device activation-scratch pool pytorch/TensorRT
By default every execution context in an ExecuTorch TensorRT delegate allocates and holds its own activation scratch, so device memory grows with layer count and a deep model can exhaust memory on layer count alone. The new opt-in shared pool keeps one buffer per device, grown to the largest figure any call on that device has requested.
