PyTorch 2.15.0 lands with rewritten NCCL backend

By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology

♥

TensorDict 0.15.0 also shipped as a pure Python wheel, 1.77x faster on everyday operations, with removals across the release.

2413 commits 1496 PRs merged 13 releases 31 briefings covered

all pytorch reviews →

← previous month pytorch wire →

PyTorch 2.15.0 Release pytorch/pytorch

by soulitzer

The release notes list backwards incompatible changes and deprecations alongside new features, and the c10d work under it makes nonblocking split and shrink wait for child communicator initialization before reporting success; previously an init failure surfaced only through the parent. If you call ncclCommSplit or ncclCommShrink in nonblocking mode, read the incompatible-changes section before ...

TensorDict v0.15.0 pytorch/tensordict

by peterdsharpe

581 pull requests since 0.14.0, 37 of which also shipped in the 0.14.1 to 0.14.3 patches, and the package now builds one 690 KB pure-Python wheel for every platform and Python version. The changelog puts everyday operations at 1.77 times faster on average; check the removal list for anything your code still imports.

[graph_trainer] Remove graph-based EP chunking (#4952) pytorch/torchtitan

by Sherlock Huang

The graph chunking path ran traced graphs through unbacked SymInts, a concretize pass and trace-input preparers to overlap EP; that path is gone and eager chunking is what remains. Expert-parallel training configs that opted into graph chunking will not trace the same way.

[RL] Preserve per-group KV cache across policy updates (#4761) pytorch/torchtitan

by wwwjn

A policy update changes weights but leaves vLLM's cached KV intact, so a fresh rollout could hit prefix state built by an older policy; per-group caches are now preserved across updates instead of being cleared wholesale. RL runs with multi-turn rollouts keep prefix reuse without serving stale state.

Preserve Python scalar semantics in Vulkan and XNNPACK quantizers pytorch/executorch

by JakeStevens

Preparing a model with the Vulkan or XNNPACK quantizer could change results when a Python scalar was materialized in the output dtype: the notes give torch.tensor([1e-4], dtype=torch.float16) * 100000.0 yielding about 10 in eager and something else after quantization. Scalar code on those backends now keeps Python semantics, which means outputs may shift from what earlier quantized builds produ...

Fix rnllama undici security update and Windows ARM64 generator mismatch (#23280) pytorch/executorch

by Scott Roy

The open-ended undici override allowed 7.x while Expo's transitive dependencies require 6.x, which blocked Dependabot; the fix constrains rnllama to ^6.28.1 and consolidates the locks, alongside a Windows ARM64 generator mismatch. If you build the rnllama example on Windows ARM64, this is the version to take.

[c10d][nccl2] Wait for nonblocking split/shrink child initialization (#199263) pytorch/pytorch

by Tristan Rice

Nonblocking shrink and split returned ncclSuccess while the child communicator was still initializing on NCCL's async thread, so a failure surfaced only later through the parent. Code that treats a successful return as a ready communicator can now rely on it.

[BugFix] Validate store indices and convert indexed write values pytorch/tensordict

by peterdsharpe

TensorDictStore and LazyStackedTensorDictStore now bounds-check integer, list, range, tensor and boolean-mask indices, raising IndexError on out-of-range values and wrong-length masks and normalizing negative indices. Indexed writes convert and broadcast to the target, so stores that silently accepted bad indices will start raising.

$ ls pytorch/month/ # the briefings behind this review

Thu Oct 1 TorchTitan drops graph-based EP chunking, ditching FlexAttention in DSV4 Fri Oct 2 NCCL split/shrink init failures now surface, torchtitan lands an RL KV-cache fix Sat Oct 3 NVFP4 kernel pipeline gets PDL, fused scaling, and tuned autotuning; torchtitan drops stale train command Sun Oct 4 AOTI fallback fixes, Vulkan reductions stop clamping, Dist-MoE gets a TorchTitan adapter Mon Oct 5 torch.compile dtype fixes, Vulkan reduction fallbacks, and a silent RL temperature bug Tue Oct 6 Executorch patches CWE-125 in Arm VGF decoder, PyTorch cleans up snapshots and CI Wed Oct 7 ReinforceLoss stops resampling actions, torch_tensorrt save fixed Thu Oct 8 Executorch closes a security hole and two silent-alias bugs Fri Oct 9 Silently wrong MoE gradients, and E8M0 wrappers decompose again Sat Oct 10 TensorDict store scalars, TorchTitan's one-line GC fix Sun Oct 11 One FSDP module at a time, packed-atomic revert, and TensorDict 0.15 removals Mon Oct 12 TorchTitan RL workers stop importing vLLM, tensordict fixes its pytree registry leak Tue Oct 13 GreenLight leaves beta on merge_rules.yaml, FSDP2 numerics fixed in TorchTitan Wed Oct 14 ExecuTorch drops Python 3.10, TensorRT trims compile startup, tensordict fixes silent h5 loss Thu Oct 15 Inductor hands Triton kernels to the worker pool, RL rollout KV cache persists across turns Fri Oct 16 TensorDict 0.15.0 goes pure Python, 1.77x faster Sat Oct 17 Tensordict compiles masked indexing under fullgraph; ExecuTorch links WebGPU into the Wasm build Sun Oct 18 Precompile artifacts learn to fall back, and int4mm's sm_80 gate never fired Mon Oct 19 TensorDict stops silently dropping nested keys, TorchTitan's tokenizer keeps Qwen3's delimiters Tue Oct 20 TensorDict pointwise fixes and deprecations, torch.sort past INT_MAX, ExecuTorch CUDA batching on by default Wed Oct 21 TensorRT drops duplicate plan copies, torchtitan compiles RoPE Thu Oct 22 Tensordict fixes DataLoader zero-parallelism, ExecuTorch CUDA wheel drops the CUDA torch dependency Fri Oct 23 Inductor goes module-level by default, then gets reverted around Sat Oct 24 Mali unroll fixes int8 linear, MPS drops MPSGraph max_pool2d Sun Oct 25 GRPO zero-advantage fix, tensordict update_ semantics corrected Mon Oct 26 Quantize hoisting fixed in ExecuTorch, torchrl raises its torch floor to 2.11 Tue Oct 27 Silent wrong results in ExecuTorch reruns, RoPE cache tied in Torchtitan Wed Oct 28 TensorDict locks down its CI supply chain Thu Oct 29 PyTorch 2.15.0 lands with a rewritten NCCL backend Fri Oct 30 TensorDict fixes h5 and zarr persistence; FBGEMM drops float-only dispatch Sat Oct 31 tensordict excludes CVE-affected PyArrow, clears its 0.15.3 print regressions

all pytorch reviews →

← previous month pytorch wire →