Who contributed to PyTorch on August 20, 2026?

14 developers shipped this update, including Ivy Zhou, cherry77-cloud, Chien-Chin Huang, felipemello1, Anthony Shoumikhin, xingguo01, Nikita Shulga, and rascani, and 6 more.

What were the notable PyTorch updates?

[nvfp4_training] Support NVFP4 GroupedExperts on Blackwell, [nvfp4_training] Support NVFP4 GroupedExperts on Blackwell (#4598), and Remove the legacy torchtitan.components.checkpoint_utils import path (#4241).

@pytorch

PyTorch and the broader machine-learning ecosystem

github ↗

Pick a date

Topics: Python AI / ML Full archive →

The Wire · Showcase

NVFP4 GroupedExperts lands on Blackwell

By RepoJournal · Filed 06:59 UTC on August 20, 2026 · About PyTorch

14 people shipped this

Ivy Zhou @ivy-zhou 2 cited

iamzainhuda @iamzainhuda 2 cited

rdspring1 @rdspring1 2 cited

Bin Bao @desertfire 1 cited

Steven Chung @schung-amd 1 cited

cherry77-cloud @taking-lying-flat

Chien-Chin Huang @fegin

felipemello1 @felipemello1

Anthony Shoumikhin @shoumikhin

xingguo01 @xingguo01

Nikita Shulga @malfet

rascani @rascani

Warren Deng @warrendeng

albanD @albanD

TorchAO's new differentiable NVFP4 grouped GEMM stack brings high-throughput grouped experts to Blackwell, and it's the headline act.

TorchAO shipped a differentiable NVFP4 grouped GEMM stack for Mixture-of-Experts training on Blackwell, fusing randomized Hadamard transforms with NVFP4 quantization and feeding scaled_grouped_mm [1]. The same PR also fixes the NVFP4 quantize path to match TransformerEngine bitwise, using correctly-rounded division instead of the reciprocal path [2]. Meanwhile, torchtitan cleared out legacy import shims, with checkpoint_utils and checkpoint now fully removed, so "there are no compatibility shims left under torchtitan/components/" [3][4]. TorchAO also deleted two unused prototypes, ParetoQ and quant_logger, cutting 310 package LOC and 787 lines repository-wide [5][6]. In pytorch core, a lowercase 'exhaustive' compared against a Literal['DEFAULT', 'EXHAUSTIVE'] meant the EXHAUSTIVE escape hatch never fired in tuned_mm; that's fixed [7]. And the ROCm team fixed a GRU training stall caused by MIOpen RNN dropout mask reuse [8], which is the kind of silent killer that eats CI cycles for days.

Action items

→ Review the new NVFP4 GroupedExperts API if you train MoE on Blackwell pytorch/ao [plan]
→ Update torchtitan imports to drop legacy checkpoint shims pytorch/torchtitan [immediate]
→ Monitor GRU training on ROCm if you use dropout with num_layers > 1 pytorch/pytorch [monitor]

References

[1] [nvfp4_training] Support NVFP4 GroupedExperts on Blackwell ↗ pytorch/ao
[2] [nvfp4_training] Support NVFP4 GroupedExperts on Blackwell (#4598) pytorch/ao
[3] Remove the legacy torchtitan.components.checkpoint_utils import path (#4241) pytorch/torchtitan
[4] Remove the legacy torchtitan.components.checkpoint import path (#4184) pytorch/torchtitan
[5] Delete unused ParetoQ prototype ↗ pytorch/ao
[6] Delete unused quantization logger prototype ↗ pytorch/ao
[7] [inductor] Fix EXHAUSTIVE search space check in tuned_mm (#192285) pytorch/pytorch
[8] [ROCm] Fix MIOpen RNN dropout mask reuse and gradient stall (#189285) pytorch/pytorch

Quick answers

What shipped in PyTorch on August 20, 2026?: TorchAO's new differentiable NVFP4 grouped GEMM stack brings high-throughput grouped experts to Blackwell, and it's the headline act. In total, 105 commits and 55 pull requests landed.
Who contributed to PyTorch on August 20, 2026?: 14 developers shipped this update, including Ivy Zhou, cherry77-cloud, Chien-Chin Huang, felipemello1, Anthony Shoumikhin, xingguo01, Nikita Shulga, and rascani, and 6 more.
What were the notable PyTorch updates?: [nvfp4_training] Support NVFP4 GroupedExperts on Blackwell, [nvfp4_training] Support NVFP4 GroupedExperts on Blackwell (#4598), and Remove the legacy torchtitan.components.checkpoint_utils import path (#4241).

Pallas HBM Windows Open on Any Tensor Axis

Pallas stagers just gained the ability to align HBM windows on any tensor axis, not just the last one, and a critical autotuning crash is fixed.

python 81 shipped

@pytorch Aug 19

Pytorch Fixes XPU Dispatch Gap and CI OOMs

PyTorch's XPU backend lost a critical dispatch for _safe_softmax, and CI is getting a GOMAXPROCS pin to stop GPU nodes from OOMKilling.

+10

python 85 shipped

@pytorch Aug 18

PYTORCH SLASHES B200 BENCH TIME 5X

The B200 microbenchmark suite just went from a 7.5-hour serial slog to a 143-minute parallel run, and that's only the start of what's moving across PyTorch today.

python 92 shipped

@pytorch Aug 17

PYTORCH CUTS MPS DEAD CODE, DREAMER V3 FIXES GPU MEMORY LEAK

PyTorch is aggressively pruning unused Metal Performance Shaders infrastructure while TorchRL patches a critical memory leak that kept loss tensors pinned to CUDA for entire training runs.

python 25 shipped

Elsewhere on the wire

AI Agents Aug 22

Cline hubs get drain and replay as Langchain plumb gates

Cline just shipped rollover-grade hub commands while Langchain exposes gateway internals, and OpenHands quietly broke its own monitoring contract.

ai-agents 42 shipped

Local LLMs Aug 22

SGLang ships v0.5.18 with 710 PRs and DeepSeek V4 FP4 auto-routing

The biggest release of the week just landed: SGLang v0.5.18 adds new models, fixes a silent disagg bug, and defaults DeepSeek V4 FP4 checkpoints to the right MoE runner.