$ the-wire · showcase
Fake-tensor hardening, ROCm graph fixes, and an XNNPACK use-after-free
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
12 people shipped this
Johnsonms
@Johnsonms
2 cited
Bob Ren
@bobrenjc93
1 cited
Jeff Daily
@jeffdaily
1 cited
vmoens
@vmoens
1 cited
Mateusz Sluszniak
@msluszniak
1 cited
Aditya Venkataraman
@aditvenk
drisspg
@drisspg
Jeremy Schoemaker
@shoemoney
Reza Sajadiany
@rezaasjd
Mergen Nachin
@mergennachin
mcremon-meta
@mcremon-meta
roman-janik-nxp
@roman-janik-nxp
The day's most consequential work is in the seams between real and fake execution: PyTorch closes a fake-tracing correctness hole, TorchTitan stops CUDA-graph and FA3 selection from misbehaving on ROCm, and ExecuTorch fixes a weights-cache free that left PReLU reading freed memory for the runtime's lifetime.
[precompile] Harden the unbacked capture mode and the input scan ahead of fake tracing (#197559) pytorch/pytorch
The unbacked capture path builds its own FakeTensorMode with allow_fallback_kernels=True, so an operator with no meta or fake kernel in an allowlisted namespace (aten, prims, quantized) had its real kernel run on zero-filled substitutes, baking whatever shape that produced into the compiled artifact. bobrenjc93 hardens that mode and the input scan ahead of fake tracing; if you enable unbacked c...
Fix use-after-free of unpacked constants in the XNNPACK weights cache (#22779) pytorch/executorch
XNNWeightsCache::finalize_for_runtime() frees every unpacked buffer on the assumption that create_runtime has packed them all, but PReLU reads its slope straight out of that memory for the life of the runtime. The weights cache now keeps the buffers that operators consume unpacked, closing a use-after-free that could corrupt PReLU output on any ExecuTorch XNNPACK build.
Fix FA3 being selected on ROCm devices (#4778) pytorch/torchtitan
get_cuda_flash_attention_impl() gates on torch.cuda.get_device_capability() alone, and gfx950 on MI350X reports (9, 5), so the (9, 0) check passed and selected FA3 on hardware with no FA3 to select; activate_flash_attention_impl("FA3") then failed because torch's flash_attn_interface is CUDA-only. It now returns None on a ROCm build.
Skip CUDA-graph config validation where graphs cannot capture pytorch/torchtitan
wrap_with_cuda_graph() already logged a warning and fell back to the eager callable on ROCm, but the validators enforcing CUDA-graph restrictions never checked the device, so they raised ValueError for configs that run fine there, and training.disable_cuda_graphs defaults to False. The new cuda_graphs_supported() export is shared by both the wrapper and every validator.
[ROCm] Attribute sm-carveout kernels per call and unskip the test (#197206) pytorch/pytorch
test_honor_sm_carveout comes off the ROCm skip list: its ROCm branch demanded exactly four kernel events in the trace, so any dropped or extra record raised, and kineto on ROCm ahead of the rocprofiler-sdk migration was the likely source. sm-carveout kernels are now attributed per call.
[Performance] Speed up eager from_module and to_module pytorch/tensordict
Elsewhere in the stack: eager from_module(as_module=True) stops re-traversing the new TensorDict to convert buffers, to_module swaps existing Parameters and tensor buffers directly, update and set get a bulk path for unlocked exact tensors, tensorclass construction caches its field order, and 135 parametrized benchmark cases now guard the whole effort; ExecuTorch also erases the _cat_nop, _slic...
