$ the-wire · showcase
Precompile artifacts learn to fall back, and int4mm's sm_80 gate never fired
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
Five precompile fixes make installed compile artifacts usable in cases they previously refused outright, while a two-year-old CUDA guard in int4mm turns out to have been dead code in every host build.
Make the int4mm sm_80 requirement an actual runtime check (#195697) pytorch/pytorch
Both _weight_int4pack_mm_cuda and _convert_weight_to_int4pack_cuda gate on `!defined(__CUDA_ARCH__) || (__CUDA_ARCH__ >= 800)`, but __CUDA_ARCH__ is only defined during nvcc's device pass, so the host-side TORCH_CHECK never ran and the sm_80 requirement was unenforced. It is now a real runtime check, which means calls on hardware below sm_80 will raise where they previously proceeded silently.
[precompile] Let an installed artifact opt into serve-time compiles (#200835) pytorch/pytorch
`load()` on a torch.compiler.precompile artifact gains a keyword-only `allow_serve_time` option. Previously an installed artifact refused any call no captured variant covered; now a job that would rather pay a compile than fail on a missed input can get torch.compile's behavior without dropping the artifact and recompiling everything.
[precompile] Seed `___unnamed_scope_*` globals when installing a package (#200823) pytorch/pytorch
Artifacts whose frames inline module-less globals (a collections.namedtuple constructor, an exec-generated helper) could not be served in a fresh process, because their guards read a `___unnamed_scope_<id>_c<n>` global only the capturing process had bound. Dynamo now records the source of that namespace alongside each minted key so the package seeds it on install. Bob Ren's companion PRs also t...
[BugFix] Reject a log-probability key that would overwrite a sample pytorch/tensordict
A ProbabilisticTensorDictModule with return_log_prob=True and a log_prob_key equal to, or nested under, a sample key used to write the log-probability over the sample, so the action a policy returned was its own log-probability with no warning. The constructor now raises a ValueError naming the clashing keys. TensorDictSequential.__setitem__ and __delitem__ also work on named ModuleDict entries...
[CP] Work around Ulysses graph registration on older NCCL pytorch/torchtitan
NCCL can reuse an incomplete CUDA-graph IPC mapping for another buffer view, producing incorrect results or invalid memory access, and the registration code is shared with all-gather and some out-of-place all-reduce paths. Torchtitan now sets NCCL_GRAPH_REGISTER=0 before process-group initialization for normal Ulysses on NCCL releases before 2.32.3.
Add typed HFP8 conversions and compile-time embedding decoders (#6505) pytorch/FBGEMM
Elsewhere on the desks: FBGEMM added byte-sized hfp8<ExponentBits, ExponentBias> storage with legacy HFP8 saturation and nearest-even rounding (distinct from E4M3FN/E5M2) and made -Wattributes fatal in OSS CMake; Torchtitan gave the RL loop an ActorSystem abstraction with a Monarch implementation and split its 1860-line VLLMGenerator into service, engine loop, dispatcher and a new GenerationEng...
Action items
