$ the-wire · showcase
torchtitan fixes multi-node HF export, torch._native opens to non-aten namespaces
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
A DTensor-unsafe permute broke llama3 HF checkpoint conversion on any FSDP degree above 8, torch._native can now override ops outside the aten namespace, and Helion's ROCm backend widens its reduction path.
fix(llama3): make q/k HF-conversion permute DTensor-safe pytorch/torchtitan
Llama3StateDictAdapter._permute and _reverse_permute ran a head-splitting view() on q/k projection weights that FSDP shards along dim 0, the same dim the view unflattens, so an FSDP degree that does not evenly divide the head count raised `Cannot unflatten unevenly sharded tensor`. For llama3-8B (n_kv_heads=8) that broke HF export above an 8-way FSDP degree, which the PR describes as any typica...
torch/_native: allow overrides on namespaces other than aten (#194509) pytorch/pytorch
register_op_override used to reject any lib_symbol other than aten and the internals hardcoded aten when resolving the op, so kernels for chunked ops registered in the torch_nn namespace, like those behind torch/nn's linear_cross_entropy, could not be installed. The override registry now accepts any namespace.
[ROCM] FlyDSL: Support wavefront cordination along the row dimension( W>1) for reduction kernels. pytorch/helion
On top of the earlier W=1 reduction work, the flydsl ROCm backend now lets W warps cooperate on a single row and adds fp16/bf16 V=8 (128-bit BufferCopy) vectorization. W=1 pinned one 64-lane warp per row, so wide rows serialized on a single warp; both changes cut reduction latency on large-N shapes.
[Feature] Add a rate-limited replay buffer pytorch/rl
RateLimitedReplayBuffer caps cumulative sampled items relative to cumulative inserts, reserves sample budget atomically for both direct and shared-memory callers with timeout, cancellation, and shutdown wakeups, and checkpoints the ratio target and wait counter alongside the existing replay-buffer counters. It rejects prefetching because prefetch can consume budget before the caller reserves it.
[dynamo] Check IMPORT_NAME's result is a module before binding an alias (#197745) pytorch/pytorch
IMPORT_NAME validated __import__'s result only after import_source had already bound it into the traced globals and a process-wide cache, leaving a non-module sys.modules entry behind; a later trace with a real module in that entry crashed with InternalTorchDynamoError instead of compiling. The module-type check now runs before the bind. The remaining items are cleanup and enablement: torch._na...
