$ the-wire · showcase
TorchTitan RL workers stop importing vLLM, tensordict fixes its pytree registry leak
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
TorchTitan's reinforcement learning stack got the day's biggest corrections, cutting a 31-minute worker startup tail and quadratic multi-turn rollout memory, while tensordict fixed a CudaGraphModule side effect that broke torch.export and torch.func for the rest of the process.
[rl] Keep Verifiers env-server workers off the trainer imports (#5294) pytorch/torchtitan
Every Verifiers env-server pool worker used to load torch, vLLM and Monarch and use none of them, so one worker took ~13 s to start and 112 starting at once on one host contended with each other. On a Terminal-Bench run with 8 generators and 112 workers on 144-CPU GB300 hosts, the last worker came up 31 minutes after the pool started; now a worker imports only Verifiers, its taskset and its har...
[rl] Store each turn's prompt as a delta on the previous turn (linear-size multi-turn rollouts) (1/2) (#5273) pytorch/torchtitan
Each RolloutTurn now stores only the prompt tokens that differ from the previous turn rather than its full prompt, so rollouts grow linearly with turns instead of quadratically. On a synthetic 120-ply chess group the token fields pickle to 17 MB instead of 236 MB and unpickle on the controller in 0.11 s instead of 1.83 s.
[BugFix] CudaGraphModule no longer removes tensordicts from the pytree registry pytorch/tensordict
Creating a CudaGraphModule no longer strips TensorDict, _SubTensorDict, PersistentTensorDict and LazyStackedTensorDict from torch's pytree registry or warns about it, so torch.export, torch.func.grad and tree_map keep working on tensordicts in the rest of the process. The warning in docs/source/compile.rst and the two sentences pointing to it are gone.
Handle empty linalg.lstsq inputs in shared code (#199696) pytorch/pytorch
linalg.lstsq used to hand degenerate shapes straight to the backend, inheriting LAPACK's mutually inconsistent quick returns: gels zeroed B when min(m, n, nrhs) == 0, so for n == 0 CPU reported zero residuals instead of ||B||^2 while the backward assumed the true residual, and gelsy set rank = 0 when nrhs == 0 even though the rank of A does not depend on B. The degenerate cases are now handled ...
[BugFix] Move a sub-tensordict to a free key in LazyStackedTensorDictStore.rename_key_ pytorch/tensordict
LazyStackedTensorDictStore.rename_key_("b", "bb") now moves the sub-tensordict "b" and every leaf under it, matching TensorDict and TensorDictStore; before it left the store unchanged and raised no error. The fix applies when the destination is free, and the other cases keep their existing behavior.
[CPP FakeTensor] Track Python CIA overrides and register_fake kernels in Fake dispatch tables (#199785) pytorch/pytorch
On the PyTorch side, the C++ FakeTensor fallback gained PythonCIA and CustomOpImpl categories in the Fake dispatch tables so it can tell without taking the GIL whether an op has a Python CompositeImplicitAutograd override or a torch.library register_fake kernel, with register_fake entries removed once the last fake is gone.
[rl] Controller GC: freeze the setup heap, fold generator metrics per rollout and group (#5290) pytorch/torchtitan
The long tail: the RL controller now freezes the ~1M post-setup objects out of the collector and folds each rollout's 16 generator-metric records into one Mean and one Max per key, the C++ fakes now pickle by metadata in Dynamo, Inductor and export picklers, and tensorclass tensor_only None fields are stored as missing keys rather than duplicated placeholders.
