$ the-wire · showcase
ReinforceLoss stops resampling actions, torch_tensorrt save fixed
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
15 people shipped this
tp5uiuc
@tp5uiuc
3 cited
wwwjn
@wwwjn
2 cited
theap06
@theap06
1 cited
Animesh Jain
@anijain2305
Ivan Kobzarev
@IvanKobzarev
karthickai
@karthickai
ethanwee1
@ethanwee1
Bob Ren
@bobrenjc93
sxxaysq
@sxxaysq
drisspg
@drisspg
peterdsharpe
@peterdsharpe
jayzuccarelli
@jayzuccarelli
Ben Sprenger
@bsprenger
bowang007
@bowang007
micwill755
@micwill755
Two correctness bugs land on the same day: a policy-gradient loss that scored actions the agent never took, and a serialization path that crashed on any fully-fallen-back graph.
[BugFix] ReinforceLoss: score stored actions instead of resampling pytorch/rl
ReinforceLoss.forward ran the actor's full forward pass, which resampled actions and overwrote the stored ones, so the policy-gradient term scored actions that were not taken; it now builds the distribution with get_dist and scores the stored action, matching PPO. If you train with ReinforceLoss through a ProbabilisticActor, any run before this fix learned from the wrong log-probs.
fix: order the placeholders first before computing serde range constraints pytorch/TensorRT
torch_tensorrt.save(..., retrace=False) raised IndexError: list index out of range for any graph that fell entirely back to PyTorch, on any device. Placeholders are now hoisted to the front of the node list, preserving their relative order, before make_constraints() runs and indexes flat_dynamic_shapes.
[rl] Fix RL CPU unit tests resolving a month-old torch nightly (#5083) pytorch/torchtitan
Four RL CPU test modules (test_alphabet_sort, test_verifiers_example, test_vllm_registry, test_vllm_wrapper) failed at collection because analyze_pipeline_activation_liveness was resolved against a month-old torch nightly, keeping the suite red on main and on every PR that runs it.
[rl] Keep only the NCCL env vars batch invariance needs (#5081) pytorch/torchtitan
set_batch_invariance had copied all ten NCCL env vars from vLLM's override_envs_for_invariance() without justification; most were no-ops or redundant, and the redundant NCCL_NVLS_ENABLE=0 was also disabling NVLS for FSDP all-gather/reduce-scatter in every batch-invariant training run.
fix: keep PyTorch-executed weights on the device after an offload pytorch/TensorRT
deallocate_module() is now gated on the user-facing offload_module_to_cpu setting at the two call sites that ignored it, and PyTorch-executed state, including the root-graph state a full fallback reads, is put back on the target device after every offload. A test pins what the setting guarantees for a partially converted model.
feat: apply_runtime_settings — permanent runtime config for AOT-loaded engines pytorch/TensorRT
For AOT engines loaded via torch_tensorrt.load(), which carry no TorchTensorRTModule wrapper, there was no path to configure runtime settings; runtime.apply_runtime_settings(target, settings) now applies a RuntimeSettings object to every reachable TRT engine and returns the count updated. The long tail includes torch_tensorrt fix: keep PyTorch-executed weights on the device after an offload and...
Action items
