$ the-wire · showcase
Softplus cross entropy in TorchTitan, ONNX export for TensorDict policies
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
10 people shipped this
TorchTitan reworks tensor-parallel cross entropy to stop softmax collapse under overconfident models, and torchrl adds a first-class ONNX exporter for TensorDict policies, at the cost of changed numerics for TP greater than 1.
fix softmax collapse with improved ce perf pytorch/torchtitan
The TP cross entropy now computes with F.softplus to fix softmax collapse when the model is overconfident, and the softmax-onehot product moved into the forward pass, so the custom eager TP autograd path no longer redoes those ops or moves buffers in backward. Numerics change for TP greater than 1 because softmax plus cross entropy is now a different implementation.
[Feature] export_onnx_policy: export TensorDict policies to ONNX with torch.onnx.export pytorch/rl
torchrl.modules.export_onnx_policy(policy, path, example, *, out_keys=None, opset_version=18) calls torch.onnx.export on the policy itself with no tensor-level wrapper, routing through tensordict dispatch so the graph takes and returns the TensorDict directly. Compatibility with MicroDuckEnv's new policy vector is a stacked change, not a prerequisite.
[BugFix] Fix compact checkpoints of partial and emptied replay buffers pytorch/rl
Partial and emptied replay buffers could save checkpoints that NestedStorageCheckpointer and H5StorageCheckpointer failed to load with an IndexError out of bounds on dimension 0. The trigger was TED2Flat counting the done flags of every row in storage, including rows past the cursor that were never written, which TED2Nested misreads as a complete last row.
[Quality] Reject stateful transforms placed after a key-dropping inverse pytorch/rl
Stack, RenameTransform(create_copy=False) and ExcludeTransform(inverse=True) drop entries when inverted, so a stateful transform placed after them, such as RewardSum under the stacked key, silently read its state as missing at every step through TransformedEnv._step. Compose now raises on that ordering via an opt-in Transform._inv_drops_key hook.
[overrides] Run the fused SwiGLU override on any accelerator (#5466) pytorch/torchtitan
The bundled fused_swiglu override now runs on any accelerator including CUDA, since its custom kernels are Triton and carry no CUDA dependency; the DSv3 and Muse Spark recipes enable it by default. The pull request describes only manual test runs and a visual source-diff review.
[BugFix] Make TD3 and TD3+BC train correctly with CUDA graphs pytorch/rl
With compile.cudagraphs=True, both sota-implementations/td3/td3.py and td3_bc/td3_bc.py wrapped update(sampled_tensordict, update_actor) in one CudaGraphModule, and a captured graph cannot branch on a changing Python bool. In td3_bc every replay then executed the branch recorded at capture, updating actor and target networks every gradient step instead of every policy_update_delay steps.
