The Wire · Showcase
JAX FIXES CRITICAL SHARD_MAP GRADIENT BUG AFTER PARTIAL ROLLBACK
By RepoJournal · Filed · About Google
JAX reverted a commit that introduced incorrect gradient computation in distributed operations, then shipped the proper fix with safeguards.
The team rolled back ac12cbaa1f [1] after discovering it was inserting unnecessary collective operations during backward passes in shard_map. The root cause: wrong pspec calculations when check_vma=False [2]. Engineers have now landed the corrected version [3] that properly uses the cotangent spec to determine which mesh axes need synchronization. New test coverage [4] ensures unreduced forward inputs trigger correct psum insertion on the backward pass. This fixes a class of silent correctness bugs in distributed training where gradients would compute but with unnecessary communication overhead.
Action items
- → Pull latest JAX and run distributed training tests to verify gradient correctness google/jax [plan]
- → Check if your shard_map usage with check_vma=False was affected by the reverted commit google/jax [monitor]
References
- [1] Reverts ac12cbaa1f877ef38a128000b8c561f136fc7f01 ↗ google/jax
- [2] Use the cotangent spec when calculating which mesh axes to do psum over if check_vma=False in shard_map transpose. google/jax
- [3] Use the cotangent spec when calculating which mesh axes to do psum over if check_vma=False in shard_map transpose. ↗ google/jax
- [4] Add a test to make sure that if input is unreduced on fwd (with check_vma=False on shard_map), shard_map inserts a psum on bwd similar to invarying. ↗ google/jax
FAQ
- What changed in Google on May 25, 2026?
- JAX reverted a commit that introduced incorrect gradient computation in distributed operations, then shipped the proper fix with safeguards.
- What should Google teams do about it?
- Pull latest JAX and run distributed training tests to verify gradient correctness • Check if your shard_map usage with check_vma=False was affected by the reverted commit
- Which Google repositories shipped on May 25, 2026?
- google/jax