$ the-wire · showcase
NVFP4 and MXFP4 quantization land in TensorRT, GreenLight tightens its merge bar
By RepoJournal · Filed · About PyTorch · Composed from the cited sources · methodology
18 people shipped this
micwill755
@micwill755
2 cited
aswanth-07
@aswanth-07
1 cited
shoumikhin
@shoumikhin
1 cited
Scott Roy
@metascroy
1 cited
winskuo-quic
@winskuo-quic
1 cited
Jean Schmidt
@jeanschmidt
1 cited
Albert Malewski
@albmalamd
Jeff Daily
@jeffdaily
Michael Jerge
@mmjerge
Han, Chao
@Chao1Han
Yeonwoo Sung
@YeonwooSung
neuriv
@neuriv
lanluo-nvidia
@lanluo-nvidia
Mateusz Sluszniak
@msluszniak
Per Held
@perheld
wwwind
@wwwind
github-actions[bot]
@github-actions[bot]
atalman
@atalman
TensorRT gains two TorchAO FP4 dequantization paths on the Dynamo route while the GreenLight merge policy now requires a PR to be provably trivial, and TensorRT's own test suite loses three checks that passed whether or not the code was correct.
Added TorchAO NVFP4 weight-only quantization + Linear and FLUX examples pytorch/TensorRT
Registers torchao_trt.dequantize_nvfp4 and maps it to a two-level TensorRT IDequantizeLayer (packed FP4 E2M1, swizzled FP8 E4M3 block scales at block size 16, plus a per-tensor FP32 scale), keeping the op impure during constant folding so DQ survives instead of collapsing into dense BF16. It is weight-only storage: activations stay BF16, and it is a different graph from the ModelOpt NVFP4 path.
Added TorchAO MXFP4 (MXDynamicActivationMXWeightConfig) + Linear and FLUX examples pytorch/TensorRT
The companion MXFP4 path registers torchao_trt.dequantize_mxfp4 for MXDynamicActivationMXWeightConfig, also marked impure in constant folding. Native MXFP4 by MXFP4 kernels need B200 or B300, so this route runs emulated, and there is no TorchAO MXFP4 weight-only config; that recipe is NVFP4.
[GreenLight] Adds a triviality definition and requirement for GL policy (#8814) pytorch/test-infra
LAND now means safe and trivial, where trivial is a claim about the review rather than the diff. The stated bar is that a competent reviewer reads the change, learns nothing, changes nothing, and types LGTM.
Make three more checks fail when the thing they check is gone pytorch/TensorRT
A parser check accepted any refusal rather than the specific one it was written for, and a guard was satisfied by the term appearing in a comment, so both passed on broken code; they were repaired alongside a third. The checks now compare the actual cause and read the body of the helper the term belongs to, which matters if you have been trusting this suite as a gate.
Qualcomm AI Engine Direct - HF LLM Optimization Stage 1 pytorch/executorch
KV IO is annotated at 8 bits instead of 16, cutting runtime memory consumption for Llama3.2 1B decode on Qualcomm AI Engine Direct. It is stage one of closing the accuracy and performance gap between the HF LLM path and static_llama.py.
[BugFix] Avoid unnecessary SAC and CQL collector spec queries pytorch/rl
SACTrainer and CQLTrainer only query the collector environment for an action spec when automatic target entropy cannot be resolved from the loss or actor, so numeric target_entropy values and actor-provided specs no longer require collector.getattr_env. The existing collector fallback is unchanged when neither source supplies the spec.
Claim getitem when its producer is claimed pytorch/executorch
NativeSupportedOperators now claims operator.getitem when its producer is also claimed, keeping a multi-output op and the getitems that unpack it in one partition. Without it every getitem was a partition boundary, fragmenting fused-projection models with chunked gate_up and qkv into many delegates.
