$ the-wire · showcase
FP8 rollout KV cache fix in verl, refresh batches overlap training in open-instruct
By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology
3 people shipped this
verl fixed the FP8 KV cache corruption that silently broke quantized rollouts, and open-instruct now prepares the next refresh batch while the current one trains.
[vllm, rollout] fix: reset uncalibrated KV cache scales after a quantized refit verl-project/verl
Combining a quantized rollout with an FP8 KV cache made vLLM generate garbage; the compressed-tensors rollout path now resets the attention q/k/v/prob scales to 1.0 after every quantized refit when the rollout quant config carries no kv_cache_scheme. If you run FP8_BLOCK rollout with kv_cache_dtype=fp8_e4m3 today, this is the patch that makes the combination usable.
[doc] feat: add H20 results with an FP8 KV cache to the FP8 guide verl-project/verl
The FP8 guide now carries H20 results for Qwen3-30B-A3B-Base on DAPO-Math-17k, BF16 against FP8 E2E with and without an FP8 rollout KV cache, plus a section on enabling the cache. The guide states it depends on the scale-reset fix, so read it as the recipe for that combination, not a standalone one.
Overlap prepared refresh batches with training allenai/open-instruct
Setting core.prepared_batch_overlap=true prepares the next refresh batch while the current batch trains, targeting an acquisition stage measured at roughly 8.7s exposed in the three-node profile. Learning batch, lag limit, reward policy and datasets stay the same, so this is a config flip rather than a training change; startup logs the first failed eligibility condition from the same runtime gate.
Pin compiler-cache reuse and convolution warm-up fixes allenai/open-instruct
Compiler-cache reuse now survives Python orchestration edits, publishable kernels are retained during active compilation and conflicts, and FLA convolution is no longer recompiled for changing pack lengths. The new compiler_cache.tilelang option is experimental and defaults to false.
[hardware, megatron, doc] feat: upgrade Ascend 950 to Megatron-Core 0.18 and add MXFP8 GRPO example verl-project/verl
The Ascend 950 CANN 9.1.0 images move from Megatron-Core 0.12 to 0.18 with MindSpeed at core_r0.18.0, alongside a single-node eight-NPU Qwen3-30B-A3B MXFP8 GRPO example for operators on that hardware.
Action items
- → Update verl before running FP8_BLOCK rollout with kv_cache_dtype=fp8_e4m3; older revisions generate garbage on that c... verl-project/verl [immediate]
