$ the-wire · showcase
MILES ships a 25-benchmark eval panel and simpler config presets
By RepoJournal · Filed · About Alignment & Post-training · Composed from the cited sources · methodology
3 people shipped this
open-instruct's MILES template now runs the full 25-entry post-training and text-safety evaluation panel at training endpoints only, while a rewritten set of env and run defaults changes how existing jobs resolve.
Add the post-training evaluation panel to the large MILES template allenai/open-instruct
The large MILES template's three starter benchmarks are replaced by all 25 distinct entries of the agreed post-training development and text-safety panel, at full dataset sizes with pinned sampling defaults. It sets evaluation.interval=0 with initial=true and final=true, so the panel runs before and after training only, even on extended runs, and zero now disables periodic evaluation globally o...
Improve MILES configuration defaults, examples, and guidance allenai/open-instruct
The four size examples collapse to dev (one-GPU mechanics check), node (one eight-GPU node: two EP2 training GPUs, six inference GPUs), and large (H1's three-node/24-GPU footprint with no idle GPUs). Operators also get written-down defaults: scoring_pass_required is false on main, Core skips the standalone scoring pass when the recipe makes it redundant, and the help text now states what forcin...
mason: an explicit --env replaces mason's default of the same name allenai/open-instruct
An explicit --env NAME=VALUE or --secret NAME=SECRET now replaces any env var mason adds by default under the same name. Previously only DEFAULT_ENV_VARS honoured the override, and the WEKA HF_HOME/HF_DATASETS_CACHE/HF_HUB_CACHE/CHECKPOINT_OUTPUT_DIR, multi-node NCCL, PATH, WANDB resume and useful-secret vars were appended after --env, so the job got both.
[vllm, rollout, perf] refactor: stage LoRA buckets in one host block (#8352) verl-project/verl
verl stages incoming LoRA adapter tensors in a single host block instead of cloning each tensor on device and again in _update_weights before moving them to pinned host memory one at a time; per-expert MoE LoRA made that tens of thousands of small transfers. The PR states it is performance only, with no behavior change, for setups syncing through update_weights_from_ipc with peft_config and bas...
Action items
- → Review mason job definitions for --env/--secret names that collide with default WEKA, NCCL, PATH or WANDB vars; those... allenai/open-instruct [plan]
