$ the-wire · showcase
Exec_remote orphans and lost hooks fixed, shard logs merge
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
2 people shipped this
inspect_ai shipped three operator-visible fixes that change how cancelled tasks, remote commands and multi-worker runs behave, plus the merge core that turns a sharded run's per-worker logs back into one eval log.
fix: on_task_end hook not called when a task is cancelled or fails (#6273) UKGovernmentBEIS/inspect_ai
Tasks that end by cancellation or failure now fire on_task_end, so a hook you rely on for teardown, scoring, or cleanup no longer silently skips those runs. The hook was documented to bracket task_run but was only called on the normal completion path, even though every aborted task still wrote a final log.
fix: exec_remote no longer leaves a command running when its start is cancelled or times out (#6275) UKGovernmentBEIS/inspect_ai
exec_remote no longer leaves a command running in the sandbox when its start request is cancelled or times out. Previously the job's pid was known only from the start reply, so a lost or timed-out reply left a running command with no handle to stop it.
feat: merge a sharded run's per-worker logs into one eval log (#5902) UKGovernmentBEIS/inspect_ai
A task split across workers writes one shard log per process under <dir>/<name>.shards/<k>/, and the merge core and public API to fold those into a single eval log with whole-task results has landed. This is the first shipped step of the eval-sharding design, so plan on shard-to-log merging being available rather than needing to stitch logs yourself.
fix: exec_remote returns fast commands without a fixed 5 s poll wait (#6313) UKGovernmentBEIS/inspect_ai
Fast remote commands return about one request after they exit instead of sitting through a fixed 5 s sleep; the release notes' \"sleep 1\" example drops from about 6.3 s. The sandbox tools server now accepts an optional wait on exec_remote_poll (0 to 60 s, default 0).
fix: read ctl --file stdin with click.open_file (#6336) UKGovernmentBEIS/inspect_ai
Smaller and host-side only: the --file - path now reads UTF-8 stdin through click.open_file after click 8.5.0 deprecated click.get_text_stream and broke the asyncio canary's type checks.
