RepoJournal
Safety Evals Safety Evals
52 wires and counting

$ follow Safety Evals

Keep up with Safety Evals in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-10
stories 35

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Slimmer inspect-ai sdist, correct dynamic sample retries

By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology

Inspect's two operator-facing fixes land today: source installs no longer ship the sandbox binaries, and task retries stop borrowing one dynamic sample's result for another.

fix: task retries no longer reuse one dynamic sample's result for another sample without an id (#6314) UKGovernmentBEIS/inspect_ai

by Ransom Richardson

A SampleSource sample admitted without an id used to take the next free integer in arrival order, and since retries and checkpoint resume key on (id, epoch) alone, a retried task could reuse a different dynamic sample's result. If you drive evals from next_samples(), sample_complete, sample_abandoned, or enqueue_sample, your retries and resumed logs now resolve to the sample that actually produ...

fix: the sdist no longer includes the sandbox tools binaries; an install built from it downloads them at first use (#6306) UKGovernmentBEIS/inspect_ai

by Eric Patey

The sdist no longer carries the two glibc sandbox tools binaries (about 29 MB of a 72.3 MB per-release footprint), so an install built from source fetches them on first use instead. If you build your eval images from source behind an air-gapped proxy, the download now happens at runtime rather than install time.

test: move Together tests and docs off retired Together models (#6431) UKGovernmentBEIS/inspect_ai

by Eric Patey

Together retired four models from serverless inference, which had left 12 live tests failing with 400s such as "Unable to access non-serverless model MiniMaxAI/MiniMax-M2.7." If your eval configs still name those Together model IDs, they will fail the same way; the tests and docs on main now point elsewhere.

MMIU: avoid caching incomplete asset directories UKGovernmentBEIS/inspect_evals

by cenab

MMIU's cache check was extracted_dir.is_dir(), which a failed download or extraction satisfies anyway, so the next run skipped preparation and died on missing images. Asset preparation now extracts into a temporary sibling and renames it into place, and caches left incomplete by earlier runs or killed downloads are recovered, so existing MMIU installs repair themselves on the next invocation.

docs: design the eval-sharding merge command (inspect log merge-shards) (#6345) UKGovernmentBEIS/inspect_ai

by Ransom Richardson

Also landed: a design for inspect log merge-shards LOG --task TASK, which will build the task from inspect eval's own options (-T, --model, --model-role, --epochs) rather than rebuilding it from the log; plus the long tail of contributor docs and CI pinning, including the ruff matrix finally running on its declared Python versions.

Quick answers

What shipped in Safety Evals on November 10, 2026?
Inspect's two operator-facing fixes land today: source installs no longer ship the sandbox binaries, and task retries stop borrowing one dynamic sample's result for another. In total, 17 commits and 18 pull requests landed.
Who contributed to Safety Evals on November 10, 2026?
4 developers shipped this update, including Ransom Richardson, Eric Patey, cenab, and Tania.
What were the notable Safety Evals updates?
fix: task retries no longer reuse one dynamic sample's result for another sample without an id (#6314), fix: the sdist no longer includes the sandbox tools binaries; an install built from it downloads them at first use (#6306), and test: move Together tests and docs off retired Together models (#6431).