RepoJournal
Safety Evals Safety Evals
52 wires and counting

$ follow Safety Evals

Keep up with Safety Evals in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-13
stories 72

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Per-sample sandbox survives log reads, and batch rows stop taking each other down

By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology

inspect_ai fixed a log-reading bug that silently returned None for per-sample sandboxes, alongside a batch-parsing fix that keeps valid rows when a neighbour is malformed, and inspect_evals shipped v0.29.0 with changed BrowseComp grading.

fix!: per-sample sandbox lost from sample_init events when reading logs and in recovered logs (#6112) UKGovernmentBEIS/inspect_ai

by Matt Brandly

Reading a log used to drop the per-sample sandbox from every sample_init event, so SampleInitEvent.sample.sandbox came back None even though the file had it; the same data was missing from recovered logs. Marked breaking, and the reproduction is one line: Sample.model_validate(s.model_dump()).sandbox now preserves what you configured.

fix: keep the other batch results when one result row fails to parse (#6217) UKGovernmentBEIS/inspect_ai

by mikemikimike

Malformed or partial file-batch result rows could discard otherwise valid rows or leave requests waiting indefinitely. OpenAI batch error rows now keep their retryable HTTP status, and non-numeric batch error codes no longer raise while being converted.

v0.29.0 UKGovernmentBEIS/inspect_evals

by github-actions[bot]

BrowseComp (v6-C) now reads the grader's verdict from the last correct: line, ignoring case and markdown emphasis, so "Correct: yes" scores as correct and "incorrect: yes" in the reasoning no longer does; a grader reply with no readable verdict is now unscored rather than counted as incorrect. MMIU also stops treating failed asset downloads or extractions as cached for later runs. Expect Browse...

feat: sandbox agent bridge inside a sandbox identity scope serves only the scope's user (#6270) UKGovernmentBEIS/inspect_ai

by Ransom Richardson

Inside sandbox_identity() scopes, several agents sharing one sandbox as different Linux users also share its loopback, so a sandbox_agent_bridge() model attached in one scope could be reached from another. The bridge now serves only the scope's user.

fix(serde): load JSON integers into float fields (#937) UKGovernmentBEIS/control-arena

by Víctor Fernández Ayllón

JSON decodes whole numbers as ints, and load_json only returned a primitive when the type matched exactly, so `control-arena eval single --trusted-config.temperature 0` raised Cannot deserialize. Integer values now load into float fields.

docs: use current Claude models in examples (#6508) UKGovernmentBEIS/inspect_ai

by Matt Brandly

Docs examples named Claude models Anthropic has retired, including claude-3-opus-20240229 and claude-sonnet-4-20250514, so copy-pasted configs errored; they now use current models. Elsewhere, petri's publish pipeline builds the PyPI dist in a job with no publishing credential and deploys the docs site from its release tag, since the site was last published by hand on Jun 8.

Quick answers

What shipped in Safety Evals on November 13, 2026?
inspect_ai fixed a log-reading bug that silently returned None for per-sample sandboxes, alongside a batch-parsing fix that keeps valid rows when a neighbour is malformed, and inspect_evals shipped v0.29.0 with changed BrowseComp grading. In total, 35 commits, 36 pull requests, and 1 releases landed.
Who contributed to Safety Evals on November 13, 2026?
11 developers shipped this update, including Ransom Richardson, mikemikimike, Eric Patey, Matt Brandly, github-actions[bot], MattFisher, Vedant-Agarwal, and Abelo9996, and 3 more.
What were the notable Safety Evals updates?
fix!: per-sample sandbox lost from sample_init events when reading logs and in recovered logs (#6112), fix: keep the other batch results when one result row fails to parse (#6217), and v0.29.0.