RepoJournal

$ cat ai-safety-evals/week/2026-11-16.log

Safety Evals

Safety Evals

the week in review · Nov 16 – Nov 22, 2026

Inspect closes log-recovery path escapes and fixes sandbox leaks

By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology

♥

Three breaking fixes harden crashed-log recovery and checkpoint snapshots, while remote MCP credentials stop landing in eval logs.

118 commits 132 PRs merged 1 releases 7 briefings covered

all ai-safety-evals reviews →

fix: a malformed hash in a shared-buffer manifest can no longer make log recovery open a file outside the buffer (#6627) UKGovernmentBEIS/inspect_ai

by Craig Walton

A malformed hash in a shared-buffer manifest can no longer steer recovery to a file outside the `.buffer/` directory. If you run `inspect log recover` or `recover_eval_log()` on logs you did not generate yourself, this is the version to be on.

fix: confine temporary attachment files during crashed-log recovery (#6626) UKGovernmentBEIS/inspect_ai

by Craig Walton

The same recovery path built temporary attachment file paths from IDs taken out of the companion `.buffer`, which let someone who controlled the log point writes elsewhere. Both recover entry points now confine those files.

fix: missed sandbox cleanup and LookupError on E2B, Modal and Daytona with several sandbox configurations or an empty SampleSource seed (#6633) UKGovernmentBEIS/inspect_ai

by Ransom Richardson

E2B, Modal and Daytona configurations could leak sandboxes or raise `LookupError` when several sandbox configs shared a run or a `SampleSource` seed came up empty. If you fan out across providers in one eval, check your sandbox accounting after upgrading.

fix: remote MCP server credentials are no longer written to eval logs (#6492) UKGovernmentBEIS/inspect_ai

by Charles Teague

An HTTP MCP server used with `execution="remote"` plus `authorization=` or `headers=` had those credentials written in plaintext into the eval log, three times per sample. Existing logs from earlier runs still contain them; rotate what you can.

fix: replay pairs each tool call with its own approvals and sentinel checks when a call id repeats UKGovernmentBEIS/inspect_ai

by dragonstyle

When a provider reused a tool call id, a cache hit, or an agent bridge handed the same call out twice, sentinel replay paired approvals with the wrong call. Replay of logs with repeated ids now reflects what each call actually got.

fix: harden checkpoint snapshot IDs against option injection (#6580) UKGovernmentBEIS/inspect_ai

by Craig Walton

Backup summaries accepted arbitrary snapshot IDs and restic operands lacked an option terminator. Existing egress and resume validation covered current callers, so the exposure was other callers; the operands are now terminated.

fix(sciknoweval): route grader failures to unscored, not fabricated grades UKGovernmentBEIS/inspect_evals

by melcheikh

The three model-graded scorers in sciknoweval wrote 0 when a grader failed to parse, and every `type: score` prompt in `evaluator_prompt.yaml` runs a 1 to 5 scale, so 0 was never a point on it. Failures now route to unscored rather than fabricated grades.

fix: apply tag filters when selecting seeds and dimensions by tag safety-research/petri

by vayllon301

Selecting seeds or judge dimensions by `tags:` skipped the separate `tags=` filter, so a run could pull in inputs or dimensions outside the subset you asked for. Filters now apply consistently with explicit IDs. Alongside: MBPP moved to a run-config layout with an agentic default solver, and three register submissions landed (FinQA, BaxBench, MonitorBench).

$ ls ai-safety-evals/week/ # the briefings behind this review

Keep up with Safety Evals in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

all ai-safety-evals reviews →