$ cat ai-safety-evals/week/2026-11-02.log
the week in review · Nov 2 – Nov 8, 2026
inspect_ai 0.3.288 drops google-genai 2.x
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
The release also stops reusing sandbox tools left by another Inspect version and hardens model-graded prompts; Petri needed a chore to keep up.
v0.3.288 UKGovernmentBEIS/inspect_ai
Environments still on google-genai 2.x fail the validate_google_client version check, so install google-genai>=3.0.0 before you bump. The release also changes how automatically generated Docker sandboxes are handled, per the changelog's breaking section.
chore: CompactNone implements the reset() that inspect_ai's Compact protocol now requires (#192) safety-research/petri
inspect_ai 0.3.287 added reset(prefix) to the public Compact protocol, so CompactNone broke strict pyright checks. Petri now pins itself to the newer inspect_ai and implements it; if you subclass or share compaction code across the two, do the same before upgrading.
fix: sandbox tools installed by another Inspect release are no longer reused (#6302) UKGovernmentBEIS/inspect_ai
Inspect used to install sandbox tools at one fixed path and reuse any trustworthy tree there, whatever version of Inspect had written it. Tools from another release are no longer reused, so a long-lived sandbox image gets a fresh install instead of a stale launcher.
fix: harden model-graded scorer prompts against fake data markers in the graded answer (#6282) UKGovernmentBEIS/inspect_ai
A submission could close the *** separators inside the [BEGIN DATA] block and have text read as instructions. model_graded_qa() and model_graded_fact() now harden the prompt against forged markers, which changes the prompt your graded runs send, so expect cached scores to shift.
feat: run each agent's sandbox tools as its own user (#6190) UKGovernmentBEIS/inspect_ai
sandbox_identity() runs an agent's tools as a named Linux user, which is what lets several agents share one sandbox without their shell commands landing in the same account. Custom tool calls inside the scope got the same treatment later in the week.
fix: on_task_end hook not called when a task is cancelled or fails (#6273) UKGovernmentBEIS/inspect_ai
on_task_end previously fired only on normal completion, so cleanup registered against it silently skipped on cancellation or failure. It now runs in those cases too, which means hooks written assuming a completed run will see partial state.
fix: computer() keeps its latest screenshots when other tools have also run (#6180) UKGovernmentBEIS/inspect_ai
Agents using computer() alongside any other tool stopped receiving screenshots after the other tool returned its first result. If you measured computer-use runs this way, those transcripts were missing images the model never saw.
fix(math): surface an unusable antlr4 runtime instead of scoring every sample wrong UKGovernmentBEIS/inspect_evals
expression_exact_match_sympy caught every exception and returned False, so a missing or incompatible antlr4-python3-runtime scored every sample wrong rather than reporting an error. The runtime problem now surfaces; the same scorer's 5-second sympy timeout no longer depends on SIGALRM, which it never had on Windows.
$ ls ai-safety-evals/week/ # the briefings behind this review
Keep up with Safety Evals in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.