$ the-wire · showcase
inspect_ai closes sandbox bridge config leaks, inspect_evals restores official SWE-bench images
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
Today's inspect_ai fixes stop a sandboxed client from overriding the eval author's billing, storage, truncation and tool-approval decisions, while inspect_evals moves swe-bench back to the official DockerHub images.
Agent bridge follows the eval's configuration for service tier, storage, truncation and tool options (#5650) UKGovernmentBEIS/inspect_ai
The sandbox agent bridge now follows the eval's configuration instead of the client's: a client inside the sandbox can no longer pick the billing tier with service_tier, turn provider-side storage on with store when the eval had it off, allow truncation of the conversation start, or continue from a stored response via previous_response_id. If you run evals with an agent bridge, requests now inh...
Fix approvers' modify decisions running the wrong call or logging the wrong arguments (#5651) UKGovernmentBEIS/inspect_ai
When an approver's modify decision changed the function name, the native path looked the tool up from the model's original proposal before approval, so a renamed function ran the original tool with the new arguments and the tool event logged the original arguments. Modified calls now resolve against the modified name, which matters to anyone relying on approval logs as the record of what actual...
fix(swe_bench): default to the official SWE-bench DockerHub images (6-D) UKGovernmentBEIS/inspect_evals
swe-bench defaults back to the official SWE-bench DockerHub images, reversing the February 2026 switch to the epoch-built images. The epoch images were built in October 2024 and lack the fix from September 2025 that landed in the official images, so runs against them were scoring against a known-broken environment; the fix is absent from the arm_64 images either way.
Fix trace log growing to many GB when generate caching is enabled (#5730) UKGovernmentBEIS/inspect_ai
Trace logs no longer balloon into many GB when generate caching is enabled. Computing a cache key traced every component in full, the whole message history, generate config and tool schemas, on every cached call, so log size grew quadratically with conversation length; entries remain identifiable by key through the fetch and store trace lines.
SciCode: fix extract_code to parse content between code fences UKGovernmentBEIS/inspect_evals
SciCode's extract_code only stripped the ```python and ``` markers, leaving any prose a model wrote before its code to be executed as Python. The damage compounded because compose_code adds each step's code to the script for every later step in the same problem, so one stray sentence could sink a whole problem's score.
Action items
- → Update inspect_ai to a build containing the bridge config fix before running agent-bridge evals with client-controlle... UKGovernmentBEIS/inspect_ai [plan]
- → Switch swe-bench runs back to the official SWE-bench DockerHub images and set the arm_64 variants aside UKGovernmentBEIS/inspect_evals [plan]
