$ the-wire · showcase
inspect_ai drops Python 3.10, grades stop failing on fenced JSON
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
inspect_ai 0.3.281 raises the Python floor to 3.11, tightens the viewer's security headers, and fixes a sandbox bridge denial of Antigravity host tool calls, while inspect_evals stops discarding correct judge grades wrapped in Markdown fences.
v0.3.281 UKGovernmentBEIS/inspect_ai
inspect_ai now requires Python 3.11 (requires-python is >=3.11) and is tested on Python 3.14, so any evaluation environment still on 3.10 must move before it can install this version. The release also adds swarm_controller and swarm_channel registry types for agent work.
fix(view): enforce the log viewer's Content-Security-Policy in inspect view and view bundle (#5552) UKGovernmentBEIS/inspect_ai
The log viewer's Content-Security-Policy is now read from content-security-policy.json in the dist and sent on every response by inspect view and the view bundle, plus frame-ancestors 'none'. A dist without that file keeps the old frame-ancestors-only header, but a malformed file fails startup, so custom builds need a valid policy file.
fix: Antigravity host tool calls being denied as unproposed by the sandbox agent bridge (#5717) UKGovernmentBEIS/inspect_ai
Antigravity host tool calls were being denied as unproposed by the sandbox agent bridge, because the execution-grant resolver matches the proposed call against the description the scaffold declared to the model. Bridged host tools now execute when the scaffold prefixes the MCP description, which unblocks those runs.
fix(hle, paperbench, sosbench): accept judge JSON wrapped in a code fence UKGovernmentBEIS/inspect_evals
Some OpenRouter hosts advertise structured output without enforcing it and wrap judge JSON in a Markdown code fence, which the strict parse rejected and threw away a correct grade. HLE's json_judge, PaperBench's parse step, and SOSBench now accept a single fence around the whole reply while prose around the JSON still fails validation, so scores from those providers are trustworthy again.
docs(design): compose-prefix verification mode and decisions for the S3 incremental flush design (#5608) UKGovernmentBEIS/inspect_ai
The S3 incremental flush design gained a compose-prefix verification mode, since the recorder's correctness under compose depends on every writer of the temp file's leading bytes clearing the compose prefix and a writer that forgets would upload a corrupt .eval file.
Action items
- → Move inspect_ai evaluation environments to Python 3.11 before upgrading to 0.3.281 UKGovernmentBEIS/inspect_ai [plan]
