$ the-wire · showcase
Per-sample sandbox survives log reads, and batch rows stop taking each other down
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
inspect_ai fixed a log-reading bug that silently returned None for per-sample sandboxes, alongside a batch-parsing fix that keeps valid rows when a neighbour is malformed, and inspect_evals shipped v0.29.0 with changed BrowseComp grading.
fix!: per-sample sandbox lost from sample_init events when reading logs and in recovered logs (#6112) UKGovernmentBEIS/inspect_ai
Reading a log used to drop the per-sample sandbox from every sample_init event, so SampleInitEvent.sample.sandbox came back None even though the file had it; the same data was missing from recovered logs. Marked breaking, and the reproduction is one line: Sample.model_validate(s.model_dump()).sandbox now preserves what you configured.
fix: keep the other batch results when one result row fails to parse (#6217) UKGovernmentBEIS/inspect_ai
Malformed or partial file-batch result rows could discard otherwise valid rows or leave requests waiting indefinitely. OpenAI batch error rows now keep their retryable HTTP status, and non-numeric batch error codes no longer raise while being converted.
v0.29.0 UKGovernmentBEIS/inspect_evals
by github-actions[bot]
BrowseComp (v6-C) now reads the grader's verdict from the last correct: line, ignoring case and markdown emphasis, so "Correct: yes" scores as correct and "incorrect: yes" in the reasoning no longer does; a grader reply with no readable verdict is now unscored rather than counted as incorrect. MMIU also stops treating failed asset downloads or extractions as cached for later runs. Expect Browse...
feat: sandbox agent bridge inside a sandbox identity scope serves only the scope's user (#6270) UKGovernmentBEIS/inspect_ai
Inside sandbox_identity() scopes, several agents sharing one sandbox as different Linux users also share its loopback, so a sandbox_agent_bridge() model attached in one scope could be reached from another. The bridge now serves only the scope's user.
fix(serde): load JSON integers into float fields (#937) UKGovernmentBEIS/control-arena
JSON decodes whole numbers as ints, and load_json only returned a primitive when the type matched exactly, so `control-arena eval single --trusted-config.temperature 0` raised Cannot deserialize. Integer values now load into float fields.
docs: use current Claude models in examples (#6508) UKGovernmentBEIS/inspect_ai
Docs examples named Claude models Anthropic has retired, including claude-3-opus-20240229 and claude-sonnet-4-20250514, so copy-pasted configs errored; they now use current models. Elsewhere, petri's publish pipeline builds the PyPI dist in a job with no publishing credential and deploys the docs site from its release tag, since the site was last published by hand on Jun 8.