RepoJournal
Safety Evals Safety Evals
16 wires and counting

$ follow Safety Evals

Keep up with Safety Evals in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-10-12
stories 4
RSS Archive Reviews

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

inspect_evals moves shared test tooling into pytest-inspect-evals 0.1.0

By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology

MattFisher
1 person shipped this
MattFisher @MattFisher 1 cited

The only change across the desk today is internal: inspect_evals stopped carrying its own copy of the shared test gates, fixtures and helpers and now pulls them from a published package.

Use pytest-inspect-evals for shared test gates, fixtures and helpers UKGovernmentBEIS/inspect_evals

by MattFisher

The shared test gates, fixtures and helpers now live in pytest-inspect-evals 0.1.0, and inspect_evals depends on it, so eval repos built from inspect-evals-template get the same tooling by adding one dev dependency instead of copying tests/conftest.py and tests/utils/. The maintainers say the copied version "has already drifted", which is why the same extraction pattern used for inspect-evals-l...

$ ls ai-safety-evals/reviews/ # the step-back reads

the week in review Control-arena v20 and inspect_evals v0.24 break APIs
Full archive → Weekly reviews →

Quick answers

What shipped in Safety Evals on October 12, 2026?
The only change across the desk today is internal: inspect_evals stopped carrying its own copy of the shared test gates, fixtures and helpers and now pulls them from a published package. In total, 2 commits and 2 pull requests landed.
Who contributed to Safety Evals on October 12, 2026?
1 developer shipped this update, including MattFisher.
What were the notable Safety Evals updates?
Use pytest-inspect-evals for shared test gates, fixtures and helpers.
♥

$ log Safety Evals --recent

Inspect adds tool state scopes, fixes usage undercounting 22 shipped inspect_ai drops Python 3.10, grades stop failing on fenced JSON 23 shipped Docker sandbox stops unpacking agent archives on your host 44 shipped Inspect blocks writable host mounts, control-arena rescues 6 weeks of swe_django side-task scores 58 shipped

$ log --all # elsewhere on the wire

Alignment & Post-training verl rejects conflicting vLLM engine_kwargs, open-instruct adds MILES OPD behind a Megatron learner Interpretability nnsight 0.8.0 cuts import time from 7.6s to 1.6s

$ ls showcases/

AI Red Teaming Agent Security Interpretability Alignment & Post-training

$ init your-desk

RepoJournal can write one of these for any repo you care about. Sign in with GitHub, pick your repos and sources, and your desk gets a wire like this one every morning.