RepoJournal
Safety Evals Safety Evals
52 wires and counting

$ follow Safety Evals

Keep up with Safety Evals in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-21
stories 26

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

Petri rejects mistyped inputs; Inspect fixes Claude Mythos 5.2 thinking-block 400s

By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology

Two operator-facing fixes land: petri no longer silently runs a mistyped path as your seed text or prompt, and inspect_ai stops a history edit from killing long Mythos 5.2 samples with a 400.

feat: recover_eval_log can finalize crashed logs without a sample buffer, with a chosen status, completion time and log update (#6652) UKGovernmentBEIS/inspect_ai

by rasmusfaber

recover_eval_log now finalizes crashed .eval logs that have no sample buffer, and accepts a chosen status, completion time and log update, so a platform killing runs can write a final log itself instead of raising RecoveryNotAvailable and hand-rolling the rest.

fix: reject mistyped inputs instead of running them as seed text safety-research/petri

by kaifronsdal

A typo like -T seed_instructions=./my_seeds on a missing directory used to run one audit whose seed was the literal string "./my_seeds", report success and give no hint anything was off; it now raises FileNotFoundError naming the option and the path. The same applies to a missing file passed as system_message or user_message, which previously became the auditor's entire prompt.

fix: send the auditor the full `create_tool` instructions (#236) safety-research/petri

by Kai Fronsdal

The auditor model was seeing only 3,262 of roughly 10,560 characters of create_tool guidance, with the description cut off at a dangling "Examples" heading, so the schema example, "Tool Design Principles", "Tool Use Example" and "Auditor Role Separation" never reached it. A bare Example: line in the docstring caused the truncation; audits now get the full instructions.

feat: deprecate `realism_filter` in favor of an approval policy safety-research/petri

by kaifronsdal

realism_filter is deprecated in favor of configuring realism_approver through an approval policy, because the dedicated parameter made the approver look like part of the default pipeline. Nothing is removed: the parameter keeps working and now prints a warning, so existing audit configs run unchanged while you migrate.

feat: logs record the tool call id a bridged agent received when the bridge gave a call a new id (#6675) UKGovernmentBEIS/inspect_ai

by Charles Teague

Two sentinel-side improvements: replay now shows a bridged agent's tool calls and results as the model saw them instead of skipping every tool step of an inspect_swe run, and logs record the new tool call id a bridge assigns when a provider reuses one id across calls (Mistral sends "null" for every omitted id).

Quick answers

What shipped in Safety Evals on November 21, 2026?
Two operator-facing fixes land: petri no longer silently runs a mistyped path as your seed text or prompt, and inspect_ai stops a history edit from killing long Mythos 5.2 samples with a 400. In total, 13 commits and 13 pull requests landed.
Who contributed to Safety Evals on November 21, 2026?
5 developers shipped this update, including jjallaire, rasmusfaber, Matt Brandly, Charles Teague, and kaifronsdal.
What were the notable Safety Evals updates?
feat: recover_eval_log can finalize crashed logs without a sample buffer, with a chosen status, completion time and log update (#6652), fix: reject mistyped inputs instead of running them as seed text, and fix: send the auditor the full `create_tool` instructions (#236).