$ the-wire · showcase
inspect_evals 0.23.0 fixes the HLE registry lookup error
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
inspect_evals v0.23.0 ships the scorer registration fix that unblocks running hle by name, and inspect_ai routes Gemini tool calls through the LiteLLM proxy without escaping failures.
v0.23.0 UKGovernmentBEIS/inspect_evals
The release registers the inspect_evals/hle/regex_judge and inspect_evals/hle/json_judge scorers at import time, so `inspect eval inspect_evals/hle` no longer fails with a registry lookup error during task construction. It also drops an unreachable float-type guard from AgentHarm's avg_full_score so it filters samples like its sibling metrics, which the notes state has no effect on reported sco...
gemini litellm proxy tool call escaping UKGovernmentBEIS/inspect_ai
Tool calls made through the Gemini LiteLLM proxy are now escaped correctly, with CI fixed up alongside the change. If you drive Gemini evals through the proxy and parse tool call arguments, this is the fix to pick up.
gemini litellm proxy tool call escaping (#5661) UKGovernmentBEIS/inspect_ai
The follow-up commit for the Gemini proxy fix lands the CI cleanup and locks in the tool call escaping behavior. Operators running Gemini through LiteLLM should track this as the version carrying the corrected path.
Update PR process: require authors be on the contributors list or be assigned to an issue UKGovernmentBEIS/inspect_evals
inspect_evals will now close most pull requests unless the author is on the approved contributors list or assigned to an issue by a maintainer, part of a shift toward issue-driven prioritization. If your team contributes eval fixes upstream, get assigned to an issue or expect the PR to be closed.
fix(actions): Update the Pangram labeller (#2615) UKGovernmentBEIS/inspect_evals
The Pangram labeller workflow was updated to cut false positives and make its logs easier to read. Internal CI maintenance, no change to how you run evals.
