$ the-wire · showcase
Haiku 5.5 lands in inspect_ai; variadic tools stop demanding args
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
inspect_ai adds first-class Claude Haiku 5.5 support and fixes a schema bug that broke any tool declaring *args or **kwargs, while inspect_evals repairs three scorers that were marking valid answers wrong.
Support for Claude Haiku 5.5 (#5752) UKGovernmentBEIS/inspect_ai
Haiku 5.5 (claude-haiku-5-5) now has a model-info entry with tiers and gating verified against the live API, instead of falling through to Fable's rules as an untiered codename. Operators get 1M context, 128K output, adaptive thinking on by default, effort default medium, and a June 2026 knowledge cutoff without hand-tuning the provider config.
fix(tool): exclude *args and **kwargs from tool schema and parameter parsing UKGovernmentBEIS/inspect_ai
Tools declaring *args or **kwargs no longer advertise phantom required args and kwargs properties to the model, which previously made calls fail with "No type annotation available for parameter args" or "Required parameter args not provided to tool call." If any of your tools use Python variadics, they can be executed and called correctly again.
fix(mind2web): seed each sample's candidate shuffle UKGovernmentBEIS/inspect_evals
Mind2Web's format_sample shuffled each sample's top-k candidates with an unseeded module-level Random, so the option letters, and therefore the correct answer for a sample, changed on every dataset load; on the first test_domain shard, 275 of 352 samples with a correct element got a different letter on a second load. The shuffle is now seeded per sample, so scores are reproducible across runs.
fix(personality): stop rejecting valid answers with trailing text UKGovernmentBEIS/inspect_evals
The Personality any_choice scorer was scoring valid answers INCORRECT because its capture group swallowed trailing spaces and prose: "ANSWER: C " captured "C " and a trailing reason captured the whole phrase. The capture now takes letter or digit tokens optionally separated by commas, so previously understated results should be re-run.
fix(bfcl): validate explicit defaults through shared type rules UKGovernmentBEIS/inspect_evals
BFCL's marker now routes explicit optional inputs through the shared type rules, closing the gap where a value like 1.0 could pass for a whole number after the earlier forgiving change. Re-check any BFCL numbers produced since that relaxation.
