$ the-wire · showcase
Nova models stop breaking reasoning_effort evals
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
inspect_ai now warns and ignores reasoning_effort on Nova Micro, Lite, Pro, and Premier instead of failing the run and dropping the output-token limit, plus two documentation corrections for HTTP range requests and max_samples.
fix: omit unsupported Nova reasoning configuration UKGovernmentBEIS/inspect_ai
Evaluations that set reasoning_effort no longer fail when pointed at Nova Micro, Lite, Pro, or Premier v1: those models now warn and ignore the setting, so the same config keeps its requested output-token limit instead of losing it at high effort. Other Nova model families still behave as before.
Correct the documented default for max_samples (#5632) UKGovernmentBEIS/inspect_ai
The docs claimed Inspect sets max_samples to max_connections + 1; there is no + 1 in the code, and the static branch of create_sample_semaphore sets it to max_connections outright, pinned at 15 in, 15 out. Worth re-reading if you sized concurrency against the old number, since adaptive connections are on by default and change which branch applies.
Mention HTTP range requests earlier (#5663) UKGovernmentBEIS/inspect_ai
Range requests now get called out the first time the static HTTP server is mentioned rather than buried in "Other Notes", so the requirement that your server support them is harder to miss when you set up file access.
fix: normalize lowercase UTC z suffix (#5410) UKGovernmentBEIS/inspect_ai
UtcDatetime and UtcTime parsing now normalizes a lowercase "z" UTC suffix on all supported Python versions, so timestamps written by tools that emit the lowercase form no longer fall through unparsed.
