$ the-wire · showcase
Inspect AI 0.3.287 resumes Gemini long decodes instead of truncating
By RepoJournal · Filed · About Safety Evals · Composed from the cited sources · methodology
Inspect's google provider now chases Gemini 4 Argon's continuation_token until a real finish reason arrives, so long decodes stop returning a silent 32K-token truncation, and 0.3.287 ships that alongside a react() runtime that can rewrite its own conversation.
fix(google): Gemini long decode responses no longer stop at the per-request output cap UKGovernmentBEIS/inspect_ai
Inspect used to send one request and hand back whatever came with finish reason CONTINUATION, which with the locked google-genai 2.19 SDK meant your log held only the first 32K tokens of the answer. The fix resends the same request with the continuation_token until a different finish reason comes back, so non-streaming evals no longer score a truncated response as a complete one and streaming r...
v0.3.287 UKGovernmentBEIS/inspect_ai
The 0.3.287 release lets a runtime hold and rewrite a react() conversation at turn boundaries, and compaction handlers gain reset(), so long agentic evals can compact history without losing the ability to reset state. It also reserves the range_goal registry type so inspect_ranges goal types can be packaged and self-discovered through the shared registry rather than a private one.
feat: reserve the range_goal registry type for inspect_ranges goal types (#6224) UKGovernmentBEIS/inspect_ai
inspect_ranges is moving its challenge goal-type registry into the shared inspect-ai registry, following the scout/sentinel/swarm entry-point pattern. If you ship your own goal types, the namespaced range_goal kind is now reserved for that package, mirroring how existing dependent-package reservations work.
Action items
- → Upgrade inspect_ai to 0.3.287 before rerunning Gemini 4 Argon evals, since earlier versions silently truncated long d... UKGovernmentBEIS/inspect_ai [plan]
