$ the-wire · showcase
promptfoo locks down derivedMetrics, GPT-6.2 Astra and Sol land
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
promptfoo 0.126.0 will refuse to run any eval whose derivedMetrics formula uses a function call, while the CLI gains both a narrower data-sharing default for full-repo scans and first-class support for the two new GPT-6.2 models.
0.126.0 promptfoo/promptfoo
by promptfoobot[bot]
Derived metrics in promptfoo 0.126.0 no longer accept function calls, property access, assignments, or formulas longer than 1,024 characters, and a bad formula now halts the evaluation before any test case runs. The release notes say to rewrite `max(a, b)` as `a > b ? a : b`; arithmetic, comparisons, ternaries, and score names like `__count` still work.
feat(providers): support GPT-6.2 Astra and Sol promptfoo/promptfoo
promptfoo adds the public models `gpt-6.2-astra` and `gpt-6.2-sol`, selected as `openai:gpt-6.2-astra` or `openai:gpt-6.2-sol` for Responses including tools, with the `openai:chat:` prefix for Chat Completions text. Your existing configs need the new model IDs spelled out before you can evaluate against either.
code-scan-action: 0.2.3 promptfoo/promptfoo
by promptfoobot[bot]
code-scan-action 0.2.3 lets full-repository scans share eligible committed text files from the compared revision with the hosted scanner, while working-tree changes and untracked files stay out of that snapshot. Set `diffsOnly: true` to share only the Git diff, and note the action now runs on bundled Node.js 24.
probes: add bounded source-to-sink tool-chain exploration NVIDIA/garak
garak gains an opt-in `probes.agent_breaker.SourceToSink` probe and its `detectors.agent_breaker.AgentBreakerChainResult` detector, which builds candidate data-flow paths across tools from an operator-authored tool manifest instead of testing each tool in isolation. Single-tool tests can miss an unsafe operation that only becomes reachable after one tool hands an artifact to another.
fix(eval): preserve pagination after rating updates promptfoo/promptfoo
Pages, page size, search, and filters now survive a manual rating or comment save in the results table, and delayed responses can no longer overwrite a newer page selection. Reviewing a long eval no longer drops you back to page one after every rating.
Action items