$ the-wire · showcase
Garak fixes BEAST beam search direction, Promptfoo locks down OpenAI provider config
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
The most operator-relevant change today is a scoring correction in garak's BEAST probe: beam search had been minimizing the score instead of maximizing it, so past BEAST results against Hugging Face pipeline targets may not mean what they appeared to.
Fix BEAST probe for HF Pipeline and scoring bugs NVIDIA/garak
The BEAST probe now resolves model and tokenizer from generator.generator and wires generation through a Conversation rather than a raw string, so it runs against Hugging Face pipeline targets at all. The scoring bug is the part that matters: beam search was minimizing the score instead of maximizing it, which means BEAST runs you recorded against HF pipelines should be treated as unreliable an...
feat: add provider-only environment overrides promptfoo/promptfoo
Embedded applications can now pass credentials to Promptfoo through an environment layer that never reaches templates or subprocess environments, but only callbacks registered with { providerLayer: true } receive it. Existing suite and file callbacks and CLI configuration are untouched, so nothing changes unless you opt in.
feat(providers): add portable OpenAI config schemas promptfoo/promptfoo
OpenAI provider config was previously unrestricted at the provider-options boundary; it now has Zod input contracts, inferred TypeScript types and Draft 7 JSON Schema for Chat Completions, Responses, legacy Completions and embeddings under promptfoo/contracts. Explicit-route unions keep each id paired with its config and reject malformed known fields rather than falling through to a catch-all, ...
chore(deps): upgrade Express to 5.3 safely promptfoo/promptfoo
Express moves from 5.2.1 to 5.3.0 here, with the minimum raised in the root package and all nine standalone Express examples. Because 5.3 inspects the whole error object when logging and parser errors can carry the submitted body, terminal handlers were added that pass only the existing diagnostic text, so request and configuration fields do not begin appearing in your logs.
chore(deps): migrate disk cache to Keyv 6 promptfoo/promptfoo
The disk cache moves to Keyv 6.1.0 behind a typed adapter that keeps the existing keyv: keys, serialized envelopes and absolute expiry. cache-manager stays on its Keyv 5 facade, so existing cache entries remain readable without a migration step.
