$ cat ai-red-teaming/week/2026-10-19.log
the week in review · Oct 19 – Oct 25, 2026
promptfoo 0.125.0 makes Claude and Codex SDKs opt-in
By RepoJournal · composed from the cited sources · human-reviewed weekly · methodology
PyRIT also shipped a wave of fixes through its converter registry and judge path, one of them requiring admin access for targets using server credentials.
0.125.0 promptfoo/promptfoo
Both the Claude Agent and Codex SDK providers now ship disabled by default, so any suite that runs them needs them enabled explicitly before you move off 0.124.1. The same release also makes jks-js an optional peer dependency, so installs that never touched Java keystores stop resolving it.
[BREAKING] FIX: Require administrator access to create targets that use the server's stored credentials microsoft/PyRIT
Only administrators can now create targets that fall back to the server's own credentials, which covers AzureBlobStorageTarget in identity mode and in api_key mode when no token is given. If your scan jobs create targets with a non-admin service account, those jobs will fail until the role is changed.
FIX: Fence conversation transcript turns against role-line forgery microsoft/PyRIT
ConversationScorer rendered judge transcripts as flat {role}: {text} lines, so a target reply containing literal User: or Assistant: lines forged extra turns inside its own transcript. Judge scores derived from conversations with adversarial targets are no longer trusted at face value, and the fix closes that forgery path.
Fix BEAST probe for HF Pipeline and scoring bugs NVIDIA/garak
Beam search was minimising where it should have maximised, so BEAST scores against HF pipeline targets are not comparable to earlier runs. The probe also resolves model and tokenizer from generator.generator and drives generation through a proper Conversation, so plan to rescore baselines taken before this fix.
MAINT: deprecate labels on message and prompt-score queries microsoft/PyRIT
Because labels moved to attack results, filtering messages by label only ever matched an attack's final conversation. The labels on get_message_pieces_async and get_prompt_scores_async still work but now warn and point you at querying attack results instead, so the warning is the signal to rewrite those queries.
feat(providers): add portable OpenAI config schemas promptfoo/promptfoo
Chat Completions, Responses, legacy Completions and embeddings configs now have Zod input contracts, inferred TypeScript types and Draft 7 JSON Schema published through promptfoo/contracts. Tooling that validates provider config, editor autocomplete included, can now check it before a run instead of failing at request time.
feat: add provider-only environment overrides promptfoo/promptfoo
Embedded applications can supply credentials to Promptfoo lookups through a new provider-only environment layer that never reaches templates or subprocess environments. Only callbacks registered with { providerLayer: true } receive it, so existing suite and file callbacks behave as before.
fix(deps): update protobufjs to 8.8.1 promptfoo/promptfoo
Alongside: protobufjs 8.8.1 raises the dependency floor for the length-prefix fix that stops crafted 2^32 lengths from wrapping to zero in OTLP decoding, Express moves to 5.3.0 with terminal handlers that log only diagnostic text rather than parser error bodies, and the Docker sandbox example falls back to Docker Hub at the identical pinned digest when the Public ECR mirror throttles.
$ ls ai-red-teaming/week/ # the briefings behind this review
Keep up with AI Red Teaming in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.
Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.