RepoJournal
AI Red Teaming AI Red Teaming
52 wires and counting

$ follow AI Red Teaming

Keep up with AI Red Teaming in about 3 minutes: what actually shipped — the commits, pull requests, releases, and security advisories that matter.

or

fair warning: these emails are deeply technical. diffs, version numbers, CVEs, benchmark deltas. if that's not your idea of a good read, this isn't your newsletter.

Folds into your digest — weekly by default, monthly if you prefer. Unsubscribe in one click.

$ status

wire 2026-11-14
stories 49

© 2026 RepoJournal Home Showcase How it works Privacy

$ the-wire · showcase

PyRIT alpha fixed for trial count, TAP errors no longer read as refusals

By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology

PyRIT shipped fixes that change what your existing scores and attack outcomes actually mean, and moved its Python floor to 3.11, while promptfoo wired in Gemini 4 and the Gemini Decisions API.

FIX: Add a trial-invariant gold-vs-model Krippendorff alpha microsoft/PyRIT

by feiiiiii5

krippendorff_alpha_combined concatenates human and model scores so each scorer trial counts as an extra rater, and the PR documents a scorer that ignores the response climbing from 0.086 at one trial to 0.828 at ten while MAE held at 0.381. The new krippendorff_alpha_gold_model measures a two-row alpha between the gold median human label and the per-response median model score, so the number st...

FIX: Report ERROR when every TAP branch crashed before a score microsoft/PyRIT

by u7k4rs6

When every TAP or PAIR branch threw before anything was scored, the run returned FAILURE at a 0.00 best score, indistinguishable from a target that refused every prompt. It now returns ERROR with the branch error count and the last error as the reason and sets error_message on the result; any-branch-scored runs keep their previous behavior.

[BREAKING] FIX: Keep a private copy's endpoint and credentials the same as its source's (part 17) (#3984) microsoft/PyRIT

by varunj-msft

A private copy of a target or converter created through the API now inherits its source's endpoint, sign-in, and credentials, with the API refusing copy settings for those fields and the converter settings dialog no longer displaying them. Scripts that set endpoint or credentials on a copy will no longer take effect.

v1.2.0 microsoft/PyRIT

by hannahwestra25

PyRIT v1.2.0 makes the CoPyRIT GUI the headline: reusable scenario configurations, grouped launches, persistent engagements and findings, and richer result analytics, plus new attack techniques, targets, converters, datasets, and scoring options. Python support is now >=3.11, <3.15, so anything still running 3.10 must move before the upgrade.

feat(providers): support Gemini 4 Argon and Hydrogen promptfoo/promptfoo

by mldangelo

The provider adds gemini-4-hydrogen, generally available on the Gemini API (free and paid tiers) and Vertex AI, and the same branch carries the earlier Gemini 4 Argon support that Hydrogen's request handling depends on. The quiet remainder: a tsdown bump to 0.23.5, Bedrock Converse cost lookups for every us./global. GPT-6 system profile rather than a fixed pair, and google:decisions:gemini-deci...

Quick answers

What shipped in AI Red Teaming on November 14, 2026?
PyRIT shipped fixes that change what your existing scores and attack outcomes actually mean, and moved its Python floor to 3.11, while promptfoo wired in Gemini 4 and the Gemini Decisions API. In total, 22 commits, 26 pull requests, and 1 releases landed.
Who contributed to AI Red Teaming on November 14, 2026?
8 developers shipped this update, including renovate, mldangelo-oai, mldangelo, hannahwestra25, varunj-msft, feiiiiii5, u7k4rs6, and behnam-o.
What were the notable AI Red Teaming updates?
FIX: Add a trial-invariant gold-vs-model Krippendorff alpha, FIX: Report ERROR when every TAP branch crashed before a score, and [BREAKING] FIX: Keep a private copy's endpoint and credentials the same as its source's (part 17) (#3984).