$ the-wire · showcase
PyRIT marks unscored objectives UNDETERMINED, and a breaking dataset-controls change lands
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
PyRIT's biggest operational changes land on the scoring and replay paths: an objective scorer that returns no score no longer reads as an attack failure, and scenario dataset controls arrive across CLI, REST and GUI as a breaking change.
[BREAKING] FEAT: Add scenario dataset controls to CLI, REST, and GUI (#3143) microsoft/PyRIT
Scenario dataset controls are now exposed across the CLI, REST API and GUI, and the change is flagged breaking. If your pipelines spend datasets or select seeds programmatically, check the new interface against your existing calls before you upgrade.
FIX: record UNDETERMINED when the objective scorer returns no score microsoft/PyRIT
When an objective scorer is set, a response comes back, and the scorer returns no score, the attack result is now UNDETERMINED with "Objective scorer returned no score" instead of FAILURE, affecting PromptSendingAttack, MultiPromptSendingAttack, RedTeamingAttack and ChunkedRequestAttack. Real false scores, blocked responses scored false by the scorer, and missing responses still read as FAILURE...
FIX: keep Responses web search calls replayable and behind the answer microsoft/PyRIT
Responses web search calls now store the whole web_search_call item (id, status, action) and replay exactly the fields the input item accepts, with tool records ordered after the text so get_value() returns the answer again. Older stored rows carrying only type and id are skipped with a warning rather than failing the whole request, so pre-existing logs replay partially.
FEAT: Add dataset example table and seed detail view microsoft/PyRIT
The /datasets/detail?selection_key= selection page gains a read-only paginated table of logical examples with an expandable detail view, backed by typed datasetsApi methods. Pages load one at a time and members fetch only on expansion via GET /api/datasets/seeds/{example_id}?selection_key=, so the catalog never prefetches seed pages.
MAINT: P311-16 remove Python 3.10 trailing-Z workaround in _parse_datetime (#3176) microsoft/PyRIT
PyRIT dropped the Python 3.10 trailing-Z workaround in _parse_datetime, a small cleanup that only matters if you still run the library on 3.10. promptfoo's day was maintenance only: unused view and telemetry hook exports removed, eval import normalization reused, and assertion fixtures consolidated, with no dependency changes.
Action items
