$ the-wire · showcase
PyRIT hardens converters and scoring, drops write-only deliveries from results
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
PyRIT shipped a batch of correctness fixes that change how results are scored, how converters build prompts, and what conversation storage accepts, several of which will alter existing scan output or raise errors at construction.
[BREAKING] FIX: Do not score write-only target deliveries as attack responses (#3757) microsoft/PyRIT
Deliveries to write-only targets are no longer recorded as attack responses, so scans against targets that do not return content will no longer log fabricated successes in your result set. The fix is flagged breaking, which means previous scores from write-only targets are not comparable to what you get now.
FIX: Require ordered template parameters in LLMGenericTextConverter microsoft/PyRIT
LLMGenericTextConverter rejected set and frozenset template parameters at construction with a ValueError, because their iteration order varies with PYTHONHASHSEED and the same config could emit different prompts across processes. If any of your converter configs pass sets, they will now fail to build until you switch to an ordered input.
FIX: Reject non-dictionary metadata before conversation updates microsoft/PyRIT
Non-dictionary prompt_metadata roots now raise ValueError before anything is written to storage, instead of breaking later message retrieval. A mixed conversation update carrying invalid metadata leaves all fields unchanged, so a rejected write no longer half-applies.
FIX Insert tatweel only between Arabic letters that join in TatweelConverter microsoft/PyRIT
TatweelConverter used to insert U+0640 between every pair of adjacent Arabic letters, producing detached strokes in words like the documented example output. It now inserts tatweel only where the preceding letter actually joins, using the 52 right-joining and 2 non-joining letters in the U+0600 to U+06FF range, so tatweel-based perturbation prompts render differently than before.
FIX: Bind Attack History requests to the runtime generation microsoft/PyRIT
Attack History requests, pagination and filters are now bound to the current runtime generation: late responses from an obsolete runtime are ignored, and switching runtimes resets pagination without carrying over the old cursor, filter options or errors.
Action items