$ the-wire · showcase
PyRIT readies attack audit views, garak patches its dependency surface
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
10 people shipped this
Two red-teaming operators shipped new plumbing today: PyRIT added a GUI for reviewing saved attack results and locked down how its backend authenticates, while garak pulled in dependency updates flagged for security concerns.
FEAT: Add saved attack results analytics GUI microsoft/PyRIT
PyRIT gained a GUI for browsing saved attack results, so past runs can be inspected and compared without hitting the database or re-parsing output files. It lands alongside a 14-package dependency bump covering datasets 5.1.0, fastapi 0.143.0 and pydantic 2.14.1, so re-check your pins if you pin transitively.
MAINT: Fail Closed on Missing GUI Authentication Configuration microsoft/PyRIT
The GUI backend now refuses to start unless you declare explicit local mode or a complete Microsoft Entra configuration; missing, partial, contradictory or unknown auth settings fail closed. Existing local, Docker, CI and Azure workflows keep their normal commands, but each must now state its authentication boundary explicitly.
task: update deps for security hygiene NVIDIA/garak
garak's dependency update targets recently reported security concerns, tightening the npm-provided dependency restrictions and capturing the latest npm entries in yarn.lock, and overlaps two commits making the same change. The author notes the npm restrictions specifically, so a report-regeneration check after upgrading is the sane verification step.
fix(detectors): stop RefusalOnlyAdversarial over-counting skipped multi-turn attempts NVIDIA/garak
`judge.RefusalOnlyAdversarial.detect` returned `[None] * len(attempt.all_outputs)` when it skipped a non-adversarial multi-turn attempt, while garak's evaluator expects detector results to align with `attempt.outputs`. The mismatch inflated refusal counts on skipped attempts, which means scores you generated from multi-turn runs with this detector may overstate how often the target refused.
fix(detectors): stop FileDetector scoring an unexamined file as a pass NVIDIA/garak
`FileDetector.detect()` yielded `0.0`, the no-hit end of the scale, whenever `_test_file` returned `None`, and `FileIsExecutable` returns that `None` when `python-magic` fails to import. A file garak could not examine was scored as a clean pass; until this lands, treat FileDetector passes with suspicion if your environment lacks python-magic.
FEAT: add ScenarioPresetRegistry owning preset storage and validation microsoft/PyRIT
PyRIT added `pyrit.registry.components.ScenarioPresetRegistry`, a durable storage layer for scenario presets that merges code-registered and stored JSON documents with stored winning and reports the origin per entry. Saving over a registered name raises `ScenarioPresetNameRegisteredError`, so preset naming collisions are now caught at save time.
Action items
