$ the-wire · showcase
PyRIT grows an attack registry, promptfoo tightens JS assertion grading
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
The day's substantive work lands on the injection and grading paths operators rely on: PyRIT is building a class-only attack registry and indexed attack analytics, while promptfoo closes a run of JavaScript assertion grading bugs that could silently change your eval results.
fix(assertions): preserve async expression grading promptfoo/promptfoo
Declaration-prefixed JavaScript assertions could fail when async code used a regular expression after await, because statement boundaries were parsed against the wrong grammar. Grading now validates the same return-inserted body that actually executes, so bare grading objects, line separators, scores, and metadata survive intact; 479 affected tests pass.
fix(assertions): preserve keyword-property grading promptfoo/promptfoo
A companion fix covers declaration-based assertions where division, multiplication, or calls follow keyword-named properties. Acorn's property tokens are normalized while the original assertion text is preserved, keeping boolean, numeric, and GradingResult outcomes unchanged.
fix: handle assertion comments and dataset limits promptfoo/promptfoo
Inline assertions no longer mistake comment semicolons for statement boundaries, and malformed Hugging Face limits are validated before they reach the rows API, with explicit argument precedence and zero-request limits preserved. npm override traversal for known-bad resolution ranges was extended to scoped and version-qualified selectors.
fix: preserve Cloud errors and check npm aliases promptfoo/promptfoo
Cloud permission-denied responses with missing or malformed error data now preserve ConfigPermissionError instead of degrading into a generic failure, and non-403 responses still continue. Known-bad npm release checks now resolve dependency aliases to their target package and range first.
FEAT: Indexed attack analytics microsoft/PyRIT
PyRIT adds indexed attack analytics, alongside a class-only attack registry foundation and nested and multiple conversion marker support, and fixes that pick the latest PyPI release and clean up SQLite on cancellation. garak, meanwhile, now respects parallel_capable when issuing generator requests.
