$ the-wire · showcase
PyRIT calms Rust false positives and fixes attacker-view reporting
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
PyRIT spent the day repairing how its own results are reported: the default Rust scenario stops flagging standard-library paths as hallucinated packages, and scenario-result views stop surfacing orchestration envelopes to operators reviewing attack runs.
FIX: Stop flagging Rust std paths, types and hyphenated crates as hallucinated microsoft/PyRIT
The Rust extraction counted every name followed by :: as a crate, so std modules, type names like HashMap::new and String::from, and crate::/super:: prefixes were all reported as hallucinated packages, and hyphenated registry names never matched their underscore spelling in code. Only the root of a path now counts, and names followed by ::< are skipped, so runs of the default Rust scenario gene...
FIX: hide orchestration envelopes from scenario-result attack views and label adaptive attempts by technique microsoft/PyRIT
Scenario-result attack views no longer display orchestration envelopes, and adaptive attempts are labelled by the technique that produced them rather than by the orchestration wrapper. Operators reading a run's output see the attack that was actually tried, which makes triage and write-ups line up with the technique names in the scenario config.
FIX: Expose filesystem-free paths to seed templates microsoft/PyRIT
Seed templates can now resolve filesystem-free paths, so scenarios that shipped templates assuming an on-disk working directory no longer depend on where you launch the run. Useful if you assemble scenario packages or run inside a container with a stripped-down mount.
FIX: Return the user request from get_request_from_response after a tool call microsoft/PyRIT
get_request_from_response was returning the message immediately before the response, which after a tool call is the tool output rather than the user's request; MemoryInterface._execute_get_request_from_response now returns the user request, fixing the DecodingScorer lookups in the garak Encoding scenario. Results from Encoding runs that involved tool calls should be re-read or re-run before you...
FEAT Add CyberSecEval prompt injection dataset loader (#3419) microsoft/PyRIT
A CyberSecEval prompt-injection dataset loader lands in PyRIT, adding a packaged source for prompt-injection test cases. Two promptfoo dependency bumps, vitest to 5.0.4 and google-auth-library to ^11.3.0, round out a quiet day.
