$ the-wire · showcase
Anthropic SDK tracing collides with promptfoo spans, and PyRIT adds a new attack technique
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
promptfoo's Anthropic dependency bump changes the shape of stored traces, while PyRIT ships a new context-wrapping attack and a fix for a Raku scorer that was scoring hallucinations as real.
fix(deps): update anthropic packages promptfoo/promptfoo
Anthropic SDK 0.134+ emits its own tracing spans alongside promptfoo's generation span, so stored traces gain extra spans and existing span-count assertions begin to fail. The fix disables SDK instrumentation on promptfoo's Anthropic client while keeping promptfoo's span, usage, error status and trace linkage, and adds `speed: null` to the four typed fixtures now that `Usage.speed` is required.
FEAT: Add context_wrap attack technique in extra.py microsoft/PyRIT
PyRIT registers `context_wrap` in `pyrit/setup/initializers/techniques/extra.py`, composing `PromptTemplateConverter` and `SearchReplaceConverter` on `PromptSendingAttack` via `AttackConverterConfig`, with both converters restricted to `prompt_data_types_to_apply=["text"]`. The request side asks the target to wrap each sentence in `<seg>` tags, so you can run the technique as-is against any tex...
FIX: Strip Raku module adverbs before the registry lookup in PackageHallucinationScorer microsoft/PyRIT
`PackageHallucinationScorer` with `PackageEcosystem.RAKU` kept the first module adverb as part of the package name, turning `use JSON::Fast:ver<0.19>;` into `JSON::Fast:ver`, which is in no registry, so replies using a real module scored `True`. The scorer now separates the name from `:ver`, `:auth` and `:api` adverbs and sends only the name to the registry lookup, which changes your Raku hallu...
FEAT: Export Operation findings to Word and Markdown microsoft/PyRIT
Findings from a PyRIT operation can now be exported to Word and Markdown, which removes the manual transcription step between an operation run and the report your stakeholders read.
docs(site): clarify security reporting boundaries (#12010) promptfoo/promptfoo
The docs now spell out the boundaries for reporting security issues in promptfoo itself. Also quiet: google-auth-library to 11.3.1 and gcp-metadata to 9.2.1 with no runtime or declaration changes, PyRIT dropping checks for its retired AML deployment, and a ten-package minor-and-patch bump in the PyRIT frontend.
