$ the-wire · showcase
promptfoo enforces body deadlines, PyRIT adds a Cyrillic homoglyph source
By RepoJournal · Filed · About AI Red Teaming · Composed from the cited sources · methodology
promptfoo closed two ways an eval could hang or misreport, and PyRIT gained a deterministic Cyrillic homoglyph table for confusable-based red teaming, so operators of both can stop working around known gaps.
fix(providers): enforce deadlines on buffered HTTP responses promptfoo/promptfoo
A target that returned HTTP 200 headers and then held the body open could occupy an evaluation slot forever, because the request timer stopped at headers before the buffered body was read. Remaining time from the current HTTP attempt now carries into buffered reads, including rate-limit classification, each retry gets its own deadline, and the transport is aborted when time runs out.
fix(eval): end only CLI evals on SIGTERM, after provider drain and OTel flush promptfoo/promptfoo
The provider registry's SIGTERM listener never exits, and any SIGTERM listener suppresses Node's default termination, so a stuck provider call kept promptfoo eval alive until SIGKILL. Only CLI evals now end on SIGTERM, after the provider drain and an OpenTelemetry flush, leaving library and embedding hosts with the drain-only behavior.
fix(assertions): fail throwing file:// and package: JavaScript assertions instead of erroring the row promptfoo/promptfoo
A javascript assertion loaded from file:// or package: that throws used to turn the entire row into an ERROR; it now returns pass: false, score: 0 with reason Custom function threw error, matching inline assertions and the existing Python and Ruby behavior. Load-time failures still throw, treated as configuration mistakes.
FEAT: Add cyrillic_homoglyph option to UnicodeConfusableConverter microsoft/PyRIT
UnicodeConfusableConverter accepts cyrillic_homoglyph as a third source_package value, alongside confusable_homoglyphs and confusables. It swaps Latin a, c, e, o, p, x, y in both cases for Cyrillic look-alikes from a fixed in-code table, so output is deterministic and requires no extra package; other characters pass through unchanged.
MAINT Return Self from WebSearchConfig.from_metadata (P311-01) microsoft/PyRIT
WebSearchConfig.from_metadata now returns typing.Self rather than WebSearchConfig, so a subclass gets its own type back from the type checker; runtime behavior is unchanged. The rest of the day was dependency bumps to @anthropic-ai/claude-agent-sdk v0.3.303 and oxc-parser ^0.155.0, an exhaustive repeat-token dispatch fix, and a chore-level test cleanup.
