chore(deps): update dependency promptfoo to v0.121.12

[renovate]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
promptfoo	0.121.2 → 0.121.12

---

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

---

Release Notes

promptfoo/promptfoo (promptfoo)

v0.121.12

Compare Source

Features

• cache: support repeat-aware fetch cache options (#​6844) (c119832)
• cli: add searchable team selector (#​9243) (247eef5)
• cli: import OpenAI Evals dashboard exports (#​9305) (70cebab)
• code-scan: add SARIF output support (#​9161) (4da26e9)
• code-scan: refine SARIF output ergonomics (#​9159) (ea3a655)
• eval: add metrics pills display toggle (#​9253) (eb7ecad)
• eval: support runtime tags (#​9322) (5a09980)
• examples: expand google adk integration (#​9047) (f190033)
• mcp: add response transforms (#​8943) (5d90bb7)
• providers: normalize OpenCode skill usage (#​9250) (8930dad)
• providers: support Agents SDK 0.9 workflows (#​9128) (0505ea1)
• redteam: support context purpose overrides (#​9260) (ca62ec9)
• telemetry: capture node runtime metadata (#​9206) (ec9d308)

Bug Fixes

• app: align table settings info buttons (#​9249) (a3252e9)
• app: improve custom metrics dialog readability (#​9248) (a3eafa4)
• app: sort custom metrics before truncation (#​9246) (74d2f9d)
• cache: include status in JSON parse errors (#​9317) (bb64b2e)
• cli: clearer errors for malformed eval imports (#​9333) (869fd54)
• code-scan: isolate action OIDC token env (#​9309) (30c99e9)
• code-scan: route logs to stderr for structured output (#​9329) (06cce72)
• code-scan: scope OIDC token to scan subprocess (#​9308) (178b57a)
• code-scan: skip unrelated default config startup (#​9230) (335d809)
• deps: avoid incompatible npm release-age config (#​9244) (b8bfb73)
• deps: bump nested ws to a non-vulnerable version (#​9330) (1fe31a4)
• deps: clear protobuf and langsmith alerts (#​9219) (d0995b8)
• deps: remove stray smithy lockfile root dependency (#​9271) (8f34641)
• deps: resolve active advisories (#​9153) (47cd609)
• deps: update anthropic packages (#​9187) (a68ed8b)
• deps: update dependency @​anthropic-ai/sdk to ^0.97.1 (#​9316) (88bc094)
• deps: update dependency @​inquirer/search to v4.1.9 (#​9277) (7a7ae06)
• deps: update dependency @​opentelemetry/exporter-trace-otlp-http to ^0.217.0 (#​9193) (082cb22)
• deps: update dependency ws to v8.20.1 [security] (#​9275) (bac18de)
• deps: update openai packages (#​9189) (f2ceff3)
• dev: use tsx watch for server (#​9301) (f0280e6)
• eval: allow header horizontal scrolling (#​9245) (4e3fb27)
• eval: preserve eval export import parity (#​9310) (ef7f98e)
• fetch: decompress fallback for compressed responses on Node 26 (#​9218) (93f5f1e)
• fetch: decompress pooled undici responses (#​9234) (e193d61)
• guard against empty choices and message=None in swe_runner example (#​9263) (b6ce944)
• isolate loadFunction cache entries (#​9255) (1e3533e)
• mcp: surface tool error results (#​9320) (b600a27)
• openclaw: support protocol negotiation for versions 3 and 4 (#​9254) (4220562)
• providers: aggregate command output deltas (#​9204) (5f89b7c)
• providers: avoid constant-key cache HMAC (#​9152) (2b5db89)
• providers: decompress mTLS HTTP responses on Node 26 (#​9325) (4ff8315)
• providers: execute Anthropic MCP tool results (#​9286) (a42dbd6)
• providers: harden codex app-server approvals and raw evidence (#​9131) (2299c31)
• providers: harden OpenAI Realtime provider after #​9137 (#​9156) (9447d18)
• providers: honor max_tool_calls: 0 for Anthropic MCP (#​9327) (5b804e0)
• providers: map Python worker stderr log levels (#​9306) (bbd683c)
• providers: preserve interleaved command output deltas (#​9210) (3c03fbe)
• providers: refresh xai grok support (#​9237) (58b1c6e)
• providers: support realtime tools in persistent sessions (#​9137) (00f6518)
• providers: tear down Realtime socket on response timeout (#​9324) (d3bb58c)
• redteam: clarify empty custom intent scans (#​8990) (93b5579)
• redteam: improve setup control accessibility (#​9188) (0ea2f86)
• redteam: redact plugin config logs (#​8661) (5ae731b)
• resolve code quality findings (#​9151) (83d80b8)
• server: cap eval table page size (#​9313) (25286aa)
• ui: improve table filter accessibility (#​9166) (cb5dab7)
• ui: wrap static table filters (#​9167) (9fffdca)
• webui: align custom intent count gating (#​9201) (007ae19)
• webui: anchor nav menus to their triggers (#​9154) (cfb474d)
• webui: improve data table keyboard access (#​9171) (c4ce1b4)
• webui: improve eval results table sizing (#​9257) (5269f22)
• webui: restore nav dropdown animations and clean up after #​9154 (#​9169) (d0337d7)

v0.121.11

Compare Source

Features

• quiverai: add Arrow 1.1 models, vectorize endpoint, and GPT Image-2 pipeline (#​9139) (ce2c62d)

Bug Fixes

• redteam: handle MCP target prompt materialization (#​9149) (a050023)
• redteam: keep fresh target type state in sync (#​9145) (770130f)
• redteam: show target type selector immediately (#​9141) (c143fa6)

v0.121.10

Compare Source

Features

• app: humanize per-cell latency in eval results table (#​8998) (509aa54)
• bedrock: add converse mcp support (#​9134) (c1fe183)
• claude-agent-sdk: bump SDK to 0.2.120 and expose new options (#​8946) (350a24c)
• cli: add eval filter range (#​8895) (f049dd3)
• examples: add integration-inspect-osworld wrapper (#​8932) (5d3279a)
• openapi: generate server spec from route DTOs (#​8928) (c92102b)
• output: add junit xml reports (#​9073) (ec8149c)
• providers: add Atlas Cloud provider (#​8980) (90da094)
• providers: add first-class n parameter support for OpenAI chat completions (#​8914) (6e120b1)
• providers: expose OpenAI prompt cache controls (#​9093) (9ec9c1f)
• providers: update mistral support (#​9018) (130f59f)
• xai: refresh provider support (#​9033) (c2c7abf)

Bug Fixes

• api: add DTO validation for core server routes (#​8922) (87f637f)
• api: correct user-visible response regressions (#​8976) (74e7f95)
• api: harden node package contracts (#​9055) (be44d4e)
• api: make modelAudit safeRespond fire-safe under parse failures (#​8977) (524c5e6)
• api: preserve error details and stack traces in shared helpers (#​8975) (ed0b67b)
• api: validate eval route DTOs (#​8924) (84d5c8c)
• api: validate media, blob, user, and trace DTOs (#​8923) (9a313f5)
• api: validate model audit route DTOs (#​8927) (684ae81)
• api: validate provider route DTOs (#​8925) (b761730)
• app: make update banner dismiss button clickable (#​8942) (79a2864)
• assertions: avoid slow XML candidate matching (#​8941) (df029d2)
• assertions: invert not-llm-rubric results (#​8986) (c45831b)
• assertions: normalize shell command arrays in trajectories (#​9060) (a39fac1)
• assertions: support indexed XML paths (#​8970) (bd3969f)
• cache: hash anthropic cache keys (#​8642) (d1f1e5b)
• cache: hash bedrock cache keys (#​8644) (d65fa5e)
• cache: hash vertex and foundry keys (#​8640) (c7b2486)
• cache: stabilize Anthropic cache hashes across runs (#​9124) (8e72bd1)
• cli: avoid hard exits in recoverable flows (#​9118) (d00bddd)
• cli: explain better-sqlite3 ABI mismatches (#​9000) (fc7573a)
• cli: fail closed on modelaudit signal exits (#​9014) (2f29f65)
• code-scan: honor enable-fork-prs (#​8938) (517ec9d)
• codex: isolate default providers by credential (#​8633) (dddd275)
• config: make resolution errors reusable (#​9008) (6b8d3db)
• deps: update dependency ai to ^6.0.168 (#​9007) (95adf3c)
• deps: update dependency fast-xml-parser to ^5.7.1 (#​9010) (72140be)
• deps: update dependency undici to ^7.25.0 (#​9017) (5be6015)
• deps: update openai packages (#​9092) (efe2cd0)
• deps: update opentelemetry (#​9103) (9106ef0)
• eval: delete traces with eval records (#​8973) (a395551)
• eval: harden OpenAI media blob exports (#​8876) (9d4948b)
• eval: honor prompt suggestion counts (#​9105) (3f98e0d)
• eval: honor sharing disable for blob uploads (#​8940) (77fa02a)
• eval: keep package APIs in library mode (#​9111) (3bc3a22)
• eval: preserve filter range on resume (#​8960) (cd16d65)
• eval: stop HuggingFace empty-page loops (#​8939) (56b3c9f)
• google: drop late live messages after resolution (#​9138) (f21272d)
• google: use AIStudioChatProvider for default grading providers (#​9108) (42ab9dd)
• model-audit: fail closed across report flows (#​9050) (f1d0a6c)
• output: strip nested response metadata (#​9038) (ba3ecaa)
• output: strip nested test case metadata (#​9039) (1179413)
• providers: claude-agent-sdk returns main agent result, not first sub-agent result (#​9056) (a694552)
• providers: drop retired groq reasoning model (#​9025) (937e1c9)
• providers: expose OpenAI prompt cache controls (#​9093) (7fbbdca)
• providers: honor disabled tools in model providers (#​9045) (cfdc9f9)
• providers: honor passthrough for OpenAI embeddings (#​9107) (3eeccdb)
• providers: improve openai pricing accuracy (#​9049) (6cb5be9)
• providers: refresh WatsonX model metadata (#​9015) (355fbaf)
• providers: separate OpenCode fork cache keys (#​9102) (9851681)
• providers: structured 429 classification in Azure assistants (#​8896) (d2c64d2)
• providers: support newer bedrock chat models (#​9024) (f3be26f)
• providers: unify coding agent working dirs (#​9043) (bfcac5b)
• providers: use agent_reference for Azure AI Foundry agents (#​8987) (989a17e)
• redteam: honor coding-agent artifact vars (#​9133) (0187a8c)
• redteam: improve SQL injection grading to distinguish from RBAC issues (#​7284) (2665cac)
• redteam: keep results toolbar visible (#​9125) (1a986ee)
• redteam: make email validation failures recoverable (#​9012) (698b9ae)
• redteam: refine report table layout (#​9127) (3e5371f)
• redteam: validate redteam route DTOs (#​8926) (1b8c4fb)
• server: honor remote generation disable for hosted helpers (#​8967) (61baa48)
• test: prevent test-hygiene race with parallel fixture writers (#​8948) (549e2b7)
• tracing: skip orphan otlp trace rows (#​9132) (a303d36)

Performance Improvements

• providers/http: parallelize createHttpsAgent file reads (#​8956) (108c80b)

Reverts

• feat(providers): add first-class n parameter support for OpenAI chat completions (#​9113) (543b5de)

v0.121.9

Compare Source

Features

• providers: add gpt-5.5 model support (#​8884) (8c5dc92)

Bug Fixes

• cli: align command-line reference with CLI (#​8900) (c4ce0d4)
• deps: update openai packages (544d1c6)
• telemetry: mirror person properties on every event (#​8902) (95ad451)
• prevent report table header icon overlap (#​8880) (ad87060)
• trailing slashes on internal routes, Docusaurus Link for internal hrefs, and test hygiene (#​8903) (0828ff7)

v0.121.8

Compare Source

Features

• claude-agent-sdk: bump to 0.2.116 and add title option (#​8858) (9bca53a)
• providers: add GPT-5.5 OpenAI support (#​8873) (6488623)

Bug Fixes

• cache: hash azure assistant keys (#​8646) (7264a73)
• deps: update dependency uuid to v14 [security] (#​8864) (3fa2956)
• providers: canonicalize Azure assistant cache keys (#​8883) (c80f431)
• redteam: support remote input materialization (#​8863) (d89382e)

v0.121.7

Compare Source

Features

• add promptfoo codex skills (#​8790) (d794c0d)
• harmbench plugin filter by category (#​8827) (73533f4)
• openai: add gpt-image-2 support (#​8835) (ac5ff4c)
• providers: add google:embedding: AI Studio embeddings (#​8825) (cb67352)
• redteam: add DOCX input wrapper generation (#​8506) (5bba957)
• webui: virtualize shared data tables (#​8537) (ae756c9)

Bug Fixes

• assertions: support Vercel AI SDK tool-call spans (#​8800) (642e925)
• auth: log file auth refresh lifecycle (#​8833) (0da6c6a)
• cache: hash mistral cache keys (#​8643) (869f98d)
• cache: hash moderation cache inputs (#​8641) (2c36415)
• cache: hash replicate cache keys (#​8647) (495e09e)
• cache: hash watsonx cache keys (#​8649) (1461b6f)
• cache: isolate fetch cache by request identity (e0da9dd)
• deps: update DOMPurify to 3.4.1 (#​8844) (29045ed)
• deps: update opentelemetry (#​8851) (4370216)
• eval: scope result ratings to eval (33be399)
• eval: show rendered assertion values instead of raw templates in UI (#​7976) (dde71c9)
• providers: convert OpenCode permissions to v2 PermissionRuleset (#​8808) (335149a)
• providers: honor OpenRouter config overrides and refresh model references (#​8804) (432d1a8)
• redteam: support named custom policy configs (#​8836) (b035d66)
• server: handle large eval table payloads (#​8745) (92d5929)
• server: redact route debug bodies (#​8639) (d160e4c)
• vertex: align VertexEmbeddingProvider constructor with ProviderOptions (#​8843) (f29cfef)

v0.121.5

Compare Source

Features

• providers: add Abliteration provider (b29fa9a)
• providers: add OpenAI Codex app-server provider (#​8578) (a403dd1)
• providers: let anthropic:messages auth via Claude Code session (#​8692) (642af70)
• webui: add system theme selector (#​8538) (2ed5e5b)

Bug Fixes

• app: chart toggle visibility (#​8439) (f9d448d)
• assertions: avoid jsdom in HTML assertions (979aee4)
• correct rouge assertion reason messages when inverse: true (#​8682) (7d64d8d)
• deps: update anthropic packages (#​8670) (0a4a7e0)
• deps: update dependency openai to ^6.34.0 (#​8701) (73f6ba1)
• deps: update dependency parse5 to v8 (#​8710) (759391d)
• deps: update openai packages (#​8672) (1ae735c)
• providers: close stdin on exec provider to prevent child process hanging (#​8686) (9cd7709)
• providers: resolve settings file path for claude-agent-sdk (#​8606) (d575bd1)
• providers: send reasoning config for Azure responses deployments (#​8255) (67437f3)
• redteam: throw immediately on non-OK HTTP responses in redteam discover (#​8677) (770a557)
• server: prevent circular JSON when Bedrock/Anthropic provider is used as llm-rubric judge from web UI (#​8688) (f26e54b)

v0.121.4

Compare Source

Features

• allow per-test opt-out of defaultTest assertions (5e5959e)
• codex: expand Codex SDK eval controls and docs (#​8433) (80c3f7f)
• eval: group serial grading by provider (#​8509) (d289602)
• examples: add traced openai agents python sdk example (#​8354) (6870717)
• http: support structured multipart requests (#​8533) (5bac47c)
• japan fiea plugin (#​8316) (f330ab3)
• matchers: expose grading provider metadata in GradingResult (#​8330) (03cbac6)
• openai: enable chatgpt login via codex (#​8327) (5a9cb96)
• providers: add Gemma 4 provider support (#​8454) (b0667ed)
• providers: add missing Anthropic SDK features and fix apiKeyRequired bug (#​8351) (3060847)
• providers: expand OpenClaw support (#​8589) (93f29ec)
• providers: support Codex local images and harden SDK (929790b)
• providers: support multimodal openai agents input (#​8397) (4065844)
• redteam: add coding-agent redteam plugins (85a0cfe)
• redteam: add FDA medical plugins (#​8456) (30e4ac3)
• redteam: add next coding-agent plugins (5ab3ba4)
• redteam: add teen safety plugins (#​8308) (61aa057)
• redteam: enforce max chars per message (#​8428) (9af7b7d)
• redteam: use Codex login for default text graders (#​8493) (9a6b61b)

Bug Fixes

• app: clarify attack success rate label (#​8386) (d088eac)
• app: clarify attack success rate label (#​8387) (7482eff)
• app: keep select-all checkbox visible (#​8549) (02f9064)
• assertions: apply weights to named scores in assertion results (#​8206) (01da019)
• assertions: handle inverse flag in finish-reason handler (#​8556) (7155f28)
• assertions: normalize javascript function assertion results (#​8377) (a7fafba)
• assertions: normalize negated script assertions (#​8485) (fe62243)
• assertions: preserve nested metric weights (#​8558) (f3c173d)
• assertions: resolve search-rubric web providers (#​8557) (af751aa)
• assertions: skip simulated-user as implicit grader (#​8429) (9bc5fdf)
• auth: honour org flag in login command (#​8430) (8bdb15d)
• bedrock: preserve zero-valued inference config (#​8272) (1cd397e)
• browser: make stealth plugin install explicit (#​8434) (aa72a34)
• cache: cache repeated evals by repeat index ([#​8480](https://redire

✂ Note

PR body was truncated to here.

---

Configuration

📅 Schedule: (in timezone Europe/London)

• Branch creation
  • "before 10am on friday"
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[github-actions]

Caution

[High Risk] Demo health service will be exposed through shared network paths on port 9090

This change creates a new t4g.nano EC2 instance in the public subnet subnet-07b5b1fb2ba02f964, starts a demo health server bound to 0.0.0.0:9090, and attaches the instance to sg-03cf38efd953aa056 and sg-089e5107637083db5. Current state shows sg-089e5107637083db5 allows port 9090 from 10.0.0.0/8, and sg-03cf38efd953aa056 is a customer-facing shared group with many external CIDRs on 443 plus unrestricted egress. Organizational policy says EC2 instances must not be directly reachable from the internet and should not be placed in public subnets.

The same change also creates a target-group attachment that registers the new instance IP on port 9090 into api-health-terraform-example, which is already connected to a load balancer. That will make the demo health service broadly reachable inside the VPC and load-balancer reachable by design, violating segmentation and least-privilege controls. If the instance receives a public IP or any listener/routing is expanded later, the service becomes externally exposed with almost no additional guardrail.
_{View reasoning tree here.}

Warning

[Medium Risk] Elastic IP will expose the production API instance directly and bypass the managed public ingress path

This change associates Elastic IP 13.134.236.98 to the production API EC2 instance, creating a direct public endpoint on production-api-server instead of forcing ingress through the managed ALB path that already exists for the service. The new instance is explicitly configured to listen on 0.0.0.0:9090, so once the EIP is attached it will be reachable directly on the instance network path rather than only through load-balancer controls.

That bypasses the organization’s requirement that EC2 instances must not be directly reachable from the internet and violates the network pattern called out in SEC05-BP01 and REL02-BP01. Even if source CIDRs are partially restricted by security groups, this still creates a separate single-instance public ingress path outside ALB-based inspection and multi-AZ resilience. Traffic sent to the EIP will hit one EC2 host directly, increasing attack surface and reducing availability compared with terminating traffic on the existing managed load balancer.
_{View reasoning tree here.}

Signals

Routine → Multiple infrastructure resources showing unusual infrequent changes at 1-2 events/week for the last 3 months, with some related resources at 1 event/week for the last 5 weeks, which is infrequent compared to typical patterns.
Policies → Multiple infrastructure resources showing unusual policy violations that may need review: an S3 bucket does not have server-side encryption configured and is missing required tags, while a security group allows SSH on port 22 from anywhere (0.0.0.0/0), which is a security risk.

_{Additional Change Details:} Items 188 Edges 330 model|risks_v6 ✨Encryption Key State Risk ✨KMS Key Creation

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Auto-blocked: Routine score (-5) is below minimum (-1)

---

📊 Signals Summary

Routine 🔴 -5

---

🔥 Risks Summary

High 0 · Medium 0 · Low 0

---

💥 Blast Radius

Items 23 · Edges 75

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Auto-blocked: Policy signal (-3) is below threshold (-2); Routine score (-5) is below minimum (-1)

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 0 · Medium 0 · Low 0

---

💥 Blast Radius

Items 5 · Edges 20

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Found 2 high risks requiring review

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 2 · Medium 0 · Low 0

---

💥 Blast Radius

Items 107 · Edges 219

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Found 1 high risk requiring review

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 1 · Medium 1 · Low 0

---

💥 Blast Radius

Items 63 · Edges 135

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Found 2 high risks requiring review

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 2 · Medium 0 · Low 0

---

💥 Blast Radius

Items 93 · Edges 217

---

View full analysis in Overmind ↗

[github-actions]

⛔ Auto-Blocked

---

🔴 Decision

Auto-blocked: Policy signal (-3) is below threshold (-2); Routine score (-5) is below minimum (-1)

---

📊 Signals Summary

Routine 🔴 -5

Policies 🔴 -3

---

🔥 Risks Summary

High 0 · Medium 0 · Low 0

---

💥 Blast Radius

Items 79 · Edges 192

---

View full analysis in Overmind ↗