fix(deps): update dependency @vercel/agent-eval to ^1.4.0

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
@vercel/agent-eval	^1.3.1 → ^1.4.0

---

Release Notes

vercel-labs/agent-eval (@​vercel/agent-eval)

v1.4.0

Compare Source

Minor Changes

• #​172 4f8732c Thanks @​gaojude! - Add expect(transcript).toContainText(needle) — a deterministic, judge-free EVAL.ts matcher over the materialized transcript. needle is an exact substring or a RegExp (use /…/i for case-insensitive). Built for .not ("the agent never reached for X"): absence checks no longer need a judge run or manual readFileSync(transcriptPath()). Misuse (wrong subject, empty needle, empty-matching regex) and a missing/empty transcript throw instead of returning a failed verdict, so .not can never invert them into a silent pass.

Patch Changes

• #​175 f661766 Thanks @​molebox! - Verify and repair the shell tool for native-default Codex runs. Codex CLI >= 0.144.0 (published 2026-07-09) exposes no shell/exec tool to the model when config.toml uses a custom model_provider (e.g. the AI Gateway) and omits the model key — exactly what native-default runs write. The model still answers, but it cannot run commands, read files, or use installed skills, and it sometimes fabricates command output instead of reporting the missing tool. run.mjs now pre-verifies native-default runs with a fabrication-proof shell canary (a command_execution item must carry a random nonce), repairs by re-stating the CLI's own resolved default model as an explicit top-level model key in the profile config, re-verifies, and fails loudly if the tool is still unavailable — preserving the canary's captured output on the failure result for triage. The verified outcome is memoized per sandbox (~/.codex/agent-eval-canary.json) so judge assertions that re-invoke the runner do not pay repeat canary calls. The repair is recorded as an optional modelRepair field propagated through RunnerResult → AgentRunResult → EvalRunResult, so persisted results show which runs needed it (and repairs dropping to zero signals the upstream fix).

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[socket-security]

Dependency limit exceeded — report not shown.

This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report.

Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard.

Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account.