Skip to content

[triage:core-secengine-03-missing-tests-for-boundary-deny-and-ask] evaluate_security_boundary has no test for its block/ask (fail-closed) outcomes #116

Description

@ebursztein

Imported from Capsem triage report core-secengine-03-missing-tests-for-boundary-deny-and-ask.md.

  • Severity: medium
  • Category: missing-test
  • Area: capsem-core/security_engine
  • Location: crates/capsem-core/src/security_engine/mod.rs:1331-1369 (function under test); crates/capsem-core/src/security_engine/tests.rs:1815 (only existing call)
  • Confidence: verified

Summary

evaluate_security_boundary is the single live enforcement entrypoint used by the HTTP, MCP, DNS,
and model boundaries (call sites: net/mitm_proxy/mod.rs:1616,1755,1967,2513,
net/mitm_proxy/mcp_frame.rs:766, net/dns/server.rs:337, mcp/builtin_tools.rs:808). The only
test that calls it asserts a single Allow outcome with an empty plugin policy. None of its
deny-relevant outcomes are covered:

  • enforcement = Block from a matched Block rule
  • enforcement = Ask from a matched Ask rule
  • enforcement = Block from a preprocess/postprocess plugin (mode = "block")
  • enforcement = Ask from a plugin (mode = "ask")
  • the post-stage upgrade branches at mod.rs:1351-1362

Evidence

Grep of the test file shows evaluate_security_boundary is invoked exactly once (tests.rs:1815),
inside default_rules_do_not_override_specific_enforcement_decisions, which asserts
enforcement.action == Allow (tests.rs:1815-1828). Every other SecurityEnforcementAction::Block
/ ::Ask assertion in the file is against emit_matching_security_rules_with_decision (the ledger
path) or against SecurityEnforcementDecision::with_ask_resolution — not against
evaluate_security_boundary. The block/ask branches of evaluate_security_boundary
(mod.rs:1351-1362) have zero coverage.

This coverage gap is what allowed the postprocess-Ask downgrade defect (see
core-secengine-01-postprocess-ask-downgraded-to-allow.md, mod.rs:1359-1362) to exist undetected:
the missing branch is on exactly the untested path.

Impact

The product's primary allow/deny gate has no regression protection on its deny and ask outcomes. A
future refactor (or the existing bug #1) can flip a Block/Ask outcome to Allow on the HTTP/MCP/DNS/
model rail with all tests still green. For a security boundary, the fail-closed paths are the ones
that most need locking down.

Suggested fix

Add direct tests for evaluate_security_boundary covering, at minimum:

  1. a matched block rule → enforcement.action == Block;
  2. a matched ask rule → enforcement.action == Ask;
  3. a preprocess plugin with mode = "block" (no matching rule) → Block;
  4. a postprocess plugin with mode = "ask"Ask (this one fails today, per bug Bump astro from 5.14.1 to 5.14.4 in /site in the npm_and_yarn group across 1 directory #1);
  5. a postprocess plugin with mode = "block"Block.
    Assert both enforcement.action and event.decision.effective in each case so the enforced action
    and the merged decision state are pinned together.

Triage

Confirmed from the local reviewed report in /Users/elie/git/capsem/tmp/bugs/core-secengine-03-missing-tests-for-boundary-deny-and-ask.md. Track implementation in the triage sprint; add regression coverage before fixing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions