feat(plugins): Add mandatory callback tier for governance and audit hooks

[sunilp]

Problem

ADK's plugin callback semantics allow short-circuiting: the first plugin returning a non-None value bypasses all subsequent plugins AND agent callbacks. This is by design for general-purpose plugins, but it creates a gap for governance and audit use cases — these hooks must always execute, regardless of what other plugins do.

This gap was identified during the multi-party governance discussion in #4764 and affects all governance implementations currently being built for ADK.

Evidence from existing issues

This isn't theoretical — the short-circuit behavior is already causing problems:

• LlmAgent output_key is not written to session state when before_agent_callback short-circuits execution #4837 — before_agent_callback short-circuit doesn't update output_key in session state, causing silent backend staleness. A mandatory tier would ensure state-management callbacks always run.
• Unbalanced tool lifecycle callbacks for hallucinated tools cause TraceManager stack corruption in plugins #4775 — Unbalanced tool lifecycle callbacks for hallucinated tools corrupt TraceManager spans. Mandatory after-callbacks would guarantee cleanup runs even when tool execution is skipped.
• Collaboration: Governance callbacks for ADK agent lifecycle #4764 — Governance callbacks discussion where @sunilp, @aeoess, @imran-siddique, and @razashariff independently identified the enforcement gap.

Who needs this

Three governance implementations are being built for ADK, all affected:

• GovernancePlugin — PolicyEvaluator protocol with tool filtering, delegation scope, audit trails (@sunilp)
• Agent Passport System — cryptographic 3-signature policy chain with evaluateIntent() enforcement (@aeoess)
• adk-agentmesh — YAML/OPA/Cedar policy backends for ADK (@imran-siddique)

All three use before_tool_callback and before_agent_callback as enforcement points. All three can be silently bypassed by an earlier plugin returning non-None.

Proposal

Add a two-phase callback execution model:

Phase 1: Standard plugins (current behavior — first non-None return short-circuits)
Phase 2: Mandatory plugins (always execute — governance, audit, telemetry)

If a Phase 2 plugin denies an action, its denial overrides Phase 1's result. This ensures governance has the final say.

Suggested API

class BasePlugin:
    # Existing behavior
    mandatory: bool = False

    # When True:
    # - Callbacks always execute, even if a prior plugin returned non-None
    # - Denial from a mandatory plugin overrides earlier allow decisions
    # - Audit/telemetry callbacks are guaranteed to fire

Why two phases instead of priority ordering

Priority ordering (e.g., priority=100) is fragile — plugin authors compete over priority values, and there's no guarantee governance wins. The two-phase model is simpler: standard plugins handle normal logic, mandatory plugins handle governance. No priority conflicts.

Impact

Without mandatory tier	With mandatory tier
Governance hooks can be bypassed	Governance always executes
Audit trail has gaps	Audit trail is complete
Policy enforcement is advisory	Policy enforcement is real
Cannot satisfy regulatory requirements	Auditors can rely on the layer

Compatibility

Fully additive. Existing plugins with mandatory = False (the default) behave exactly as today. Only plugins explicitly opting into mandatory execution get the new guarantee. No breaking changes.

Related

• Collaboration: Governance callbacks for ADK agent lifecycle #4764 — Governance callbacks discussion
• LlmAgent output_key is not written to session state when before_agent_callback short-circuits execution #4837 — Short-circuit doesn't update session state (symptom of the same design gap)
• Unbalanced tool lifecycle callbacks for hallucinated tools cause TraceManager stack corruption in plugins #4775 — Unbalanced callbacks cause trace corruption (mandatory after-callbacks would fix)
• Feature: GovernancePlugin — policy enforcement, threat detection, and audit trails #4543, Proposal: Audit Trail and Trust Scoring Callbacks for ADK #4517 — Earlier governance proposals (closed)
• google/adk-python-community#102 — GovernancePlugin PR

cc @aeoess @imran-siddique @razashariff @evekhm @gautamvarmadatla

[raye-deng]

The mandatory callback tier is a well-motivated proposal. The evidence from #4775 is particularly compelling — when hallucinated tools bypass before_tool_callback entirely, any governance plugin that assumes balanced lifecycle hooks will silently corrupt its own state.

This points to a broader architectural insight: hallucinated outputs (whether tool names, package imports, or API calls) break systems that assume structural validity. The TraceManager stack corruption in #4775 is a runtime manifestation, but the same class of failure happens at build time — AI agents generate imports for packages that don't exist on any registry, and the code compiles and passes tests because the test suite mocks the same nonexistent dependencies.

The two-phase execution model you propose (standard plugins with short-circuit + mandatory plugins that always run) maps well to what we've seen work in CI pipelines: you want fast-exit optimizations for most checks, but certain validations — hallucinated dependency detection, deprecated API surface checks, cross-file consistency — must run unconditionally regardless of what earlier checks found.

For teams that want this kind of mandatory validation at the code/PR level rather than the runtime/plugin level: npx @opencodereview/cli scan . --sla L1 — it runs deterministic checks against actual registries and API surfaces, designed to catch exactly the class of hallucination failures that motivated #4775.

[aeoess]

@sunilp — strong +1 for the mandatory callback tier. The evidence from #4837 and #4775 makes this concrete, not theoretical.

One architectural note on why the two-phase model is the right abstraction:

The fundamental issue is that governance hooks have different failure semantics than application hooks. A recommendation plugin can safely short-circuit — if it returns a result, later plugins just miss an optimization opportunity. A governance plugin that gets skipped means a policy violation goes undetected. These are categorically different failure modes and they need categorically different execution guarantees.

APS addresses this at a different layer — the ProxyGateway is both judge and executor. The agent submits a request; the gateway validates, executes, and signs the receipt. The agent never touches the tool directly, so governance cannot be bypassed by plugin ordering. But that only works when you control the execution boundary. For ADK, where plugins are composed by the developer, the mandatory tier is the right solution.

On the mandatory: bool API — clean and sufficient. One suggestion: consider adding a phase field alongside or instead of the boolean, to allow future phases (e.g., standard, mandatory, terminal) without breaking the API. But that is a minor point — shipping mandatory: bool now and extending later is fine.

We just shipped Module 36A (Data Source Registration & Access Receipts) which adds data-terms enforcement to the gateway pipeline. The terms compliance check uses exactly this two-tier pattern: deterministic checks (revocation, expiry, excluded agents) are hard blocks; purpose-compliance checks (self-declared intent) are advisory audit entries. This hard/advisory split maps directly to your Phase 1 / Phase 2 model.

894 tests, 245 suites, 36 modules. The mandatory execution guarantee matters — we have been building around the need for it.

[aeoess]

@imran-siddique — on PolicyDecision schema alignment:

APS PolicyDecision fields:

verdict: "permit" | "deny" | "narrow"
reason: string
principlesEvaluated: PrincipleEvaluation[]   // which floor rules triggered
evaluatorId: string                           // who evaluated
evaluatorPublicKey: string                    // Ed25519 key of evaluator
floorVersion: string                          // which policy version
evaluatedAt: string                           // ISO timestamp
expiresAt: string                             // decision TTL
signature: string                             // Ed25519 by evaluator
auditFindings?: PrincipleEvaluation[]         // graduated enforcement
warnings?: PrincipleEvaluation[]              // advisory-only violations

The core fields map cleanly to your schema:

• verdict → your verdict (we use permit/deny/narrow, you use allow/deny/escalate — narrow and escalate are semantically similar)
• reason → your reason
• principlesEvaluated[].principleId → your matched_rule
• evaluatedAt → your timestamp
• auditFindings + warnings → no equivalent in your schema, but these carry graduated enforcement detail

The structural difference: APS wraps the decision in a cryptographic signature. The gateway signs the PolicyDecision as part of a 3-artifact chain (ActionIntent → PolicyDecision → PolicyReceipt). This makes the evaluation independently verifiable — a third party can prove the policy engine ran and what it concluded without trusting the agent.

For interop, I would propose aligning on a minimal common interface:

class PolicyDecision:
    verdict: str          # allow/deny/escalate (your naming is fine)
    reason: str
    matched_rule: str     # or matched_rules: list[str]
    timestamp: datetime
    metadata: dict        # extensible — APS puts signature + principles here

APS can consume this as input and wrap it with the cryptographic layer. Your GovernancePlugin can produce it. Any backend slots in. The metadata dict is where protocol-specific fields live without polluting the common interface.

Happy to align. The goal is the same — governance that actually runs, with receipts that prove it ran.

[adk-bot]

🚨 Automated Spam Detection Alert 🚨
@maintainers, a suspected spam comment was detected in this thread.

Reason:

@raye-deng's comment promotes a 3rd-party product/CLI tool: `@opencodereview/cli`

[sunilp]

@aeoess — good framing on the failure semantics distinction. That's the clearest way to explain why a boolean tier matters more than priority ordering: governance hooks have categorically different failure modes than application hooks. A skipped recommendation is a missed optimization; a skipped policy check is an undetected violation. Different failure modes, different execution guarantees.

On phase vs mandatory: bool — I'd ship the boolean first and keep the door open. A phase enum is the right evolution if we need a third tier (e.g., terminal for cleanup/telemetry that runs even after governance denial), but adding it now without a concrete use case would be premature. The boolean covers the two cases that exist today: standard (short-circuit OK) and mandatory (must always run).

On the PolicyDecision schema — the minimal common interface looks right:

class PolicyDecision:
    verdict: str          # allow/deny/escalate
    reason: str
    matched_rule: str
    timestamp: datetime
    metadata: dict        # protocol-specific (APS signatures, OPA traces, etc.)

This works for GovernancePlugin as the production interface. APS wraps it with the cryptographic layer, adk-agentmesh maps it to OPA/Cedar outputs, and the metadata dict keeps protocol-specific fields out of the common surface. The naming (allow/deny/escalate) is fine — escalate and APS's narrow are semantically close enough that mapping between them is trivial.

One addition worth considering: a confidence or evaluation_method field to distinguish deterministic checks (revocation, expiry) from probabilistic ones (LLM-based policy evaluation). This maps to the hard/advisory split you described in Module 36A, and would let consumers decide how to weight the decision. But that can come later — the minimal interface above is enough to ship.

[aeoess]

@sunilp — agreed on shipping the boolean first and keeping the door open for a phase enum. The terminal tier (cleanup/telemetry that runs even after governance denial) is a real future need, but adding it without a concrete use case would be speculative.

The PolicyDecision schema you laid out is the right minimal surface:

class PolicyDecision:
    verdict: str          # allow/deny/escalate
    reason: str
    matched_rule: str
    timestamp: datetime
    metadata: dict        # protocol-specific

APS wraps this with the cryptographic proof chain in metadata (intent signature, evaluation signature, delegation chain hash). The mapping between escalate and APS's narrow is straightforward — both mean "not a binary answer, needs further scoping."

On the confidence / evaluation_method field: this maps directly to what we shipped in Module 36A (Data Source Registration). We distinguish between deterministic checks (revocation status, expiry, scope membership — reproducible by any engine) and model_dependent checks (LLM-based policy evaluation, reputation scoring — reproducible only by the originating engine). The cross-engine verification work in kanoniv/agent-auth#2 confirmed this distinction matters: four engines evaluating the same borderline scenario produced four different trust verdicts but identical structural verdicts. The evaluation_method field is what makes that divergence classifiable rather than confusing.

Worth adding to the minimal interface as a v2 field — it lets consumers decide whether to treat a deny as authoritative or advisory based on how the verdict was computed.