Add runtime policy, revocation, and audit correlation for generated tools#2547
Open
vaddisrinivas wants to merge 1 commit into
Open
Conversation
Contributor
📝 WalkthroughWalkthroughThis PR implements runtime policy enforcement and audit correlation for generated tools. It adds configurable provider/capability revocation, risk-based action mappings, and decision variants (Allow, RequireApproval, Deny) in the policy layer; refactors tool execution to extract denial/approval reasons; and extends audit logging to correlate policy decisions and approval IDs with tool execution events. ChangesGenerated Tool Policy and Audit
Sequence DiagramsequenceDiagram
participant Agent as Agent::execute_tool_call
participant Policy as GeneratedToolRuntimePolicy
participant Decision as ToolPolicyDecision
participant Audit as AuditLogger
Agent->>Policy: check(request + generated_tool context)
Policy->>Decision: Allow / RequireApproval / Deny
Decision->>Agent: decision with optional reason
Agent->>Agent: blocking_reason() → check denial/approval
alt blocked or requires approval
Agent->>Audit: log_generated_tool_event(provider, capability, policy_decision, approval_id)
else allowed
Agent->>Agent: proceed with execution
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
coderabbitai
Bot
added
feature
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/openhuman/agent/tool_policy.rs (1)
236-310: ⚡ Quick winAdd branch-level debug tracing for runtime policy decisions.
At Lines 236-310, generated-tool policy evaluation has multiple enforcement branches (revoked provider/capability, provider/capability/risk mappings, default allow) but emits no local debug/trace diagnostics, which makes runtime policy outcomes hard to audit during incidents.
♻️ Suggested patch
impl GeneratedToolRuntimePolicy { @@ fn action_for( &self, tool_name: &str, context: &GeneratedToolRuntimeContext, ) -> (RuntimeToolPolicyAction, String) { @@ - if let Some(action) = self.config.risk_actions.get(&context.risk) { + if let Some(action) = self.config.risk_actions.get(&context.risk) { return ( *action, format!("risk `{:?}` matched runtime policy", context.risk), ); } ( RuntimeToolPolicyAction::Allow, format!("tool `{tool_name}` allowed"), ) } } @@ async fn check(&self, request: &ToolPolicyRequest) -> ToolPolicyDecision { if !self.config.enabled { + tracing::trace!( + policy = self.name(), + tool = request.tool_name.as_str(), + "[rpc] generated tool runtime policy disabled; allowing" + ); return ToolPolicyDecision::Allow; } let Some(context) = request.generated_tool.as_ref() else { + tracing::trace!( + policy = self.name(), + tool = request.tool_name.as_str(), + "[rpc] generated tool context missing; allowing" + ); return ToolPolicyDecision::Allow; }; let (action, reason) = self.action_for(&request.tool_name, context); + tracing::debug!( + policy = self.name(), + tool = request.tool_name.as_str(), + provider_id = context.provider_id.as_str(), + capability_id = context.capability_id.as_str(), + risk = ?context.risk, + action = ?action, + reason = reason.as_str(), + "[rpc] generated tool runtime policy decision" + ); match action { RuntimeToolPolicyAction::Allow => ToolPolicyDecision::Allow, RuntimeToolPolicyAction::RequireApproval => { ToolPolicyDecision::require_approval(reason) } RuntimeToolPolicyAction::Deny => ToolPolicyDecision::deny(reason), } } }As per coding guidelines: "Use
log/tracingatdebugortracelevel on ... state transitions, and any branch that is hard to infer from tests alone."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/agent/tool_policy.rs` around lines 236 - 310, The action_for/check flow in GeneratedToolRuntimePolicy lacks branch-level debug tracing; add debug or trace logs inside GeneratedToolRuntimePolicy::action_for for each branch (revoked provider, revoked capability, capability_actions hit, provider_actions hit, risk_actions hit, and default allow) emitting the chosen RuntimeToolPolicyAction, the formatted reason, tool_name, context.provider_id, context.capability_id, and context.risk as appropriate, and also log the final decision in GeneratedToolRuntimePolicy::check right before returning the ToolPolicyDecision (use ToolPolicyDecision::require_approval / deny / Allow paths); reference the action_for function, the RuntimeToolPolicyAction enum, and the check method to locate insertion points.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/agent/harness/session/turn.rs`:
- Around line 1202-1205: The policy check is using ToolPolicyRequest::new which
leaves generated_tool as None so generated-tool runtime policy rules never run;
before calling self.tool_policy.check(&policy_request).await, populate the
request's generated_tool context (e.g., via the request builder/constructor that
accepts generated tool metadata or by setting the generated_tool field on the
ToolPolicyRequest you created) so the generated-tool runtime policy (revoked
provider/capability/risk action mappings) is available to the check; update the
code path around ToolPolicyRequest::new and the call site of
self.tool_policy.check to pass the generated tool context into the request.
---
Nitpick comments:
In `@src/openhuman/agent/tool_policy.rs`:
- Around line 236-310: The action_for/check flow in GeneratedToolRuntimePolicy
lacks branch-level debug tracing; add debug or trace logs inside
GeneratedToolRuntimePolicy::action_for for each branch (revoked provider,
revoked capability, capability_actions hit, provider_actions hit, risk_actions
hit, and default allow) emitting the chosen RuntimeToolPolicyAction, the
formatted reason, tool_name, context.provider_id, context.capability_id, and
context.risk as appropriate, and also log the final decision in
GeneratedToolRuntimePolicy::check right before returning the ToolPolicyDecision
(use ToolPolicyDecision::require_approval / deny / Allow paths); reference the
action_for function, the RuntimeToolPolicyAction enum, and the check method to
locate insertion points.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e9224089-8318-4a32-ba09-0fbac4e981b7
📒 Files selected for processing (3)
src/openhuman/agent/harness/session/turn.rssrc/openhuman/agent/tool_policy.rssrc/openhuman/security/audit.rs
Comment on lines
1202
to
+1205
| let policy_request = | ||
| ToolPolicyRequest::new(call.name.clone(), call.arguments.clone(), context); | ||
| if let ToolPolicyDecision::Deny { reason } = | ||
| self.tool_policy.check(&policy_request).await | ||
| { | ||
| let policy_decision = self.tool_policy.check(&policy_request).await; | ||
| if let Some(reason) = policy_decision.blocking_reason() { |
Contributor
There was a problem hiding this comment.
Generated-tool runtime policy context is not wired into this policy check.
At Line 1203, ToolPolicyRequest::new(...) is used without attaching generated-tool context before self.tool_policy.check(...). Since ToolPolicyRequest::new initializes generated_tool to None, generated runtime policy rules (revoked provider/capability/risk action mapping) cannot execute on this path.
🐛 Suggested direction
- let policy_request =
- ToolPolicyRequest::new(call.name.clone(), call.arguments.clone(), context);
+ let mut policy_request =
+ ToolPolicyRequest::new(call.name.clone(), call.arguments.clone(), context);
+ // Populate from parsed generated-tool metadata when available.
+ if let Some(generated_ctx) = /* extract GeneratedToolRuntimeContext from call/tool metadata */ {
+ policy_request = policy_request.with_generated_tool_context(generated_ctx);
+ }
let policy_decision = self.tool_policy.check(&policy_request).await;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/openhuman/agent/harness/session/turn.rs` around lines 1202 - 1205, The
policy check is using ToolPolicyRequest::new which leaves generated_tool as None
so generated-tool runtime policy rules never run; before calling
self.tool_policy.check(&policy_request).await, populate the request's
generated_tool context (e.g., via the request builder/constructor that accepts
generated tool metadata or by setting the generated_tool field on the
ToolPolicyRequest you created) so the generated-tool runtime policy (revoked
provider/capability/risk action mappings) is available to the check; update the
code path around ToolPolicyRequest::new and the call site of
self.tool_policy.check to pass the generated tool context into the request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem
Solution
ToolPolicyRequestwith optionalGeneratedToolRuntimeContext.ToolPolicyDecision::RequireApprovaland a sharedblocking_reasonhelper so the existing session runner blocks non-allow decisions.GeneratedToolRuntimePolicyand structured generated-tool audit helpers while preserving the default allow-all policy.Submission Checklist
## Related: N/A, no coverage-matrix feature ID applies.docs/RELEASE-MANUAL-SMOKE.md): N/A, no release-cut UI/runtime smoke surface changed.Closes #NNNin the## Relatedsection.Impact
Related
AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
codex/oh-2543-generated-tool-runtime-policy4b144df2e542c25190a4853a18a650d0c97fdd8bValidation Run
pnpm --filter openhuman-app format:check: N/A, Rust-only backend policy/audit change.pnpm typecheck: N/A, Rust-only backend policy/audit change.cargo test --manifest-path Cargo.toml generated_runtime_policy --lib;cargo test --manifest-path Cargo.toml audit_log_generated_tool --libcargo fmt --manifest-path Cargo.toml --all --check;git diff --checknode scripts/check-pr-checklist.mjs /tmp/openhuman-pr-body-2543.mdValidation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
Parity Contract
Duplicate / Superseded PR Handling
vaddisrinivas:codex/oh-2543-generated-tool-runtime-policy.Summary by CodeRabbit