🔍 Agentic Workflow Audit Report - December 26, 2025

[github-actions[bot]]

Audit Summary

• Period: Last 24 hours (December 25-26, 2025)
• Runs Analyzed: 68
• Workflows Active: 35+ workflows
• Success Rate: 64.7% (44 success, 24 failures)
• Total Token Usage: 17,845,524 tokens
• Total Cost: $5.79
• Issues Found: 1,384 errors/warnings detected, 4 missing tool reports, 5 MCP failures

---

📊 Trend Analysis

Workflow Health Trends

Analysis: Success rate has been fluctuating around 45-65% over the past 2 days. December 25 saw 20 successful runs and 11 failures (64.5%), while December 26 had 24 successes and 13 failures (64.9%). The consistency suggests systemic issues rather than random failures.

Token Usage & Cost Trends

Analysis: Average daily token usage is 8.9M tokens at an average daily cost of $2.90. The 7-day moving average shows stable token consumption patterns. Total cost over the analysis period: $5.79.

---

🚨 Critical Issues

1. MCP Server Initialization Errors (High Priority)

Issue: MCP clients failing to start with "Invalid URL" and missing module errors.

Error Type	Count	Affected Workflows
Invalid URL	36	Smoke Copilot No Firewall, Smoke Copilot Playwright, Smoke Copilot Safe Inputs
Cannot find module './read_buffer.cjs'	34	Multiple Copilot workflows
MCP error -32000: Connection closed	30+	Multiple workflows

Impact: These errors prevent MCP servers (safeinputs, safeoutputs) from initializing properly, causing workflow failures.

Root Cause: Recent changes to MCP HTTP transport configuration may have introduced bugs (see commit 2dd6edb).

2. MCP Server Failures

Server Name	Failure Count	Workflows Affected
safeoutputs	5	Smoke Claude

Impact: The safeoutputs MCP server is consistently failing in Smoke Claude workflows, preventing proper output handling.

3. Permission Denied Warnings (Medium Priority)

Issue: 90 instances of "Permission denied and could not request permission from user" warnings.

Affected Workflows (10 workflows):

• AI Moderator
• Daily Malicious Code Scan Agent
• Issue Monster
• Smoke Copilot variants (No Firewall, Playwright, Safe Inputs)
• Spec-Kit Execute
• The Great Escapi
• Tidy

Impact: Workflows cannot perform certain operations due to permission restrictions in the sandbox environment.

---

🔧 Missing Tools

Tool Name	Request Count	Workflows	Reason
safeinputs-gh	2	Smoke Codex, Smoke Copilot Safe Inputs	Required to test GitHub CLI safe-input access
File system write permissions	1	Spec-Kit Execute	Cannot create directories/files needed for spec-kit features
make, go, golangci-lint, npm	1	Tidy	Build and linting tools not available in execution environment

Recommendation: These are legitimate tool requests. Consider:

1. Adding safeinputs-gh tool to the MCP server
2. Reviewing file system permissions for Spec-Kit Execute
3. Ensuring build tools are available for the Tidy workflow

---

⚠️ Warnings & Non-Critical Issues

Safe-outputs File Not Found (76 occurrences)

• Workflows: Smoke Codex, Smoke Codex Firewall
• Impact: Low - workflows complete successfully despite warnings

Squid Proxy Warnings (53 occurrences)

• Message: "HTTP requires the use of Via"
• Impact: Low - configuration warnings that don't affect functionality

Core is Not Defined (18 occurrences)

• Workflows: Changeset Generator, Smoke Codex variants, Smoke Copilot variants
• Impact: Medium - JavaScript runtime errors that may affect workflow behavior

---

📈 Performance Metrics

• Average Token Usage: 262,434 tokens per run
• Average Cost per Run: $0.085
• Average Turns: 3.7 turns per run
• Highest Cost Workflow: Multiple smoke test workflows running comprehensive tests

---

Affected Workflows

Top workflows with issues:

1. Smoke Test Workflows (Copilot, Claude, Codex variants) - MCP initialization errors
2. AI Moderator - Permission denied warnings
3. Daily Malicious Code Scan Agent - Permission denied warnings
4. Issue Monster - Permission denied warnings
5. Tidy - Missing build tools

---

📋 Recommendations

Immediate Actions

1. Fix MCP HTTP Transport Issues ⚡

   • Investigate commit 2dd6edb changes to mcp_http_transport.cjs and safe_inputs_mcp_server_http.cjs
   • Fix "Invalid URL" errors in MCP client initialization
   • Resolve missing ./read_buffer.cjs module issue

2. Address safeoutputs MCP Failures ⚡

   • Debug why safeoutputs server fails in Smoke Claude workflow
   • Check server startup logs and connection handling

3. Review Sandbox Permissions 🔒

   • Evaluate which operations are being blocked by permission restrictions
   • Determine if these restrictions are intentional or if workflows need permission adjustments

Short-term Improvements

4. Add Missing Tools 🛠️

   • Implement safeinputs-gh tool for GitHub CLI testing
   • Ensure build tools (make, go, golangci-lint, npm) are available for Tidy workflow
   • Review file system permissions for Spec-Kit Execute

5. Fix JavaScript Runtime Errors 🐛

   • Resolve "core is not defined" errors in Codex and Copilot workflows
   • Review JavaScript module dependencies

Long-term Monitoring

6. Track Success Rate Trends 📊

   • Current 64.7% success rate indicates room for improvement
   • Set target of 85%+ success rate for stable workflows

7. Optimize Token Usage 💰

   • Review high-token workflows for optimization opportunities
   • Current average of 262K tokens/run is reasonable for complex agentic workflows

---

Historical Context

This is the first comprehensive audit using the gh-aw MCP server analysis tools. Key findings have been stored in /tmp/gh-aw/cache-memory/audits/2025-12-26.json for future trend analysis.

Baseline metrics established:

• Success rate: 64.7%
• Daily tokens: ~8.9M
• Daily cost: ~$2.90
• Most common errors: MCP initialization (36), missing modules (34), permission denied (90)

---

Next Steps

• Investigate and fix MCP HTTP transport issues (Priority: High)
• Debug safeoutputs server failures in Smoke Claude
• Review permission model for affected workflows
• Add missing tools: safeinputs-gh, build tools
• Schedule follow-up audit in 7 days to measure improvement
• Consider creating GitHub issues for tracking high-priority items

---

Audit completed: 2025-12-26T04:07:00Z
Data source: 68 workflow runs from /tmp/gh-aw/aw-mcp/logs
Analysis tools: gh-aw MCP server, Python pandas/matplotlib

AI generated by Agentic Workflow Audit Agent

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 3 days ago.