Schema Metadata Quality Audit - December 29, 2025

[github-actions[bot]]

Executed Strategy-008 (Schema Completeness & Internal Reference Analysis) focusing on schema metadata quality, internal consistency, and documentation completeness across all three schema files.

Key Results:

• Total Issues Found: 8 (2 critical, 6 moderate)
• New Issues: 0 (regression check)
• Status vs Last Run (2025-11-18): No changes - all issues persist
• Positive Findings: 6 areas of excellent quality
• Overall Assessment: Root-level quality excellent, metadata and example coverage need improvement

This is a regression check confirming that schema quality has not degraded since November 2025. All identified issues remain from previous audits, indicating stable but unimproved metadata quality.

Critical Issues

1. Missing Top-Level Metadata (All 3 Schemas)

Priority: HIGH | Category: Schema Metadata / IDE Integration

All three schemas are missing critical top-level metadata:

• No $id field for schema identification
• No title field for human-readable name
• No description field for schema purpose

Affected Files:

• pkg/parser/schemas/main_workflow_schema.json
• pkg/parser/schemas/included_file_schema.json
• pkg/parser/schemas/mcp_config_schema.json

Impact:

• IDE Integration: Reduced autocomplete quality and inline documentation
• Schema Evolution: Cannot track schema versions or breaking changes
• Discoverability: Users can't identify schema purpose from metadata
• Tooling: External tools can't properly identify/cache schemas

Recommendation: Add metadata to all schemas:

{
  "$schema": "(redacted)
  "$id": "https://github.com/githubnext/gh-aw/schemas/main_workflow_schema.json",
  "title": "GitHub Agentic Workflow Schema",
  "description": "JSON Schema for validating agentic workflow frontmatter configuration",
  "version": "1.0.0",
  ...
}

2. applyTo Field Missing Description

Priority: MEDIUM | Category: Documentation Completeness

The applyTo field in included_file_schema.json is the only root-level property without a description.

Location: pkg/parser/schemas/included_file_schema.json - applyTo property

Recommendation: Add description explaining that applyTo filters which jobs use the included file.

Moderate Issues

3. Major Fields Missing Examples

Priority: MEDIUM | Category: Discoverability / Learning

Critical workflow configuration fields lack inline examples in schema.

Missing Examples:

• on - Trigger events (most important field)
• engine - AI engine configuration
• tools - Tool configuration
• safe-outputs - Safe output configuration
• mcp-servers - MCP server configuration
• steps - Workflow steps

Current Coverage: Root properties with examples: 19 of 39 (49%)

Impact: Steeper learning curve, reduced IDE autocomplete usefulness, users rely on external documentation

4. Implicit vs Explicit Default Values

Priority: LOW | Category: Schema Quality

6 fields describe default values in description text but lack explicit default field:

• name, timeout-minutes, timeout_minutes, concurrency, roles, github-token

Only 2 fields have explicit default: engine: "copilot", strict: true

5. Single-Value Enums Should Use const

Priority: LOW | Category: Schema Style

10 instances use enum: ["value"] instead of const: "value" (JSON Schema best practice).

6. Zero Format Constraints

Priority: LOW | Category: Validation Strength

No format constraints used despite opportunities for stronger validation (email, uri, date-time, etc.).

Pattern constraints: 16 ✓ | Format constraints: 0 ✗

7. 66 Nested Properties Missing Descriptions

Priority: LOW | Category: Documentation Completeness

Deep nested fields lack descriptions (many are structural artifacts from oneOf patterns).

8. MCP Schema: Zero Examples

Priority: MEDIUM | Category: Usability

mcp_config_schema.json has zero example fields despite being complex configuration.

• Main schema: 76 examples
• Included schema: 3 examples
• MCP schema: 0 examples

Positive Findings

✓ All Root-Level Descriptions Complete (Main Schema): 39 of 39 properties (100%)

✓ All $defs Are Used (No Dead Code):

• engine_config: 2 references
• github_token: 31 references
• http_mcp_tool: 1 reference
• stdio_mcp_tool: 1 reference

✓ Excellent additionalProperties Discipline: 167 constraints prevent typos and invalid fields

✓ Good $comment Usage: 28 fields document implementation notes and compiler behavior

✓ Pattern Constraints Present: 16 pattern constraints validate string formats

✓ Array Constraints Present: 15 minItems + 1 uniqueItems constraints

Key Insights

1. Quality Bimodal: Root descriptions 100% complete ✓, but top-level metadata 0% complete ✗
2. No Regression Since 2025-11-18: All 8 issues existed in previous audit
3. Internal Consistency Excellent: All $defs used, 167 additionalProperties constraints, strong validation
4. Example Coverage Inverted: Less critical fields have examples, most critical fields don't
5. Informal Defaults Common Pattern: Text-described defaults are industry pattern, not necessarily bad

Recommendations

Immediate Actions (Priority 1)

1. Add Top-Level Metadata to All 3 Schemas

   • Add $id, title, description fields
   • Include schema version
   • Effort: 30 minutes

2. Add applyTo Description

   • Complete the only missing root-level description
   • Effort: 5 minutes

Short-Term Actions (Priority 2)

3. Add Examples to Major Fields

   • Priority: on, engine, tools, steps, safe-outputs, mcp-servers
   • Effort: 2-3 hours

4. Add Examples to MCP Schema

   • Currently 0 examples
   • Effort: 30 minutes

Long-Term Actions (Priority 3)

5. Review 66 nested field descriptions (case-by-case)
6. Consider explicit default fields for simple defaults
7. Convert 10 single-value enums to const
8. Consider adding format constraints

Strategy Performance

• Effectiveness: VERY HIGH (consistent with previous runs)
• Success Count: 3 (updated)
• Use Case: Metadata quality audits and regression checks
• Recommendation: Use every 6-8 weeks for quality regression checks

Comparison with Last Run (2025-11-18):

• Then: 9 findings
• Now: 8 findings
• Status: All issues persist, no regression, no improvement

---

Analysis Date: December 29, 2025
Strategy: strategy-008 (Schema Completeness & Internal Reference Analysis)
Files Analyzed: main_workflow_schema.json, included_file_schema.json, mcp_config_schema.json
Status: Regression check PASS - Quality stable, improvements needed

AI generated by Schema Consistency Checker

[pelikhan]

/plan

[github-actions[bot]]

🎉 Yo ho ho! Plan Command found the treasure and completed successfully! ⚓💰

[github-actions[bot]]

Plan Created ✅

I've created a structured implementation plan to address the schema metadata quality findings from this audit:

Parent Tracking Issue: Improve schema metadata and documentation quality

• Links back to this discussion
• Organizes all work under a single tracking issue

5 Sub-Issues Created:

1. Add top-level metadata to all three schema files (Critical)

   • Adds $id, title, description, version to all schemas
   • Estimated effort: 30 minutes

2. Add description to applyTo field (Medium)

   • Completes the only missing root-level documentation
   • Estimated effort: 5 minutes

3. Add inline examples to major workflow fields (Medium)

   • Covers on, engine, tools, steps, safe-outputs, mcp-servers
   • Estimated effort: 2-3 hours

4. Add examples to MCP configuration schema (Medium)

   • Currently has zero examples despite complexity
   • Estimated effort: 30-45 minutes

5. Convert single-value enums to const and add format constraints (Low)

   • JSON Schema best practices and stronger validation
   • Estimated effort: 1-2 hours

Total Estimated Effort: ~5-7 hours

These focused, actionable tasks can be assigned to GitHub Copilot agents for implementation. Each sub-issue has clear objectives, files to modify, implementation guidance, and acceptance criteria.

AI generated by Plan Command for discussion #8130