Skip to content

feat(atomic-a11y): add merge-shards for parallel a11y report aggregation#7137

Open
y-lakhdar wants to merge 11 commits intofeat/a11y-reporterfrom
feat/a11y-merge-shards
Open

feat(atomic-a11y): add merge-shards for parallel a11y report aggregation#7137
y-lakhdar wants to merge 11 commits intofeat/a11y-reporterfrom
feat/a11y-merge-shards

Conversation

@y-lakhdar
Copy link
Contributor

@y-lakhdar y-lakhdar commented Feb 17, 2026

TL;DR

Merge shard JSON reports from parallel CI runs into a single consolidated a11y report.

Context

When Storybook tests run with --shard=N/M, each shard writes its own a11y-report.shard-N.json. This PR adds the merge step that combines them.

CI (parallel shards)
  ├─ shard 1/3 → a11y-report.shard-1.json   ─┐
  ├─ shard 2/3 → a11y-report.shard-2.json   ─┤──→  mergeA11yShardReports()  →  a11y-report.json
  └─ shard 3/3 → a11y-report.shard-3.json   ─┘
                                                     • Deduplicate components across shards
                                                     • Sum violation/pass/incomplete counts
                                                     • Union criteria coverage + affected components
                                                     • Recompute summary from merged data

How the shard merge works

When Storybook tests run with --shard, each shard writes its own a11y-report.shard-N.json. The merge step recombines them into a single a11y-report.json.

mergeA11yShardReports() — entry point

  1. Scan the output directory for files matching a11y-report.shard-{N}.json
  2. Read & validate each shard file in parallel (Promise.allSettled — invalid shards are skipped with a warning)
  3. Call mergeComponents()mergeCriteria()createSummary()
  4. Take report metadata (product, versions, etc.) from the first shard, stamp a fresh reportDate
  5. Write the merged a11y-report.json

mergeComponents() — deduplicate by component name

  • First shard wins as the base record for each component
  • Subsequent shards sum counts (storyCount, violations, passes, incomplete, inapplicable)
  • Union criteriaCovered sets and incompleteDetails arrays
  • Output sorted alphabetically by name

example
Shard 1 tested atomic-search-box with commerce stories, shard 2 tested it with search stories:

Shard 1:                                  Shard 2:
├─ name: "atomic-search-box"              ├─ name: "atomic-search-box"
├─ storyCount: 3                          ├─ storyCount: 2
└─ automated:                             └─ automated:
   ├─ violations: 1                          ├─ violations: 0
   ├─ passes: 12                             ├─ passes: 8
   ├─ incomplete: 0                          ├─ incomplete: 1
   ├─ inapplicable: 4                        ├─ inapplicable: 3
   ├─ criteriaCovered: ["1.4.3"]             ├─ criteriaCovered: ["1.4.3", "2.1.1"]
   └─ incompleteDetails: []                  └─ incompleteDetails: [{ruleId: "color-contrast", ...}]

Merged:
├─ name: "atomic-search-box"
├─ category: "search"                                       ← backfilled from shard 2 ("unknown" → "search")
├─ storyCount: 5                                            ← 3 + 2
└─ automated:
   ├─ violations: 1                                         ← 1 + 0
   ├─ passes: 20                                            ← 12 + 8
   ├─ incomplete: 1                                         ← 0 + 1
   ├─ inapplicable: 7                                       ← 4 + 3
   ├─ criteriaCovered: ["1.4.3", "2.1.1"]                   ← union of both sets
   └─ incompleteDetails: [{ruleId: "color-contrast", ...}]  ← concatenated

mergeCriteria() — deduplicate by criterion ID, two passes

  1. Pass 1 — merge from shard criteria lists: union affectedComponents across shards for each criterion ID
  2. Pass 2 — infer from merged components: if a component covers a criterion ID that no shard's criteria list included, synthesize a new criterion entry from WCAG metadata (logs a warning with the count of inferred criteria)
  • Output sorted by numeric criterion ID (1.1.1 < 1.4.3 < 4.1.2)

example
Pass 1 — merge from shard criteria lists:

Shard 1 criteria:                         Shard 2 criteria:
├─ id: "1.4.3"                            ├─ id: "1.4.3"
│  └─ affectedComponents:                 │  └─ affectedComponents:
│     ["atomic-search-box"]               │     ["atomic-search-box", "atomic-result-list"]
└─ id: "2.1.1"                            └─ (no 2.1.1 entry)
   └─ affectedComponents:
      ["atomic-search-box"]
After pass 1:
├─ id: "1.4.3"
│  └─ affectedComponents: ["atomic-result-list", "atomic-search-box"]  ← union, sorted
└─ id: "2.1.1"
   └─ affectedComponents: ["atomic-search-box"]

Pass 2 — infer from merged components. Suppose atomic-result-list covers criterion "4.1.2" (from its criteriaCovered array), but no shard had a "4.1.2" criteria entry:

After pass 2:
├─ id: "1.4.3"  (from shards)
├─ id: "2.1.1"  (from shards)
└─ id: "4.1.2"  ← synthesized from WCAG metadata
   ├─ ...
   └─ affectedComponents: ["atomic-result-list"]
   [merge-shards] 1 criteria were inferred from component coverage.

PR Chain (4 of 7)

# PR Description
1 #7111 Package scaffolding
2 #7122 Shared types, constants, utilities
3 #7123 VitestA11yReporter + wiring
4 #7137 Shard merging ← this PR
5 #7124 OpenACR report generator
6 #7125 CLI scripts
7 #7117 Weekly a11y scan workflow

KIT-5469

@y-lakhdar y-lakhdar force-pushed the feat/a11y-merge-shards branch from 5cb5fe5 to 61496ce Compare February 17, 2026 14:37
@y-lakhdar y-lakhdar added the a11y Accessibility issues label Feb 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants