Skip to content

Commit d37fa8d

Browse files
authored
Merge of #9077
2 parents b2d0a90 + 0e461ae commit d37fa8d

File tree

3 files changed

+265
-1
lines changed

3 files changed

+265
-1
lines changed

src/content/docs/index.mdx

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import DocsetGrid from '../../components/DocsetGrid/DocsetGrid.astro';
99
import CommunityButton from '../../components/CommunityButton.astro';
1010
import { MdOutlineLightbulb, MdMonitorHeart } from 'react-icons/md';
1111
import { GoGitMerge, GoCodeReview } from 'react-icons/go';
12-
import { BsRobot, BsRocket, BsPatchQuestion, BsCommand, BsBook, BsPlugin, BsStack, BsLightbulb } from 'react-icons/bs';
12+
import { BsRobot, BsRocket, BsPatchQuestion, BsCommand, BsBook, BsPlugin, BsStack, BsLightbulb, BsBoxes } from 'react-icons/bs';
1313
import { BiRuler, BiBadgeCheck, BiCut, BiSolidCoinStack } from 'react-icons/bi';
1414
import { TbPackages, TbGitBranch } from 'react-icons/tb';
1515
import { TiFlowParallel } from 'react-icons/ti';
@@ -96,6 +96,9 @@ import { SlRefresh, SlSpeedometer } from 'react-icons/sl';
9696
<Docset title="Two‑Step CI" path="/merge-queue/two-step" icon={FaTrafficLight}>
9797
Fast + full validation strategy.
9898
</Docset>
99+
<Docset title="Monorepo" path="/merge-queue/monorepo" icon={BsBoxes}>
100+
Optimization for large repositories.
101+
</Docset>
99102
<Docset title="Deployment" path="/merge-queue/deploy" icon={AiOutlineDeploymentUnit}>
100103
Adopt safely in production.
101104
</Docset>
Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
---
2+
title: Monorepo
3+
description: Optimize merge queue batching for monorepos with scopes.
4+
---
5+
6+
In monorepo environments, not every pull request affects the entire codebase. Running all tests for
7+
every change wastes time and CI resources. Mergify's **scopes** feature allows you to intelligently
8+
batch pull requests based on which parts of your codebase they modify, dramatically improving merge
9+
queue efficiency.
10+
11+
## Understanding Scopes
12+
13+
Scopes define discrete areas of your monorepo (like packages, services, or components). When a pull
14+
request is created, Mergify automatically determines which scopes are affected and uses this
15+
information to optimize batching.
16+
17+
Scopes can be determined in several ways depending on your project's needs:
18+
19+
- **File patterns**: Match file paths to identify affected scopes (currently supported)
20+
21+
- **Build system integration**: Support for tools like Bazel, Nx, Turborepo, and others that
22+
can provide scope information based on dependency graphs and build targets
23+
24+
This flexibility allows you to use the approach that best fits your monorepo's architecture and
25+
existing tooling.
26+
27+
### The Batching Challenge in Monorepos
28+
29+
Without scopes, Mergify batches pull requests together regardless of what they change. This means:
30+
31+
- A Python service change and a JavaScript frontend change might be batched together
32+
- Both sets of tests run even though they're completely independent
33+
- If one fails, both PRs are affected by the batch split process
34+
35+
With scopes, Mergify can:
36+
37+
- Batch together PRs that affect the same scopes (e.g., multiple Python changes)
38+
- Keep independent changes in separate batches
39+
- Reduce unnecessary CI runs for unrelated parts of your codebase
40+
41+
## Configuring Scopes
42+
43+
Define scopes in your `.mergify.yml` configuration file using file patterns:
44+
45+
```yaml
46+
scopes:
47+
source:
48+
files:
49+
python-api:
50+
includes:
51+
- api/**/*.py
52+
- libs/shared/**/*.py
53+
frontend:
54+
includes:
55+
- web/**/*.js
56+
- web/**/*.jsx
57+
- web/**/*.ts
58+
- web/**/*.tsx
59+
docs:
60+
includes:
61+
- docs/**/*.md
62+
- docs/**/*.mdx
63+
64+
queue_rules:
65+
- name: default
66+
batch_size: 5
67+
```
68+
69+
In this example:
70+
- Changes to Python files in `api/` or `libs/shared/` get the `python-api` scope
71+
- Changes to frontend files in `web/` get the `frontend` scope
72+
- Documentation changes get the `docs` scope
73+
74+
:::tip
75+
Mergify will intelligently batch PRs with overlapping scopes together. For example, if PR1 affects
76+
`python-api` and PR2 affects both `python-api` and `frontend`, they'll be batched together since
77+
they share a common scope.
78+
:::
79+
80+
## Setting Up CI with Scopes
81+
82+
To leverage scopes in your CI workflow, use the
83+
[gha-mergify-ci](https://github.com/Mergifyio/gha-mergify-ci) GitHub Action. This action detects
84+
which scopes are affected by a pull request and allows you to run only the relevant tests.
85+
86+
### GitHub Actions Integration
87+
88+
Here's a complete example showing how to set up scope-aware CI:
89+
90+
```yaml
91+
name: Continuous Integration
92+
on:
93+
pull_request:
94+
95+
jobs:
96+
scopes:
97+
runs-on: ubuntu-24.04
98+
outputs:
99+
python-api: ${{ fromJSON(steps.scopes.outputs.scopes).python-api }}
100+
frontend: ${{ fromJSON(steps.scopes.outputs.scopes).frontend }}
101+
docs: ${{ fromJSON(steps.scopes.outputs.scopes).docs }}
102+
merge-queue: ${{ fromJSON(steps.scopes.outputs.scopes).merge-queue }}
103+
steps:
104+
- uses: actions/checkout@v5
105+
- name: Get PR scopes
106+
id: scopes
107+
uses: Mergifyio/gha-mergify-ci@v9
108+
with:
109+
action: scopes
110+
token: ${{ secrets.MERGIFY_TOKEN }}
111+
112+
python-tests:
113+
if: ${{ needs.scopes.outputs.python-api == 'true' }}
114+
needs: scopes
115+
uses: ./.github/workflows/python-tests.yaml
116+
secrets: inherit
117+
118+
frontend-tests:
119+
if: ${{ needs.scopes.outputs.frontend == 'true' }}
120+
needs: scopes
121+
uses: ./.github/workflows/frontend-tests.yaml
122+
secrets: inherit
123+
124+
docs-tests:
125+
if: ${{ needs.scopes.outputs.docs == 'true' }}
126+
needs: scopes
127+
uses: ./.github/workflows/docs-tests.yaml
128+
secrets: inherit
129+
130+
integration-tests:
131+
if: ${{ needs.scopes.outputs.merge-queue == 'true' }}
132+
needs: scopes
133+
uses: ./.github/workflows/integration-tests.yaml
134+
secrets: inherit
135+
136+
alls-green:
137+
if: ${{ !cancelled() }}
138+
needs:
139+
- python-tests
140+
- frontend-tests
141+
- docs-tests
142+
- integration-tests
143+
runs-on: ubuntu-latest
144+
steps:
145+
- name: Verify all jobs succeeded
146+
uses: re-actors/alls-green@release/v1
147+
with:
148+
allowed-skips: ${{ toJSON(needs) }}
149+
jobs: ${{ toJSON(needs) }}
150+
```
151+
152+
### Key Components
153+
154+
1. **Scopes Job**: Detects which scopes are affected and outputs boolean values
155+
156+
2. **Conditional Jobs**: Each test suite runs only if its scope is affected
157+
158+
3. **Integration Tests**: The special `merge-queue` scope is automatically set to `true` when
159+
running in the merge queue context
160+
161+
4. **Alls-Green**: Aggregates all job results, handling skipped jobs correctly
162+
163+
## The Merge Queue Scope
164+
165+
The `gha-mergify-ci` action automatically provides a special `merge-queue` scope that returns `true`
166+
only when running in a merge queue context (on temporary merge queue branches).
167+
168+
This is useful for:
169+
170+
- **Integration tests** that only need to run before merging
171+
- **End-to-end tests** that are expensive and should only run on final batches
172+
- **Deployment validation** that needs to happen before code reaches the main branch
173+
174+
```yaml
175+
integration-tests:
176+
if: ${{ needs.scopes.outputs.merge-queue == 'true' }}
177+
needs: scopes
178+
runs-on: ubuntu-22.04
179+
steps:
180+
- uses: actions/checkout@v5
181+
- name: Run expensive integration tests
182+
run: npm run test:integration
183+
```
184+
185+
## Important Behaviors
186+
187+
### Scope Detection is PR-Specific
188+
189+
The `gha-mergify-ci` action only analyzes files changed by the specific pull request, **not** files
190+
from other PRs in the merge queue batch. This ensures:
191+
192+
- Each PR's scopes reflect only its own changes
193+
- Batching decisions remain consistent even as the queue changes
194+
- Tests run for the correct scopes regardless of what else is in the batch
195+
196+
### Path Filtering vs Scopes
197+
198+
GitHub Actions offers path filtering (`on.pull_request.paths`), but it has critical limitations in
199+
merge queue scenarios:
200+
201+
```yaml
202+
# ❌ Don't use path filtering for merge queues
203+
on:
204+
pull_request:
205+
paths:
206+
- 'api/**'
207+
```
208+
209+
**Problems with path filtering:**
210+
211+
- When a job doesn't run, you can't distinguish between "filtered out" and "CI failed to start"
212+
213+
- Required status checks fail if jobs are skipped due to filtering
214+
215+
- In merge queues, you don't want to skip tests on PR2 just because PR1 in the batch modified
216+
different files
217+
218+
**✅ Use scopes instead:**
219+
220+
- Jobs always run but can conditionally skip work based on scope detection
221+
- Status checks always report (success or skipped)
222+
- Merge queue batching respects scope boundaries
223+
224+
## Example: Multi-Language Monorepo
225+
226+
Here's a real-world example for a monorepo with Python, JavaScript, and Go services:
227+
228+
```yaml
229+
scopes:
230+
source:
231+
files:
232+
python-api:
233+
includes:
234+
- services/api/**/*.py
235+
- libs/python/**/*.py
236+
user-service:
237+
includes:
238+
- services/users/**/*.go
239+
frontend:
240+
includes:
241+
- apps/web/**/*.{js,jsx,ts,tsx}
242+
shared-config:
243+
includes:
244+
- config/**/*
245+
- docker/**/*
246+
247+
queue_rules:
248+
- name: default
249+
batch_size: 8
250+
batch_max_wait_time: 5 min
251+
```
252+
253+
With this configuration:
254+
- PRs affecting only `frontend` will batch together
255+
256+
- PRs affecting `python-api` will batch together
257+
258+
- PRs affecting `shared-config` will batch with everything (since config affects all
259+
services)

src/content/navItems.tsx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { AiOutlineApi, AiOutlineDeploymentUnit, AiOutlineFile } from 'react-icon
44
import { BiBadgeCheck, BiCut, BiRuler, BiSolidCoinStack } from 'react-icons/bi';
55
import {
66
BsBook,
7+
BsBoxes,
78
BsCommand,
89
BsGear,
910
BsLightbulb,
@@ -141,6 +142,7 @@ const navItems: NavItem[] = [
141142
{ title: 'Parallel Checks', path: '/merge-queue/parallel-checks', icon: TiFlowParallel },
142143
{ title: 'Batches', path: '/merge-queue/batches', icon: TbPackages },
143144
{ title: 'Two-Step CI', path: '/merge-queue/two-step', icon: FaStairs },
145+
{ title: 'Monorepo', path: '/merge-queue/monorepo', icon: BsBoxes },
144146
{ title: 'Deployment', path: '/merge-queue/deploy', icon: AiOutlineDeploymentUnit },
145147
{ title: 'Monitoring', path: '/merge-queue/monitoring', icon: MdMonitorHeart },
146148
],

0 commit comments

Comments
 (0)