Skip to content

Commit 1fbc3ef

Browse files
silehtsileht
committed
feat: add documentation about scope
Change-Id: I7a91bbdad01645bbd849da5b892e20c64791a85a
1 parent 9e78147 commit 1fbc3ef

File tree

2 files changed

+325
-0
lines changed

2 files changed

+325
-0
lines changed

‎src/content/docs/merge-queue/monorepo.mdx‎

Lines changed: 324 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,324 @@
1+
---
2+
title: Monorepo Optimization
3+
description: Optimize merge queue batching for monorepos with scopes.
4+
---
5+
6+
In monorepo environments, not every pull request affects the entire codebase. Running all tests for
7+
every change wastes time and CI resources. Mergify's **scopes** feature allows you to intelligently
8+
batch pull requests based on which parts of your codebase they modify, dramatically improving merge
9+
queue efficiency.
10+
11+
## Understanding Scopes
12+
13+
Scopes define discrete areas of your monorepo (like packages, services, or components) and map them
14+
to file patterns. When a pull request modifies files, Mergify automatically determines which scopes
15+
are affected and uses this information to optimize batching.
16+
17+
### The Batching Challenge in Monorepos
18+
19+
Without scopes, Mergify batches pull requests together regardless of what they change. This means:
20+
21+
- A Python service change and a JavaScript frontend change might be batched together
22+
- Both sets of tests run even though they're completely independent
23+
- If one fails, both PRs are affected by the batch split process
24+
25+
With scopes, Mergify can:
26+
27+
- Batch together PRs that affect the same scopes (e.g., multiple Python changes)
28+
- Keep independent changes in separate batches
29+
- Reduce unnecessary CI runs for unrelated parts of your codebase
30+
31+
## Configuring Scopes
32+
33+
Define scopes in your `.mergify.yml` configuration file using file patterns:
34+
35+
```yaml
36+
scopes:
37+
source:
38+
files:
39+
python-api:
40+
includes:
41+
- api/**/*.py
42+
- libs/shared/**/*.py
43+
frontend:
44+
includes:
45+
- web/**/*.js
46+
- web/**/*.jsx
47+
- web/**/*.ts
48+
- web/**/*.tsx
49+
docs:
50+
includes:
51+
- docs/**/*.md
52+
- docs/**/*.mdx
53+
54+
queue_rules:
55+
- name: default
56+
batch_size: 5
57+
queue_conditions:
58+
- check-success=alls-green
59+
merge_conditions:
60+
- check-success=alls-green
61+
```
62+
63+
In this example:
64+
- Changes to Python files in `api/` or `libs/shared/` get the `python-api` scope
65+
- Changes to frontend files in `web/` get the `frontend` scope
66+
- Documentation changes get the `docs` scope
67+
68+
:::tip
69+
Mergify will intelligently batch PRs with overlapping scopes together. For example, if PR1 affects
70+
`python-api` and PR2 affects both `python-api` and `frontend`, they'll be batched together since
71+
they share a common scope.
72+
:::
73+
74+
## Setting Up CI with Scopes
75+
76+
To leverage scopes in your CI workflow, use the
77+
[gha-mergify-ci](https://github.com/Mergifyio/gha-mergify-ci) GitHub Action. This action detects
78+
which scopes are affected by a pull request and allows you to run only the relevant tests.
79+
80+
### GitHub Actions Integration
81+
82+
Here's a complete example showing how to set up scope-aware CI:
83+
84+
```yaml
85+
name: Continuous Integration
86+
on:
87+
pull_request:
88+
89+
jobs:
90+
scopes:
91+
runs-on: ubuntu-24.04
92+
outputs:
93+
python-api: ${{ fromJSON(steps.scopes.outputs.scopes).python-api }}
94+
frontend: ${{ fromJSON(steps.scopes.outputs.scopes).frontend }}
95+
docs: ${{ fromJSON(steps.scopes.outputs.scopes).docs }}
96+
merge-queue: ${{ fromJSON(steps.scopes.outputs.scopes).merge-queue }}
97+
steps:
98+
- uses: actions/checkout@v5
99+
- name: Get PR scopes
100+
id: scopes
101+
uses: Mergifyio/gha-mergify-ci@v9
102+
with:
103+
action: scopes
104+
token: ${{ secrets.MERGIFY_TOKEN }}
105+
106+
python-tests:
107+
if: ${{ needs.scopes.outputs.python-api == 'true' }}
108+
needs: scopes
109+
uses: ./.github/workflows/python-tests.yaml
110+
secrets: inherit
111+
112+
frontend-tests:
113+
if: ${{ needs.scopes.outputs.frontend == 'true' }}
114+
needs: scopes
115+
uses: ./.github/workflows/frontend-tests.yaml
116+
secrets: inherit
117+
118+
docs-tests:
119+
if: ${{ needs.scopes.outputs.docs == 'true' }}
120+
needs: scopes
121+
uses: ./.github/workflows/docs-tests.yaml
122+
secrets: inherit
123+
124+
integration-tests:
125+
if: ${{ needs.scopes.outputs.merge-queue == 'true' }}
126+
needs: scopes
127+
uses: ./.github/workflows/integration-tests.yaml
128+
secrets: inherit
129+
130+
alls-green:
131+
if: ${{ !cancelled() }}
132+
needs:
133+
- python-tests
134+
- frontend-tests
135+
- docs-tests
136+
- integration-tests
137+
runs-on: ubuntu-latest
138+
steps:
139+
- name: Verify all jobs succeeded
140+
uses: re-actors/alls-green@release/v1
141+
with:
142+
allowed-skips: ${{ toJSON(needs) }}
143+
jobs: ${{ toJSON(needs) }}
144+
```
145+
146+
### Key Components
147+
148+
1. **Scopes Job**: Detects which scopes are affected and outputs boolean values
149+
150+
2. **Conditional Jobs**: Each test suite runs only if its scope is affected
151+
152+
3. **Integration Tests**: The special `merge-queue` scope is automatically set to `true` when
153+
running in the merge queue context
154+
155+
4. **Alls-Green**: Aggregates all job results, handling skipped jobs correctly
156+
157+
## The Merge Queue Scope
158+
159+
The `gha-mergify-ci` action automatically provides a special `merge-queue` scope that returns `true`
160+
only when running in a merge queue context (on temporary merge queue branches).
161+
162+
This is useful for:
163+
164+
- **Integration tests** that only need to run before merging
165+
- **End-to-end tests** that are expensive and should only run on final batches
166+
- **Deployment validation** that needs to happen before code reaches the main branch
167+
168+
```yaml
169+
integration-tests:
170+
if: ${{ needs.scopes.outputs.merge-queue == 'true' }}
171+
needs: scopes
172+
runs-on: ubuntu-22.04
173+
steps:
174+
- uses: actions/checkout@v5
175+
- name: Run expensive integration tests
176+
run: npm run test:integration
177+
```
178+
179+
## Important Behaviors
180+
181+
### Scope Detection is PR-Specific
182+
183+
The `gha-mergify-ci` action only analyzes files changed by the specific pull request, **not** files
184+
from other PRs in the merge queue batch. This ensures:
185+
186+
- Each PR's scopes reflect only its own changes
187+
- Batching decisions remain consistent even as the queue changes
188+
- Tests run for the correct scopes regardless of what else is in the batch
189+
190+
### Path Filtering vs Scopes
191+
192+
GitHub Actions offers path filtering (`on.pull_request.paths`), but it has critical limitations in
193+
merge queue scenarios:
194+
195+
```yaml
196+
# ❌ Don't use path filtering for merge queues
197+
on:
198+
pull_request:
199+
paths:
200+
- 'api/**'
201+
```
202+
203+
**Problems with path filtering:**
204+
205+
- When a job doesn't run, you can't distinguish between "filtered out" and "CI failed to start"
206+
207+
- Required status checks fail if jobs are skipped due to filtering
208+
209+
- In merge queues, you don't want to skip tests on PR2 just because PR1 in the batch modified
210+
different files
211+
212+
**✅ Use scopes instead:**
213+
214+
- Jobs always run but can conditionally skip work based on scope detection
215+
- Status checks always report (success or skipped)
216+
- Merge queue batching respects scope boundaries
217+
218+
## Example: Multi-Language Monorepo
219+
220+
Here's a real-world example for a monorepo with Python, JavaScript, and Go services:
221+
222+
```yaml
223+
scopes:
224+
source:
225+
files:
226+
python-api:
227+
includes:
228+
- services/api/**/*.py
229+
- libs/python/**/*.py
230+
user-service:
231+
includes:
232+
- services/users/**/*.go
233+
frontend:
234+
includes:
235+
- apps/web/**/*.{js,jsx,ts,tsx}
236+
shared-config:
237+
includes:
238+
- config/**/*
239+
- docker/**/*
240+
241+
queue_rules:
242+
- name: default
243+
batch_size: 8
244+
batch_max_wait_time: 5 min
245+
queue_conditions:
246+
- check-success=alls-green
247+
merge_conditions:
248+
- check-success=alls-green
249+
```
250+
251+
With this configuration:
252+
- PRs affecting only `frontend` will batch together
253+
254+
- PRs affecting `python-api` will batch together
255+
256+
- PRs affecting `shared-config` will batch with everything (since config affects all
257+
services)
258+
259+
- Independent service changes can be tested in parallel batches
260+
261+
## Best Practices
262+
263+
### 1. Define Clear Scope Boundaries
264+
265+
```yaml
266+
# ✅ Good: Clear, non-overlapping scopes
267+
scopes:
268+
source:
269+
files:
270+
backend:
271+
includes:
272+
- backend/**/*
273+
frontend:
274+
includes:
275+
- frontend/**/*
276+
```
277+
278+
### 2. Handle Shared Dependencies
279+
280+
If you have shared libraries, consider whether changes should affect all dependent
281+
scopes:
282+
283+
```yaml
284+
# Shared lib changes affect everything
285+
shared-libs:
286+
includes:
287+
- libs/common/**/*
288+
```
289+
290+
### 3. Use Integration Tests Wisely
291+
292+
Reserve expensive tests for the merge queue context:
293+
294+
```yaml
295+
unit-tests:
296+
if: ${{ needs.scopes.outputs.backend == 'true' }}
297+
# Runs on every PR update
298+
299+
integration-tests:
300+
if: ${{ needs.scopes.outputs.merge-queue == 'true' }}
301+
# Only runs in merge queue
302+
```
303+
304+
## Monitoring Scope-Based Batching
305+
306+
You can monitor how scopes affect your merge queue performance in the
307+
[Mergify dashboard](https://dashboard.mergify.com):
308+
309+
- View which scopes are active in each batch
310+
- See how scopes influence batch composition
311+
- Track CI time savings from scope-aware batching
312+
313+
## Conclusion
314+
315+
Scopes transform merge queue performance in monorepos by:
316+
317+
- Batching related changes together while keeping unrelated changes separate
318+
- Reducing CI costs by running only relevant tests
319+
- Maintaining merge velocity by creating more efficient batches
320+
- Preserving reliability by ensuring appropriate tests run for each change
321+
322+
By combining scopes with [batch merging](/merge-queue/batches) and
323+
[parallel checks](/merge-queue/parallel-checks), you can achieve optimal merge queue performance
324+
tailored to your monorepo's structure.

‎src/content/navItems.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,7 @@ const navItems: NavItem[] = [
140140
{ title: 'Performance', path: '/merge-queue/performance', icon: SlSpeedometer },
141141
{ title: 'Parallel Checks', path: '/merge-queue/parallel-checks', icon: TiFlowParallel },
142142
{ title: 'Batches', path: '/merge-queue/batches', icon: TbPackages },
143+
{ title: 'Monorepo Optimization', path: '/merge-queue/monorepo', icon: BsStack },
143144
{ title: 'Two-Step CI', path: '/merge-queue/two-step', icon: FaStairs },
144145
{ title: 'Deployment', path: '/merge-queue/deploy', icon: AiOutlineDeploymentUnit },
145146
{ title: 'Monitoring', path: '/merge-queue/monitoring', icon: MdMonitorHeart },

0 commit comments

Comments
 (0)