Skip to content

Commit cf13790

Browse files
Copilotleofangpre-commit-ci[bot]
authored
Add CI infrastructure overview with hand-drawn SVG diagram, Mermaid alternative, and enhanced artifact flow visualization to CONTRIBUTING.md (#916)
* Initial plan * Add CI infrastructure overview section to CONTRIBUTING.md - Added cuda-pathfinder component to the component guidelines list - Added footnote explaining cuda.bindings follows cuda-python contributing guide - Added comprehensive CI infrastructure overview section with: - Visual flowchart diagram showing build → test → release pipeline - Platform and runner details (self-hosted vs GitHub-hosted) - Artifact storage and propagation (GitHub Artifacts vs Cache) - Branch-specific workflows (main, backport, pull-request) - Key infrastructure details Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Address CI infrastructure feedback: update flowchart, fix footnotes, add SVG diagram Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Update SVG diagram with requested changes and move to ci/ directory Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Remove nv-gha-runners implementation detail and improve SVG diagram layout Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Convert SVG diagram to hand-drawn Excalidraw-like style Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Remove pull-request/* branches section from CONTRIBUTING.md Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Add artifact flow arrows and regeneration reference file Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Fix SVG arrow styling and add parallel execution documentation Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Fix SVG arrow styling and restructure CONTRIBUTING.md Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Fix SVG arrow styling, move footnote to bottom, and add Mermaid diagram Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Fix SVG arrow positioning to prevent overlap with boxes and remove unnecessary markdown separator Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Manually adjust SVG marker sizes and text positions * Replace duplicated Mermaid content with file reference and update regeneration instructions Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Rename ci-pipeline.mmd to ci-pipeline.md * try this Updated the CI pipeline representation with a new image format. * Inline Mermaid diagram in CONTRIBUTING.md instead of file reference Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Delete ci/ci-pipeline.md * [pre-commit.ci] auto code formatting * Add SPDX headers to CI pipeline files Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Fix footnote formatting and convert .txt to .md for pipeline regeneration docs Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * md files don't need spdx * Remove pull-request/* branches trigger event references from CI documentation Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Update ci/ci-pipeline.svg * Add artifact flow arrows to TEST STAGE and remove empty line from SVG Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> * Apply suggestions from code review * Fix SVG artifact flow arrows and add wheel upload label Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: leofang <5534781+leofang@users.noreply.github.com> Co-authored-by: Leo Fang <leof@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent d64120b commit cf13790

File tree

3 files changed

+411
-1
lines changed

3 files changed

+411
-1
lines changed

‎CONTRIBUTING.md‎

Lines changed: 133 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,15 @@ Thank you for your interest in contributing to CUDA Python! Based on the type of
1111
2. You want to implement a feature, improvement, or bug fix:
1212
- Please refer to each component's guideline:
1313
- [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/contribute.html)
14-
- [`cuda.bindings`](https://nvidia.github.io/cuda-python/cuda-bindings/latest/contribute.html)
14+
- [`cuda.bindings`](https://nvidia.github.io/cuda-python/cuda-bindings/latest/contribute.html)<sup>[1](#footnote1)</sup>
15+
- [`cuda.pathfinder`](https://nvidia.github.io/cuda-python/cuda-pathfinder/latest/contribute.html)
16+
17+
## Table of Contents
18+
19+
- [Pre-commit](#pre-commit)
20+
- [Code signing](#code-signing)
21+
- [Developer Certificate of Origin (DCO)](#developer-certificate-of-origin-dco)
22+
- [CI infrastructure overview](#ci-infrastructure-overview)
1523

1624

1725
## Pre-commit
@@ -78,3 +86,127 @@ By making a contribution to this project, I certify that:
7886
maintained indefinitely and may be redistributed consistent with
7987
this project or the open source license(s) involved.
8088
```
89+
90+
## CI infrastructure overview
91+
92+
The CUDA Python project uses a comprehensive CI pipeline that builds, tests, and releases multiple components across different platforms. This section provides a visual overview of our CI infrastructure to help contributors understand the build and release process.
93+
94+
### CI Pipeline Flow
95+
96+
![CUDA Python CI Pipeline Flow](ci/ci-pipeline.svg)
97+
98+
Alternative Mermaid diagram representation:
99+
100+
```mermaid
101+
flowchart TD
102+
%% Trigger Events
103+
subgraph TRIGGER["🔄 TRIGGER EVENTS"]
104+
T1["• Push to main branch"]
105+
T2["• Pull request<br/>• Manual workflow dispatch"]
106+
T1 --- T2
107+
end
108+
109+
%% Build Stage
110+
subgraph BUILD["🔨 BUILD STAGE"]
111+
subgraph BUILD_PLATFORMS["Parallel Platform Builds"]
112+
B1["linux-64<br/>(Self-hosted)"]
113+
B2["linux-aarch64<br/>(Self-hosted)"]
114+
B3["win-64<br/>(GitHub-hosted)"]
115+
end
116+
BUILD_DETAILS["• Python versions: 3.9, 3.10, 3.11, 3.12, 3.13<br/>• CUDA version: 13.0.0 (build-time)<br/>• Components: cuda-core, cuda-bindings,<br/> cuda-pathfinder, cuda-python"]
117+
end
118+
119+
%% Artifact Storage
120+
subgraph ARTIFACTS["📦 ARTIFACT STORAGE"]
121+
subgraph GITHUB_ARTIFACTS["GitHub Artifacts"]
122+
GA1["• Wheel files (.whl)<br/>• Test artifacts<br/>• Documentation<br/>(30-day retention)"]
123+
end
124+
subgraph GITHUB_CACHE["GitHub Cache"]
125+
GC1["• Mini CTK cache"]
126+
end
127+
end
128+
129+
%% Test Stage
130+
subgraph TEST["🧪 TEST STAGE"]
131+
subgraph TEST_PLATFORMS["Parallel Platform Tests"]
132+
TS1["linux-64<br/>(Self-hosted)"]
133+
TS2["linux-aarch64<br/>(Self-hosted)"]
134+
TS3["win-64<br/>(GitHub-hosted)"]
135+
end
136+
TEST_DETAILS["• Download wheels from artifacts<br/>• Test against multiple CUDA runtime versions<br/>• Run Python unit tests, Cython tests, examples"]
137+
ARTIFACT_FLOWS["Artifact Flows:<br/>• cuda-pathfinder: main → backport<br/>• cuda-bindings: backport → main"]
138+
end
139+
140+
%% Release Pipeline
141+
subgraph RELEASE["🚀 RELEASE PIPELINE"]
142+
subgraph RELEASE_STAGES["Sequential Release Steps"]
143+
R1["Validation<br/>• Artifact integrity<br/>• Git tag verification"]
144+
R2["Publishing<br/>• PyPI/TestPyPI<br/>• Component or all releases"]
145+
R3["Documentation<br/>• GitHub Pages<br/>• Release notes"]
146+
R1 --> R2 --> R3
147+
end
148+
RELEASE_DETAILS["• Manual workflow dispatch with run ID<br/>• Supports individual component or full releases"]
149+
end
150+
151+
%% Main Flow
152+
TRIGGER --> BUILD
153+
BUILD -.->|"wheel upload"| ARTIFACTS
154+
ARTIFACTS -.-> TEST
155+
TEST --> RELEASE
156+
157+
%% Artifact Flow Arrows (Cache Reuse)
158+
GITHUB_CACHE -.->|"mini CTK reuse"| BUILD
159+
GITHUB_CACHE -.->|"mini CTK reuse"| TEST
160+
161+
%% Artifact Flow Arrows (Wheel Fetch)
162+
GITHUB_ARTIFACTS -.->|"wheel fetch"| TEST
163+
GITHUB_ARTIFACTS -.->|"wheel fetch"| RELEASE
164+
165+
%% Styling
166+
classDef triggerStyle fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#1976D2
167+
classDef buildStyle fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#7B1FA2
168+
classDef artifactStyle fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#F57C00
169+
classDef testStyle fill:#e8f5e8,stroke:#4CAF50,stroke-width:2px,color:#388E3C
170+
classDef releaseStyle fill:#ffebee,stroke:#f44336,stroke-width:2px,color:#D32F2F
171+
172+
class TRIGGER,T1,T2 triggerStyle
173+
class BUILD,BUILD_PLATFORMS,B1,B2,B3,BUILD_DETAILS buildStyle
174+
class ARTIFACTS,GITHUB_ARTIFACTS,GITHUB_CACHE,GA1,GC1 artifactStyle
175+
class TEST,TEST_PLATFORMS,TS1,TS2,TS3,TEST_DETAILS,ARTIFACT_FLOWS testStyle
176+
class RELEASE,RELEASE_STAGES,R1,R2,R3,RELEASE_DETAILS releaseStyle
177+
```
178+
179+
### Pipeline Execution Details
180+
181+
**Parallel Execution**: The CI pipeline leverages parallel execution to optimize build and test times:
182+
- **Build Stage**: Different architectures/operating systems (linux-64, linux-aarch64, win-64) are built in parallel across their respective runners
183+
- **Test Stage**: Different architectures/operating systems/CUDA versions are tested in parallel; documentation preview is also built in parallel with testing
184+
185+
### Branch-specific Artifact Flow
186+
187+
#### Main Branch
188+
- **Build** → **Test** → **Documentation** → **Potential Release**
189+
- Artifacts stored as `{component}-python{version}-{platform}-{sha}`
190+
- Full test coverage across all platforms and CUDA versions
191+
- **Artifact flow out**: `cuda-pathfinder` artifacts → backport branches
192+
193+
#### Backport Branches
194+
- **Build** → **Test** → **Backport PR Creation**
195+
- Artifacts used for validation before creating backport pull requests
196+
- Maintains compatibility with older CUDA versions
197+
- **Artifact flow in**: `cuda-pathfinder` artifacts ← main branch
198+
- **Artifact flow out**: older `cuda-bindings` artifacts → main branch
199+
200+
### Key Infrastructure Details
201+
202+
- **Self-hosted runners**: Used for Linux builds and GPU testing (more resources, faster builds)
203+
- **GitHub-hosted runners**: Used for Windows builds and general tasks
204+
- **Artifact retention**: 30 days for GitHub Artifacts (wheels, docs, tests)
205+
- **Cache retention**: GitHub Cache for build dependencies and environments
206+
- **Security**: All commits must be signed, untrusted code blocked
207+
- **Parallel execution**: Matrix builds across Python versions and platforms
208+
- **Component isolation**: Each component (core, bindings, pathfinder, python) can be built/released independently
209+
210+
---
211+
212+
<a>1</a>: The `cuda-python` meta package shares the same license and the contributing guidelines as those of `cuda-bindings`.

‎ci/.ci-pipeline-regen.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# CUDA Python CI Pipeline SVG Regeneration Instructions
2+
3+
This file contains the prompt and requirements for regenerating `ci-pipeline.svg` with the same styling and content.
4+
5+
## Styling Requirements
6+
7+
- Hand-drawn Excalidraw-style design with rough, sketchy borders
8+
- Comic Sans MS font family for all text
9+
- Imperfect lines and curves that mimic hand-drawn aesthetics
10+
- Canvas size: 900x800 pixels
11+
- Color scheme:
12+
- Trigger Events: #e8f4fd background, #2196F3 border, #1976D2 text
13+
- Build Stage: #f3e5f5 background, #9C27B0 border, #7B1FA2 text
14+
- Artifact Storage: #fff3e0 background, #FF9800 border, #F57C00 text
15+
- Test Stage: #e8f5e8 background, #4CAF50 border, #388E3C text
16+
- Release Pipeline: #ffebee background, #f44336 border, #D32F2F text
17+
18+
## Content Structure
19+
20+
1. **Title**: "CUDA Python CI Pipeline Flow"
21+
22+
2. **Trigger Events** (top blue box):
23+
- Push to main branch
24+
- Pull request
25+
- Manual workflow dispatch
26+
27+
3. **Build Stage** (purple box):
28+
- Three platform boxes: linux-64 (Self-hosted), linux-aarch64 (Self-hosted), win-64 (GitHub-hosted)
29+
- Details: Python versions 3.9-3.13, CUDA 13.0.0 (build-time)
30+
- Components: cuda-core, cuda-bindings, cuda-pathfinder, cuda-python
31+
32+
4. **Artifact Storage** (orange box):
33+
- GitHub Artifacts box: Wheel files (.whl), Test artifacts, Documentation (30-day retention)
34+
- GitHub Cache box: Mini CTK cache
35+
36+
5. **Test Stage** (green box):
37+
- Three platform boxes: linux-64 (Self-hosted), linux-aarch64 (Self-hosted), win-64 (GitHub-hosted)
38+
- Details: Download wheels from artifacts, Test against multiple CUDA runtime versions, Run Python unit tests, Cython tests, examples
39+
- Artifact Flows (in red text):
40+
• cuda-pathfinder: main → backport
41+
• cuda-bindings: backport → main
42+
43+
6. **Release Pipeline** (red box):
44+
- Three sequential boxes: Validation → Publishing → Documentation
45+
- Validation: Artifact integrity, Git tag verification
46+
- Publishing: PyPI/TestPyPI, Component or all releases
47+
- Documentation: GitHub Pages, Release notes
48+
- Details: Manual workflow dispatch with run ID, Supports individual component or full releases
49+
50+
## Arrow Requirements
51+
52+
- Main flow arrows: Trigger → Build → Artifact → Test → Release
53+
- Additional artifact flow arrows (dashed, orange #FF9800):
54+
- From GitHub Cache (mini CTK) back to Build Stage with "mini CTK reuse" label
55+
- From GitHub Artifacts (wheels) to Release Pipeline with "wheel fetch" label
56+
- **NEW**: From GitHub Cache (mini CTK) to Test Stage with "mini CTK reuse" label
57+
- **NEW**: From GitHub Artifacts (wheels) to Test Stage with "wheel fetch" label
58+
- Arrow marker definition with hand-drawn style (orange arrow heads, not black)
59+
- Use stroke-dasharray="5,3" for artifact flow arrows
60+
61+
## Critical Arrow Positioning Requirements (UPDATED)
62+
63+
**IMPORTANT**: Arrows must NOT overlap with stage boxes. Ensure proper clearance:
64+
65+
1. **Mini CTK reuse arrow** (GitHub Cache → Build Stage):
66+
- Arrow endpoint Y coordinate must be BELOW the Build Stage box edge (y=292)
67+
- Use y=295 or greater for the endpoint to ensure no overlap
68+
- Position "mini CTK reuse" text to the RIGHT of the arrow (not left) for less visual clutter
69+
- Text color should be orange (#FF9800) to match arrow
70+
71+
2. **Wheel fetch arrow** (GitHub Artifacts → Release Pipeline):
72+
- Arrow endpoint Y coordinate must be ABOVE the Release Pipeline box edge (y=652)
73+
- Use y=645 or smaller for the endpoint to provide proper margin
74+
- Position "wheel fetch" text between Test Stage and Release Pipeline boxes
75+
- Text should be to the LEFT of the arrow for better spacing
76+
77+
## Font Size Requirements (UPDATED)
78+
79+
- ALL text labels must use consistent 12pt font size for readability
80+
- No 9pt text - this is too small and hard to read
81+
- Title: 16pt, Stage headers: 14pt, All other text: 12pt
82+
83+
## Key Features
84+
85+
- All boxes use rough, hand-drawn paths (not perfect rectangles)
86+
- Text should be properly sized and positioned within boxes
87+
- Platform boxes within each stage should be clearly separated
88+
- Maintain consistent spacing and alignment
89+
- Orange arrow heads must match the orange arrow color
90+
91+
## Text Positioning
92+
93+
- Use text-anchor="middle" for centered headers
94+
- Use text-anchor="start" for left-aligned bullet points
95+
- Ensure all text fits within their enclosing boxes
96+
- Use transforms for angled text labels on artifact flow arrows
97+
- Artifact flow arrow text positioning is critical - follow positioning requirements above
98+
99+
## Recent Manual Adjustments Applied
100+
101+
- Fixed arrow endpoint positioning to prevent overlap with stage boxes
102+
- Moved mini CTK reuse arrow endpoint from y=285 to y=295
103+
- Moved wheel fetch arrow endpoint from y=650 to y=645
104+
- Repositioned text labels for better visual separation
105+
- Standardized all text to 12pt font size for consistency
106+
- Changed arrow heads from black to orange to match arrow color

0 commit comments

Comments
 (0)