Skip to content

Commit 1bc0439

Browse files
msarahanmsarahan
committed
flesh out 3-doc readme, copy img from CCCL
1 parent ad07fbb commit 1bc0439

9 files changed

+486
-213
lines changed

‎DEVELOP.md

Lines changed: 157 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,30 @@
11
# Contributing to the RAPIDS devcontainers
22

3+
This document contains implementation details and design overviews for
4+
[rapidsai/devcontainers](https://github.com/rapidsai/devcontainers) as a
5+
centralized source of common scripts and patterns for providing devcontainers.
6+
7+
For the user-level overview providing instructions on how to use the
8+
devcontainer as a development environment, see
9+
[USAGE_IN_PROJECT.md](USAGE_IN_PROJECT.md)
10+
11+
For the project maintainer-level overview providing instructions on how to add
12+
and change .devcontainer.json to suit your project, see [USAGE.md](USAGE.md)
13+
14+
The code in this repository fits into a few main categories:
15+
1. Features
16+
1. GitHub Actions automations
17+
1. Scripts
18+
1. matrix.yml
19+
1. Dockerfiles
20+
321
## Features
422

523
From the official devcontainer [documentation on Features](https://containers.dev/implementors/features/):
6-
> Development container "Features" are self-contained, shareable units of installation code and development container configuration.
7-
8-
In short, a "feature" is a layer in a Dockerfile which encapsulates a reusable
9-
bit of logic executed when building a Docker image. It is not a docker layer to
10-
put on top of or copied into other layers. It is the script that creates a
11-
layer.
24+
> Development container "Features" are self-contained, shareable units of installation code and development container configuration. Each "feature" specified becomes a `RUN` statement in a temporary Dockerfile,
25+
and as such each "feature" results in an image layer.
1226

13-
This repository defines features to install the following dev tools, compilers, and SDKs:
27+
This repository defines `features` to install the following dev tools, compilers, and SDKs:
1428

1529
* [CMake](features/src/cmake/)
1630
* [CUDA Toolkit](features/src/cuda/)
@@ -27,8 +41,141 @@ This repository defines features to install the following dev tools, compilers,
2741
* [devcontainer-utils](features/src/utils/)
2842
* [rapids-build-utils](features/src/rapids-build-utils/)
2943

30-
These scripts assume that apt utilities are available, and thus only run on debian-based images.
44+
These scripts assume that apt utilities are available, and thus generally only
45+
run on debian-based images.
46+
47+
### Utility features
48+
49+
A few of the features install custom tools for the devcontainer ecosystem here,
50+
rather than just installation scripts of external tools and libraries. A brief
51+
overview of responsibilities for these features follows.
52+
53+
#### Rapids-build-utils
54+
55+
Most of the scripts here serve to prepare the devcontainer prior to use, but you may use them to update
56+
the devcontainer after adding to your container's /opt/rapids-build-utils/manifest.yml file to add new projects.
57+
58+
If you are wondering where a command or behavior in a devcontainer is coming
59+
from, this is a good place to start.
60+
61+
These scripts are installed by
62+
[`install.sh`](./features/src/rapids-build-utils/install.sh), which creates
63+
aliases for the .sh scripts. if you see `rapids-*` for a script or command name
64+
in the devcontainer, look in
65+
[`install.sh`](./features/src/rapids-build-utils/install.sh) to see how it is
66+
mapped back to one of these scripts.
67+
68+
* [manifest.yaml](./features/src/rapids-build-utils/opt/rapids-build-utils/manifest.yaml): This enumerates where projects should be cloned from and how they depend on each other. It is used to generate build scripts. If you project is not in manifest.yaml, you will not get build scripts generated in your devcontainer. Refer to [docs on manifest.yaml](./USAGE.md#generated-build-scripts)
69+
* [generate-scripts.sh](./features/src/rapids-build-utils/opt/rapids-build-utils/bin/generate-scripts.sh): generate `build-*`, `clone-*`, etc. scripts
70+
* [make-pip-env.sh](./features/src/rapids-build-utils/opt/rapids-build-utils/bin/make-pip-env.sh) and [make-conda-env.sh](./features/src/rapids-build-utils/opt/rapids-build-utils/bin/make-conda-env.sh):
71+
creating pip and conda python virtual environments
72+
* [pull-repositories.sh](./features/src/rapids-build-utils/opt/rapids-build-utils/bin/pull-repositories.sh) and [push-repositories.sh](./features/src/rapids-build-utils/opt/rapids-build-utils/bin/push-repositories.sh) help you manage git operations on multiple repos that you may have cloned
73+
* [update-content-command.sh](./features/src/rapids-build-utils/opt/rapids-build-utils/bin/update-content-command.sh): calls `rapids-generate-script` and `rapids-make-vscode-workspace`. Called by VS Code in the [`postAttachCommand`](https://containers.dev/implementors/json_reference/#lifecycle-scripts), which is a reliable hook when the project is reopened or its configuration is changed.
74+
75+
There are more scripts here, dealing with CMake variable parsing and
76+
pass-through, python package dependency installation, and more.
77+
78+
#### devcontainers-utils
79+
80+
These scripts handle mostly git-related configuration, setting up SSH deploy
81+
keys, GitHub authorization, and the vault setup for S3. The commands here are
82+
prefixed with `devcontainer-` during installation, so if you see that prefix,
83+
look in here.
84+
85+
## Github Actions automations
86+
87+
Github Actions runs the build matrix of the many base images define in [matrix.yml](./matrix.yml).
88+
These actions are broken up into many reusable pieces. The image build jobs start in
89+
[release.yml](./.github/workflows/release.yml).
90+
91+
```mermaid
92+
flowchart TD
93+
subgraph Legend
94+
workflow
95+
action(action)
96+
script{{script}}
97+
end
98+
subgraph Image build Github Actions flowchart
99+
release.yml --> release-features.yml[Publish features]
100+
release-features.yml --> image-matrix(Determine image matrix)
101+
image-matrix --> write-matrix{{Call feature-matrix/action.sh}}
102+
write-matrix --> build-test-and-push-linux-image.yml
103+
write-matrix --> build-test-and-push-windows-image.yml
104+
build-test-and-push-linux-image.yml --> write-json(Write .devcontainer.json)
105+
write-json --> write-script{{Call devcontainer-json/action.sh}}
106+
build-test-and-push-windows-image.yml --> build-windows(Build windows image)
107+
write-script --> build-linux-image(Build linux limage)
108+
end
109+
```
110+
111+
These are divided into 3 categories:
112+
* Workflows are `.yml` files in `.github/workflows`
113+
* Actions are folders in `.github/actions`. One folder per action. Actions must have a `action.yml` file, but may also have accompanying shell scripts.
114+
* Scripts are the shell scripts that are in some of the actions. They are broken out in this diagram to help show where the functionality actually lives.
115+
116+
## Base images (matrix.yml)
117+
118+
Base images are composed from individual features in [matrix.yml](./matrix.yml)
119+
using [YAML
120+
anchors](https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/).
121+
These get built on Github Actions as described
122+
[above](#github-actions-automations) The devcontainers repo does not contain
123+
scripts to facilitate building one particular image. Some downstream repos, such
124+
as CCCL, have such scripts for their reduced subspace of the matrix. CCCL's
125+
example is their
126+
[make_devcontainers.sh](https://github.com/NVIDIA/cccl/blob/main/.devcontainer/make_devcontainers.sh)
127+
script.
128+
129+
## Dockerfiles
130+
131+
Dockerfiles do not play much role in this scheme. They serve to [set a few global
132+
variables and extend from the base image](./.devcontainer/rapids.Dockerfile). If you think you need to modify a
133+
Dockerfile, make sure that your goal wouldn't be better achieved by adding or
134+
changing a feature script instead.
135+
136+
## Build caching with `sccache`
137+
138+
The devcontainers configure CMake to use
139+
[sccache](https://github.com/mozilla/sccache) as C, C++, CUDA, and Rust compiler
140+
launchers. Refer to the [sccache
141+
docs](https://github.com/mozilla/sccache/tree/main/docs) for configuring the
142+
various storage back-ends.
143+
144+
### Build caching with private S3 buckets
145+
146+
You can use a private S3 bucket as the `sccache` storage back-end.
147+
148+
If you're using a [GitHub action](https://github.com/aws-actions/configure-aws-credentials) to assume AWS roles in CI, or are comfortable distributing and managing S3 credentials, you can define the `SCCACHE_BUCKET`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY` variables in the container environment.
149+
150+
### Using GitHub OAuth to issue S3 credentials via Hashicorp Vault
151+
152+
The [`devcontainer-utils`](features/src/utils/) feature includes a `devcontainer-utils-vault-s3-init` script that uses GitHub OAuth and Hashicorp Vault to issue temporary S3 credentials to authorized users.
153+
154+
> **NOTE:** This script runs in the devcontainer's [`postAttachCommand`](https://containers.dev/implementors/json_reference/#lifecycle-scripts), but it does nothing unless `SCCACHE_BUCKET` and `VAULT_HOST` are in the container environment.
155+
156+
The `devcontainer-utils-vault-s3-init` script performs the following actions, exiting early if any step is unsuccessful:
157+
158+
1. Log in via the [GitHub CLI](https://cli.github.com/)
159+
2. Authenticate via [Vault's GitHub auth method](https://developer.hashicorp.com/vault/docs/auth/github#authentication)
160+
3. Use Vault to [generate temporary AWS credentials](https://developer.hashicorp.com/vault/api-docs/secret/aws#generate-credentials)
161+
4. Store results in `~/.aws` and install crontab to re-authenticate
162+
163+
The above steps can be customized via the following environment variables:
164+
```
165+
# The hostname of the Vault instance to use
166+
VAULT_HOST="https://vault.ops.k8s.rapids.ai"
167+
168+
# List of GitHub organizations for which Vault can generate credentials.
169+
# The scripts assumes the Vault instance exposes an authentication endpoint
170+
# for each org at `$VAULT_HOST/v1/auth/github-$org/login`.
171+
# https://developer.hashicorp.com/vault/docs/auth/github#authentication
172+
VAULT_GITHUB_ORGS="nvidia nv-morpheus nv-legate rapids"
31173
32-
## Base images
174+
# The TTL for the generated AWS credentials
175+
VAULT_S3_TTL=43200
33176
34-
Base images are composed in [matrix.yml](./matrix.yml) using [YAML anchors](https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/). These get built on Github Actions ([release.yml](./.github/workflows/release.yml) and [test.yml](.github/workflows/test.yml))
177+
# The URI to the Vault API that generates AWS credentials
178+
# The full URL expands to `$VAULT_HOST/$VAULT_S3_URI?ttl=$VAULT_S3_TTL`
179+
# https://developer.hashicorp.com/vault/api-docs/secret/aws#generate-credentials
180+
VAULT_S3_URI="v1/aws/creds/devs"
181+
```

‎README.md

Lines changed: 16 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,23 @@
11
# RAPIDS [devcontainers](https://containers.dev/)
22

3-
This repository contains features and workflows for building development
4-
containers to support local dev and CI for NVIDIA
5-
[RAPIDS](https://github.com/rapidsai), [CCCL](https://github.com/nvidia/cccl),
6-
and [Legate](https://github.com/nv-legate).
3+
This repository contains centralized features and workflows for building
4+
development containers ([devcontainers](https://containers.dev/)) to support
5+
local dev and CI for projects in NVIDIA [RAPIDS](https://github.com/rapidsai),
6+
[CCCL](https://github.com/nvidia/cccl), and
7+
[Legate](https://github.com/nv-legate).
8+
79
[Devcontainers](https://containers.dev/) are an open standard for specifying the
8-
creation and execution of Docker containers for developing a codebase. It's like
9-
using a docker image to develop, but there's some extra configuration and
10-
installation that can be done. It also provides some alternative ways of
11-
composing functionality and configuration that can augment Docker's
12-
capabilities.
10+
creation and execution of Docker containers for developing a codebase.
11+
12+
Downstream repositories that utilize devcontainers use both the `feature`
13+
scripts that install software, as well as docker images that serve to cache sets
14+
of installed software. These images serve as base images for the devcontainers
15+
specified in the downstream repositories.
1316

14-
In addition to scripts that set up the devcontainer environment for things like GitHub auth, this repo contains reusable scripts to install software in arbitrary containers, aiding in composition and code sharing. A "feature" in VSCode terms refers to these installation scripts. The script for each feature runs when creating the devcontainer.
17+
## Usage
1518

16-
We've chosen to use a monorepo for the features here, but it is similar in spirit to the official [devcontainers/features](https://github.com/devcontainers/features) and [devcontainers/images](https://github.com/devcontainers/images) repositories.
19+
### [Using devcontainers to provide a build environment on a project](./USAGE_IN_PROJECT.md)
1720

18-
## For details on using the RAPIDS devcontainers, see [`USAGE.md`](USAGE.md).
21+
### [Setting up and maintaining devcontainer configuration in other projects](./USAGE.md)
1922

20-
## For details on contributing to this repository, see [`DEVELOP.md`](DEVELOP.md).
21-
## See the list of `rapidsai/devcontainers` tags [on DockerHub](https://hub.docker.com/r/rapidsai/devcontainers/tags). These tags are used as base images in devcontainers, and aren't really meant to be used directly.
23+
### [Developing the centralized `feature` scripts and base images in this repository](DEVELOP.md).

0 commit comments

Comments
 (0)