Merge pull request #3205 from cal-itp/3200-streamline-k8s-deploy

Streamline k8s deploy process

.github/workflows/README.md

@@ -13,33 +13,18 @@ While we're using GCP Composer, "deployment" of Airflow consists of two parts:
 
 This workflow builds a static website from the Svelte app and deploys it to Netlify.
 
+## deploy-kubernetes.yml
+
+This workflow deploys changes to the production Kubernetes cluster when they get merged into the `main` branch.
+
 ## build-\*.yml workflows
 
 Workflows prefixed with `build-` generally lint, test, and (usually) publish either a Python package or a Docker image.
 
-## service-\*.yml workflows
+## preview-\*.yml workflows
 
-Workflows prefixed with `service-` deal with Kubernetes deployments.
+Workflows prefixed with `preview-` deal with generating previews for pull request changes
 
-- `service-release-candidate.yml` creates candidate branches, using [hologit](https://github.com/JarvusInnovations/hologit) to bring in external Helm charts and remove irrelevant (i.e. non-infra) code
-- `service-release-diff.yml` renders kubectl diffs on PRs targeting release branches
-- `service-release-channel.yml` deploys to a given channel (i.e. environment) on updates to a release branch
+- `preview-kubernetes.yml` renders kubectl diffs on PRs changing cluster content
 
 Some of these workflows use hologit or invoke. See the READMEs in [.holo](../../.holo) and [ci](../../ci) for documentation regarding hologit and invoke, respectively.
-
-## GitOps
-
-The workflows described above also define their triggers. In general, developer workflows should follow these steps.
-
-1. Check out a feature branch
-2. Put up a PR for that feature branch, targeting `main`
-   - `service-release-candidate` will run and create a remote branch named `candidate/<feature-branch-name`
-3. Create and merge a PR from the candidate branch to `releases/test`
-   - `service-release-diff` will run on the PR and print the expected changes
-   - `service-release-channel` will run on merge (i.e. push on `releases/test`) and deploy
-4. Merge the original PR
-   - `service-release-candidate` will then update the remote `candidates/main` branch
-5. Create and merge a PR from `candidates/main` to `releases/prod`
-   - `service-release-channel` will run and deploy to `prod` this time
-
-Note: One alternative would be to use `candidates/main` to deploy into both `test` and `prod`. This is very possible but can be a bit annoying if GitHub is configured to delete branches on merge and the `cleanup-release-candidates` action then deletes `candidates/main` after it has been merged into `releases/test`.

.github/workflows/deploy-kubernetes.yml

@@ -0,0 +1,65 @@
+name: Generate diff of Kubernetes changes
+
+on:
+  push:
+    branches:
+      - 'main'
+    paths:
+      - '.holo/config.toml'
+      - '.holo/sources/jarvus-cluster-template.toml'
+      - '.holo/branches/releases/**'
+      - '.github/workflows/*-kubernetes.yml'
+      - 'ci/**'
+      - 'kubernetes/apps/**'
+      - 'kubernetes/system/**'
+
+env:
+  GITHUB_TOKEN: ${{ secrets.GH_ACTIONS_TOKEN }}
+
+jobs:
+  preview-kubernetes:
+    runs-on: ubuntu-latest
+    env:
+      CLOUDSDK_CORE_PROJECT: cal-itp-data-infra
+      GKE_NAME: data-infra-apps
+      GKE_REGION: us-west1
+      USE_GKE_GCLOUD_AUTH_PLUGIN: True
+    steps:
+      # Setup
+      - uses: actions/checkout@v4
+
+      - uses: 'google-github-actions/auth@v2'
+        with:
+          credentials_json: '${{ secrets.GCP_SA_KEY }}'
+      - uses: google-github-actions/setup-gcloud@v2
+      - run: gcloud components install gke-gcloud-auth-plugin
+      - uses: google-github-actions/get-gke-credentials@v1
+        with:
+          cluster_name: ${{ env.GKE_NAME }}
+          location: ${{ env.GKE_REGION }}
+      - run: curl -sSL https://install.python-poetry.org | python -
+      - name: Set up hologit
+        env:
+          BIO_RELEASE: 1.6.821
+        run: |
+          curl -LO "https://github.com/biome-sh/biome/releases/download/v${BIO_RELEASE}/bio-${BIO_RELEASE}-x86_64-linux.tar.gz"
+          tar xzvf "bio-${BIO_RELEASE}-x86_64-linux.tar.gz"
+          sudo mv bio /usr/local/bin/bio
+          sudo bio pkg install --binlink jarvus/hologit
+
+          git config user.name "Github Action $GITHUB_JOB"
+          git config user.email "$(whoami)@$(uname -n)"
+
+      # Render Kubernetes content with parent underlay and checkout
+      - run: git holo project kubernetes-workspace --commit-to=kubernetes
+      - run: git checkout kubernetes
+
+      # Diff and write back to PR
+      - id: diff
+        name: Run poetry invoke
+        shell: bash
+        working-directory: ci
+        run: |
+          poetry install
+          poetry run invoke secrets -f "./channels/prod.yaml"
+          poetry run invoke release -f "./channels/prod.yaml"

.github/workflows/service-release-diff.yml → .github/workflows/preview-kubernetes.yml

@@ -1,19 +1,23 @@
-name: Show diff for release channel
+name: Generate diff of Kubernetes changes
 
 on:
   pull_request:
     branches:
-      - 'releases/*'
-    types:
-      - opened
-      - synchronize
-      - reopened
+      - 'main'
+    paths:
+      - '.holo/config.toml'
+      - '.holo/sources/jarvus-cluster-template.toml'
+      - '.holo/branches/releases/**'
+      - '.github/workflows/*-kubernetes.yml'
+      - 'ci/**'
+      - 'kubernetes/apps/**'
+      - 'kubernetes/system/**'
 
 env:
   GITHUB_TOKEN: ${{ secrets.GH_ACTIONS_TOKEN }}
 
 jobs:
-  release:
+  preview-kubernetes:
     runs-on: ubuntu-latest
     env:
       CLOUDSDK_CORE_PROJECT: cal-itp-data-infra
@@ -22,9 +26,7 @@ jobs:
       USE_GKE_GCLOUD_AUTH_PLUGIN: True
     steps:
       # Setup
-      - uses: actions/checkout@v2
-        with:
-          fetch-depth: 0
+      - uses: actions/checkout@v4
 
       - uses: 'google-github-actions/auth@v2'
         with:
@@ -36,17 +38,30 @@ jobs:
           cluster_name: ${{ env.GKE_NAME }}
           location: ${{ env.GKE_REGION }}
       - run: curl -sSL https://install.python-poetry.org | python -
+      - name: Set up hologit
+        env:
+          BIO_RELEASE: 1.6.821
+        run: |
+          curl -LO "https://github.com/biome-sh/biome/releases/download/v${BIO_RELEASE}/bio-${BIO_RELEASE}-x86_64-linux.tar.gz"
+          tar xzvf "bio-${BIO_RELEASE}-x86_64-linux.tar.gz"
+          sudo mv bio /usr/local/bin/bio
+          sudo bio pkg install --binlink jarvus/hologit
+
+          git config user.name "Github Action $GITHUB_JOB"
+          git config user.email "$(whoami)@$(uname -n)"
+
+      # Render Kubernetes content with parent underlay and checkout
+      - run: git holo project kubernetes-workspace --commit-to=kubernetes
+      - run: git checkout kubernetes
 
       # Diff and write back to PR
       - id: diff
         name: Run poetry invoke
         shell: bash
         working-directory: ci
         run: |
-          export RELEASE_CHANNEL=${GITHUB_BASE_REF#releases/}
-          printf 'WORKFLOW: service-release-diff; RELEASE_CHANNEL=%s\n' "$RELEASE_CHANNEL"
           poetry install
-          poetry run invoke diff -f "./channels/$RELEASE_CHANNEL.yaml" --outfile=diff.txt
+          poetry run invoke diff -f "./channels/prod.yaml" --outfile=diff.md
 
       - uses: peter-evans/find-comment@v2
         id: fc
@@ -58,5 +73,5 @@ jobs:
         with:
           comment-id: ${{ steps.fc.outputs.comment-id }}
           issue-number: ${{ github.event.number }}
-          body-file: "ci/diff.txt"
+          body-file: "ci/diff.md"
           edit-mode: replace

.holo/README.md

@@ -10,13 +10,9 @@ hologit allows:
 3. Applying transformations to files as part of #1
    - These transformations are called "lenses"
 
-In this repository, we declare one holobranch named [release-candidate](../branches/release-candidate).
-By projecting this holobranch in GitHub Actions, individual "candidate" branches end up containing
-only the code relevant to infra/Kubernetes as well as Kubernetes code from the upstream [cluster-template](https://github.com/JarvusInnovations/cluster-template)
-repository. Then, a PR from a `candidate/<some-branch>` to `releases/<env>` (such as `releases/test`) will only show changes/content
-relevant to infra in addition to `releases/*` branches only ever containing infra code. For example:
+In this repository, we declare one holobranch named [kubernetes-workspace](../branches/kubernetes-workspace).
+By projecting this holobranch in GitHub Actions, a tree containing only the code relevant to infra/Kubernetes
+as well as Kubernetes code from the upstream [cluster-template](https://github.com/JarvusInnovations/cluster-template)
+repository is generated.
 
-1. Create a [PR making an infra-related change](https://github.com/cal-itp/data-infra/pull/2828)
-2. Create and merge a [PR to deploy a candidate branch to test](https://github.com/cal-itp/data-infra/pull/2829)
-3. Merge the PR from #1
-4. After merge, [PR to deploy the main candidate branch to prod](https://github.com/cal-itp/data-infra/pull/2832)
+See [`ci/README.md`](../ci/README.md) for details on the pull request workflow for previewing and deploying Kubernetes changes.

.holo/branches/release-candidate/_data-infra.toml → .holo/branches/kubernetes-workspace/_data-infra.toml

@@ -1,3 +1,3 @@
 [holomapping]
-files = [ "ci/**", "kubernetes/apps/**", "kubernetes/system/**", ".github/workflows/service-*" ]
+files = [ "ci/**", "kubernetes/apps/**", "kubernetes/system/**", ".github/workflows/*-kubernetes.yml" ]
 before = "*"

ci/channels/prod.yaml

@@ -26,6 +26,8 @@ calitp:
       namespace: jupyterhub
       helm_name: jupyterhub
       helm_chart: kubernetes/apps/charts/jupyterhub
+      secret_helm_values:
+        - jupyterhub_jupyterhub-sensitive-helm-values
       secrets:
         - jupyterhub_jupyterhub-gcloud-service-key
         - jupyterhub_jupyterhub-github-config