Skip to content

Commit 501e7af

Browse files
vikrantpuppalavikrantpuppala
committed
Migrate CI to databricks-protected runners and route PyPI through JFrog
Protected runners are required for Databricks OSS repos. Add a setup-jfrog composite action (OIDC-based, matching databricks-odbc) that sets PIP_INDEX_URL so all pip/poetry installs go through the JFrog PyPI proxy. Every workflow now runs on the databricks-protected-runner-group with id-token: write for the OIDC exchange. Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
1 parent 4793353 commit 501e7af

File tree

6 files changed

+81
-10
lines changed

6 files changed

+81
-10
lines changed

‎.github/actions/setup-jfrog/action.yml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
name: Setup JFrog OIDC
2+
description: Obtain a JFrog access token via GitHub OIDC and configure pip to use JFrog PyPI proxy
3+
4+
runs:
5+
using: composite
6+
steps:
7+
- name: Get JFrog OIDC token
8+
shell: bash
9+
run: |
10+
set -euo pipefail
11+
ID_TOKEN=$(curl -sLS \
12+
-H "User-Agent: actions/oidc-client" \
13+
-H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
14+
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=jfrog-github" | jq .value | tr -d '"')
15+
echo "::add-mask::${ID_TOKEN}"
16+
ACCESS_TOKEN=$(curl -sLS -XPOST -H "Content-Type: application/json" \
17+
"https://databricks.jfrog.io/access/api/v1/oidc/token" \
18+
-d "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"${ID_TOKEN}\", \"provider_name\": \"github-actions\"}" | jq .access_token | tr -d '"')
19+
echo "::add-mask::${ACCESS_TOKEN}"
20+
if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
21+
echo "FAIL: Could not extract JFrog access token"
22+
exit 1
23+
fi
24+
echo "JFROG_ACCESS_TOKEN=${ACCESS_TOKEN}" >> "$GITHUB_ENV"
25+
echo "JFrog OIDC token obtained successfully"
26+
27+
- name: Configure pip
28+
shell: bash
29+
run: |
30+
set -euo pipefail
31+
echo "PIP_INDEX_URL=https://gha-service-account:${JFROG_ACCESS_TOKEN}@databricks.jfrog.io/artifactory/api/pypi/db-pypi/simple" >> "$GITHUB_ENV"
32+
echo "pip configured to use JFrog registry"

‎.github/workflows/code-coverage.yml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,15 @@ name: Code Coverage
22

33
permissions:
44
contents: read
5+
id-token: write
56

67
on: [pull_request, workflow_dispatch]
78

89
jobs:
910
test-with-coverage:
10-
runs-on: ubuntu-latest
11+
runs-on:
12+
group: databricks-protected-runner-group
13+
labels: linux-ubuntu-latest
1114
environment: azure-prod
1215
env:
1316
DATABRICKS_SERVER_HOSTNAME: ${{ secrets.DATABRICKS_HOST }}
@@ -23,6 +26,8 @@ jobs:
2326
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
2427
with:
2528
fetch-depth: 0
29+
- name: Setup JFrog
30+
uses: ./.github/actions/setup-jfrog
2631
- name: Set up python
2732
id: setup-python
2833
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5

‎.github/workflows/code-quality-checks.yml‎

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,13 @@ on: [pull_request]
44

55
permissions:
66
contents: read
7+
id-token: write
78

89
jobs:
910
run-unit-tests:
10-
runs-on: ubuntu-latest
11+
runs-on:
12+
group: databricks-protected-runner-group
13+
labels: linux-ubuntu-latest
1114
strategy:
1215
matrix:
1316
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
@@ -27,6 +30,8 @@ jobs:
2730
#----------------------------------------------
2831
- name: Check out repository
2932
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
33+
- name: Setup JFrog
34+
uses: ./.github/actions/setup-jfrog
3035
- name: Set up python ${{ matrix.python-version }}
3136
id: setup-python
3237
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
@@ -92,7 +97,9 @@ jobs:
9297
- name: Run tests
9398
run: poetry run python -m pytest tests/unit
9499
run-unit-tests-with-arrow:
95-
runs-on: ubuntu-latest
100+
runs-on:
101+
group: databricks-protected-runner-group
102+
labels: linux-ubuntu-latest
96103
strategy:
97104
matrix:
98105
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
@@ -111,6 +118,8 @@ jobs:
111118
#----------------------------------------------
112119
- name: Check out repository
113120
uses: actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2
121+
- name: Setup JFrog
122+
uses: ./.github/actions/setup-jfrog
114123
- name: Set up python ${{ matrix.python-version }}
115124
id: setup-python
116125
uses: actions/setup-python@e9aba2c848f5ebd159c070c61ea2c4e2b122355e # v2
@@ -179,7 +188,9 @@ jobs:
179188
- name: Run tests
180189
run: poetry run python -m pytest tests/unit
181190
check-linting:
182-
runs-on: ubuntu-latest
191+
runs-on:
192+
group: databricks-protected-runner-group
193+
labels: linux-ubuntu-latest
183194
strategy:
184195
matrix:
185196
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
@@ -189,6 +200,8 @@ jobs:
189200
#----------------------------------------------
190201
- name: Check out repository
191202
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
203+
- name: Setup JFrog
204+
uses: ./.github/actions/setup-jfrog
192205
- name: Set up python ${{ matrix.python-version }}
193206
id: setup-python
194207
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
@@ -232,7 +245,9 @@ jobs:
232245
run: poetry run black --check src
233246

234247
check-types:
235-
runs-on: ubuntu-latest
248+
runs-on:
249+
group: databricks-protected-runner-group
250+
labels: linux-ubuntu-latest
236251
strategy:
237252
matrix:
238253
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
@@ -242,6 +257,8 @@ jobs:
242257
#----------------------------------------------
243258
- name: Check out repository
244259
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
260+
- name: Setup JFrog
261+
uses: ./.github/actions/setup-jfrog
245262
- name: Set up python ${{ matrix.python-version }}
246263
id: setup-python
247264
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5

‎.github/workflows/daily-telemetry-e2e.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,13 @@ on:
1414

1515
permissions:
1616
contents: read
17+
id-token: write
1718

1819
jobs:
1920
telemetry-e2e-tests:
20-
runs-on: ubuntu-latest
21+
runs-on:
22+
group: databricks-protected-runner-group
23+
labels: linux-ubuntu-latest
2124
environment: azure-prod
2225

2326
env:
@@ -33,7 +36,10 @@ jobs:
3336
#----------------------------------------------
3437
- name: Check out repository
3538
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
36-
39+
40+
- name: Setup JFrog
41+
uses: ./.github/actions/setup-jfrog
42+
3743
- name: Set up python
3844
id: setup-python
3945
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5

‎.github/workflows/dco-check.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@ permissions:
88

99
jobs:
1010
check:
11-
runs-on: ubuntu-latest
11+
runs-on:
12+
group: databricks-protected-runner-group
13+
labels: linux-ubuntu-latest
1214
steps:
1315
- name: Check for DCO
1416
id: dco-check

‎.github/workflows/integration.yml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,13 @@ on:
88

99
permissions:
1010
contents: read
11+
id-token: write
1112

1213
jobs:
1314
run-non-telemetry-tests:
14-
runs-on: ubuntu-latest
15+
runs-on:
16+
group: databricks-protected-runner-group
17+
labels: linux-ubuntu-latest
1518
environment: azure-prod
1619
env:
1720
DATABRICKS_SERVER_HOSTNAME: ${{ secrets.DATABRICKS_HOST }}
@@ -25,6 +28,8 @@ jobs:
2528
#----------------------------------------------
2629
- name: Check out repository
2730
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
31+
- name: Setup JFrog
32+
uses: ./.github/actions/setup-jfrog
2833
- name: Set up python
2934
id: setup-python
3035
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
@@ -71,7 +76,9 @@ jobs:
7176
-n auto
7277
7378
run-telemetry-tests:
74-
runs-on: ubuntu-latest
79+
runs-on:
80+
group: databricks-protected-runner-group
81+
labels: linux-ubuntu-latest
7582
needs: run-non-telemetry-tests # Run after non-telemetry tests complete
7683
environment: azure-prod
7784
env:
@@ -83,6 +90,8 @@ jobs:
8390
steps:
8491
- name: Check out repository
8592
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
93+
- name: Setup JFrog
94+
uses: ./.github/actions/setup-jfrog
8695
- name: Set up python
8796
id: setup-python
8897
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5

0 commit comments

Comments
 (0)