-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/dockerfile #156
Feature/dockerfile #156
Changes from 32 commits
9b9e740
a919051
139529d
64c9d0b
8504080
43096ba
7112c69
591b1b2
8095486
8b5a6d0
28432b7
6b91971
6f90c77
1bd421d
84b713d
7e1549d
9df1bd6
9b78d68
3b95300
e64cc23
d8ba325
cdd8486
624eb9f
e0b4506
66d0aa8
84e2137
85c84b2
60c1e67
d49ad5c
a9fd875
31e7c4a
8d4a071
5a9ecaa
8452f00
de4d204
6318154
be48412
0f98bbf
d1a4ac9
9dc5d1b
8afce36
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,7 +14,7 @@ | |
# limitations under the License. | ||
# | ||
|
||
name: Build | ||
name: Build all modules | ||
|
||
on: | ||
pull_request: | ||
|
@@ -23,9 +23,26 @@ on: | |
types: [ opened, synchronize, reopened ] | ||
|
||
jobs: | ||
test: | ||
build-agent-and-model: | ||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: | ||
scala: [2.11.12, 2.12.18, 2.13.11] | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
- uses: coursier/cache-action@v5 | ||
|
||
- name: Setup Scala | ||
uses: olafurpg/setup-scala@v10 | ||
with: | ||
java-version: "[email protected]" | ||
|
||
- name: Build and run tests | ||
run: sbt "project agent" ++${{matrix.scala}} test doc "project model" ++${{matrix.scala}} test doc | ||
|
||
build-server: | ||
runs-on: ubuntu-latest | ||
services: | ||
postgres: | ||
image: postgres:15 | ||
|
@@ -39,14 +56,9 @@ jobs: | |
--health-retries 5 | ||
ports: | ||
- 5432:5432 | ||
|
||
strategy: | ||
fail-fast: false | ||
lsulak marked this conversation as resolved.
Show resolved
Hide resolved
|
||
matrix: | ||
scala: [2.11.12, 2.12.18, 2.13.11] | ||
|
||
name: Scala ${{matrix.scala}} | ||
|
||
scala: [2.13.11] | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
|
@@ -55,10 +67,10 @@ jobs: | |
- name: Setup Scala | ||
uses: olafurpg/setup-scala@v10 | ||
with: | ||
java-version: "adopt@1.8" | ||
java-version: "adopt@1.11.0-11" | ||
|
||
- name: Prepare testing database | ||
run: sbt flywayMigrate | ||
|
||
- name: Build and run tests | ||
run: sbt ++${{matrix.scala}} test doc | ||
run: sbt "project server" ++${{matrix.scala}} test doc |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# | ||
# Copyright 2021 ABSA Group Limited | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
name: Docker Image CI | ||
lsulak marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
on: | ||
pull_request: | ||
branches: [ master ] | ||
types: [ opened, edited, synchronize, reopened ] | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-latest | ||
|
||
services: | ||
postgres: | ||
image: postgres:15 | ||
env: | ||
POSTGRES_PASSWORD: postgres | ||
POSTGRES_DB: atum_db | ||
options: >- | ||
--health-cmd pg_isready | ||
--health-interval 10s | ||
--health-timeout 5s | ||
--health-retries 5 | ||
ports: | ||
- 5432:5432 | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- name: Setup Scala environment | ||
uses: olafurpg/setup-scala@v14 | ||
with: | ||
java-version: "[email protected]" | ||
|
||
- name: Prepare testing database | ||
run: sbt flywayMigrate | ||
|
||
- name: Run sbt assembly | ||
run: sbt server/assembly | ||
|
||
- name: Build the Docker image | ||
run: | | ||
cd server | ||
docker buildx build --platform linux/amd64 -t absaoss/atum-service:latest --build-arg CONFIG=./src/main/resources/reference.conf --build-arg SSL=false ./ | ||
|
||
- name: Run the Docker image and execute HTTP call | ||
run: | | ||
docker run -d -p 8080:8080 absaoss/atum-service:latest | ||
docker container ls | ||
sleep 15 # Wait for the service to start | ||
curl -Lf http://127.0.0.1:8080/docs | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess that if cURL fails, this step fails? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am going to remove this file entirely for now. It turns out that you can't establish a connection between a container running in one of the steps of your pipeline and the postgres service. Basically we would have to first deploy the image some place and then fetch it in the pipeline from registry in order to be able to start it alongside the postgres service in the same network. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,7 +14,7 @@ | |
# limitations under the License. | ||
# | ||
|
||
name: JaCoCo report | ||
name: JaCoCo report agent and module | ||
|
||
on: | ||
pull_request: | ||
|
@@ -24,8 +24,6 @@ on: | |
env: | ||
scalaLong12: 2.12.18 | ||
scalaShort12: "2.12" | ||
scalaLong13: 2.13.11 | ||
scalaShort13: "2.13" | ||
overall: 80.0 | ||
changed: 80.0 | ||
|
||
|
@@ -34,45 +32,19 @@ jobs: | |
name: Build and test | ||
runs-on: ubuntu-latest | ||
|
||
services: | ||
postgres: | ||
image: postgres:15 | ||
env: | ||
POSTGRES_PASSWORD: postgres | ||
POSTGRES_DB: atum_db | ||
options: >- | ||
--health-cmd pg_isready | ||
--health-interval 10s | ||
--health-timeout 5s | ||
--health-retries 5 | ||
ports: | ||
- 5432:5432 | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v4 | ||
- name: Setup Scala | ||
uses: olafurpg/setup-scala@v14 | ||
with: | ||
java-version: "[email protected]" | ||
- name: Prepare testing database | ||
run: sbt flywayMigrate | ||
- name: Build and run tests | ||
continue-on-error: true | ||
id: jacocorun | ||
run: sbt jacoco | ||
# server module code coverage | ||
- name: Add coverage to PR | ||
if: steps.jacocorun.outcome == 'success' | ||
id: jacoco-server | ||
uses: madrapps/[email protected] | ||
with: | ||
paths: ${{ github.workspace }}/server/target/jvm-${{ env.scalaShort13 }}/jacoco/report/jacoco.xml | ||
token: ${{ secrets.GITHUB_TOKEN }} | ||
min-coverage-overall: ${{env.overall }} | ||
min-coverage-changed-files: ${{ env.changed }} | ||
title: JaCoCo server module code coverage report - scala ${{ env.scalaLong13 }} | ||
update-comment: true | ||
run: | | ||
sbt "project agent; jacoco" | ||
sbt "project model; jacoco" | ||
# agent module code coverage | ||
- name: Add coverage to PR | ||
if: steps.jacocorun.outcome == 'success' | ||
|
@@ -102,8 +74,6 @@ jobs: | |
- name: Get the Coverage info | ||
if: steps.jacocorun.outcome == 'success' | ||
run: | | ||
echo "Total sever module coverage ${{ steps.jacoco-server.outputs.coverage-overall }}" | ||
echo "Changed Files coverage ${{ steps.jacoco-server.outputs.coverage-changed-files }}" | ||
echo "Total agent module coverage ${{ steps.jacoco-agent.outputs.coverage-overall }}" | ||
echo "Changed Files coverage ${{ steps.jacoco-agent.outputs.coverage-changed-files }}" | ||
echo "Total model module coverage ${{ steps.jacoco-model.outputs.coverage-overall }}" | ||
|
@@ -114,7 +84,6 @@ jobs: | |
with: | ||
script: | | ||
const coverageCheckFailed = | ||
Number('${{ steps.jacoco-server.outputs.coverage-changed-files }}') < Number('${{ env.changed }}') || | ||
Number('${{ steps.jacoco-agent.outputs.coverage-changed-files }}') < Number('${{ env.changed }}') || | ||
Number('${{ steps.jacoco-model.outputs.coverage-changed-files }}') < Number('${{ env.changed }}'); | ||
if (coverageCheckFailed) { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
# | ||
# Copyright 2021 ABSA Group Limited | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
name: JaCoCo report server | ||
|
||
on: | ||
pull_request: | ||
branches: [ master ] | ||
types: [ opened, edited, synchronize, reopened ] | ||
|
||
env: | ||
scalaLong13: 2.13.11 | ||
scalaShort13: "2.13" | ||
overall: 80.0 | ||
changed: 80.0 | ||
|
||
jobs: | ||
test: | ||
name: Build and test | ||
runs-on: ubuntu-latest | ||
|
||
services: | ||
postgres: | ||
image: postgres:15 | ||
env: | ||
POSTGRES_PASSWORD: postgres | ||
POSTGRES_DB: atum_db | ||
options: >- | ||
--health-cmd pg_isready | ||
--health-interval 10s | ||
--health-timeout 5s | ||
--health-retries 5 | ||
ports: | ||
- 5432:5432 | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v4 | ||
- name: Setup Scala | ||
uses: olafurpg/setup-scala@v14 | ||
with: | ||
java-version: "[email protected]" | ||
- name: Prepare testing database | ||
run: sbt flywayMigrate | ||
- name: Build and run tests | ||
continue-on-error: true | ||
id: jacocorun | ||
run: sbt "project server; jacoco" | ||
# server module code coverage | ||
- name: Add coverage to PR | ||
if: steps.jacocorun.outcome == 'success' | ||
id: jacoco-server | ||
uses: madrapps/[email protected] | ||
with: | ||
paths: ${{ github.workspace }}/server/target/jvm-${{ env.scalaShort13 }}/jacoco/report/jacoco.xml | ||
token: ${{ secrets.GITHUB_TOKEN }} | ||
min-coverage-overall: ${{env.overall }} | ||
min-coverage-changed-files: ${{ env.changed }} | ||
title: JaCoCo server module code coverage report - scala ${{ env.scalaLong13 }} | ||
update-comment: true | ||
- name: Get the Coverage info | ||
if: steps.jacocorun.outcome == 'success' | ||
run: | | ||
echo "Total sever module coverage ${{ steps.jacoco-server.outputs.coverage-overall }}" | ||
echo "Changed Files coverage ${{ steps.jacoco-server.outputs.coverage-changed-files }}" | ||
- name: Fail PR if changed files coverage is less than ${{ env.changed }}% | ||
if: steps.jacocorun.outcome == 'success' | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
const coverageCheckFailed = | ||
Number('${{ steps.jacoco-server.outputs.coverage-changed-files }}') < Number('${{ env.changed }}'); | ||
if (coverageCheckFailed) { | ||
core.setFailed('Changed files coverage is less than ${{ env.changed }}%!'); | ||
} | ||
- name: Edit JaCoCo comments on build failure | ||
if: steps.jacocorun.outcome != 'success' | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
const issue_number = context.issue.number; | ||
const owner = context.repo.owner; | ||
const repo = context.repo.repo; | ||
const jacocoReportRegExp = /^### JaCoCo .* code coverage report .*/; | ||
|
||
const comments = await github.rest.issues.listComments({ | ||
owner, | ||
repo, | ||
issue_number, | ||
}); | ||
|
||
for (const comment of comments.data) { | ||
const lines = comment.body.split('\n'); | ||
if (lines.length > 0 && jacocoReportRegExp.test(lines[0])) { | ||
await github.rest.issues.updateComment({ | ||
owner, | ||
repo, | ||
comment_id: comment.id, | ||
body: lines[0] + "\n\n### Build Failed", | ||
}); | ||
} | ||
} | ||
|
||
core.setFailed('JaCoCo test coverage report generation failed, and related PR comments were updated.'); |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -88,3 +88,6 @@ _site | |
utils/resources/*.conf | ||
|
||
.bsp | ||
/server/certs/ | ||
/server/selfsigned.crt | ||
/server/selfsigned.p12 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like the way you split it. However, I read, in https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions#jobs, that
and I'm wondering how isolated it will be - i.e. if at the same time the runner would run build for scala 2.11 for the agent and also 2.13 for server - would there be a potential clash?
https://github.com/orgs/community/discussions/26769#discussioncomment-3253321
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My assumption is that this is the same
runs-on: ubuntu-latest
but probably it will be different instance of this OS. (But...I remember that there is some magic caching potentially happening...we'll see :D )There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems to work fine :).