-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: modify to reusable workflow #482
base: main
Are you sure you want to change the base?
Changes from 17 commits
0e9ab68
95bdf3e
fa82ebe
55c303e
b0ea833
ce0da9e
ef701ab
b7dcc8b
5371770
df793dd
b00bb1e
3ffae7b
93aa63a
a62c1cc
f65b397
1b93218
f6eb6db
2b3bf15
8e3cf21
9d0b89d
9a4da90
d565e5b
c8f6547
e1c8c55
483526a
653707f
5ddc728
58f05e1
24e20d2
a84d5d9
29b8d1b
5928cd8
90f4781
6d4a2e1
2147cb8
8701afa
0776892
e1ef485
6c21b2c
e3d60cd
604ffcd
82fd457
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,12 @@ | ||
#!/bin/sh | ||
#!/usr/bin/env sh | ||
set -x | ||
|
||
cmake \ | ||
-B build \ | ||
-S "$EXT_MOUNT/source" \ | ||
-G Ninja \ | ||
-DEVC_ENABLE_CCACHE=1 \ | ||
-DISO15118_2_GENERATE_AND_INSTALL_CERTIFICATES=OFF \ | ||
-DCMAKE_INSTALL_PREFIX="$WORKSPACE_PATH/dist" \ | ||
-DBUILD_TESTING=ON | ||
-DBUILD_TESTING=ON \ | ||
"${CMAKE_FLAGS_EXTRA}" | ||
|
||
ninja -j$(nproc) -C build |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
#!/usr/bin/env sh | ||
set -x | ||
|
||
ninja -j$(nproc) -C build tests/test |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,14 @@ | ||
#!/bin/sh | ||
#!/usr/bin/env sh | ||
set -x | ||
|
||
ninja -j$(nproc) -C build install | ||
|
||
# install everestpy via cmake target from everest-framework | ||
ninja -C build everestpy_pip_install_dist | ||
|
||
rsync -a "$EXT_MOUNT/source/tests" ./ | ||
|
||
rm -rf build | ||
|
||
cd tests | ||
pytest --everest-prefix ../dist core_tests/*.py framework_tests/*.py |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,27 @@ | ||
name: Build and test | ||
on: | ||
pull_request: {} | ||
workflow_call: # defines it as reusable workflow | ||
inputs: | ||
repository: | ||
description: The repo to checkout and run the workflow steps in | ||
type: string | ||
required: true | ||
repository_ref: | ||
description: The ref (branch, tag, commit-hash) of the `inputs.repository` | ||
type: string | ||
required: true | ||
cmake_flags_extra: | ||
description: additional flags to add to the common set of flags (see `.ci/build-kit/compile.sh`) | ||
type: string | ||
unit_tests: | ||
description: Whether to run unit-tests or not | ||
type: boolean | ||
required: true | ||
integration_tests: | ||
description: Whether to run integration-tests or not | ||
type: boolean | ||
required: true | ||
workflow_dispatch: | ||
inputs: | ||
runner: | ||
|
@@ -23,47 +44,67 @@ jobs: | |
run: | | ||
BRANCH_NAME_FOR_CACHE="${GITHUB_REF_NAME//-/_}" | ||
echo "branch_name_for_cache=${BRANCH_NAME_FOR_CACHE}" >> "$GITHUB_ENV" | ||
- name: Setup cache | ||
uses: actions/cache@v3 | ||
- name: Restore cache | ||
uses: actions/cache/restore@v3 | ||
id: restore-cache | ||
with: | ||
path: cache | ||
path: cache # keep in sync with "Store cache" | ||
key: compile-${{ env.branch_name_for_cache }}-${{ github.sha }} | ||
restore-keys: | | ||
compile-${{ env.branch_name_for_cache }}- | ||
compile- | ||
- name: Checkout everest-core | ||
uses: actions/checkout@v3 | ||
- name: Checkout from `everest-core` the `.ci/` directory (reusable workflow mode, see https://stackoverflow.com/a/74123003/1168315) | ||
uses: actions/checkout@v4 | ||
with: | ||
sparse-checkout: | | ||
.ci | ||
repository: EVerest/everest-core | ||
ref: CI/reusable-workflow # TODO get reference in general, see https://stackoverflow.com/questions/74784735 resp. https://github.com/actions/toolkit/issues/1264 | ||
- name: Checkout repo ${{ inputs.repository || 'EVerest/everest-core' }} to directory `source/` | ||
uses: actions/checkout@v4 | ||
with: | ||
path: source | ||
path: source/ | ||
repository: ${{ inputs.repository }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This doesn't seem like the right way to set up a reusable workflow. When someone makes a fork, the CI instructions on their forked branches will point to whatever original repository is passed in, for example, EVerest/libocpp. What should happen is that the current branch that initialized the workflow should run, fetching the branch from the forked repository. In the current state, this fails at the checkout stage because we can't check out the original repository from our fork. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see the problem in the way the In the example of To enable workflows running on forks I suggest the use of the env variables -> In both examples I would replace the line with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't the reusable workflow use that by default? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah, maybe this parameter is obsolete, not sure about this.. I will spend some time on this in the next weeks to figure those things out |
||
ref: ${{ inputs.repository_ref }} | ||
- name: Run clang-format | ||
uses: everest/everest-ci/github-actions/[email protected] | ||
with: | ||
source-dir: source | ||
extensions: hpp,cpp | ||
exclude: cache | ||
- name: Setup run scripts | ||
- name: Setup run scripts # FIXME update `build-kit` to not use the `scripts/`-dir implicitly | ||
run: | | ||
mkdir scripts | ||
rsync -a source/.ci/build-kit/ scripts | ||
rsync -a .ci/build-kit/ scripts | ||
- name: Pull docker container | ||
run: | | ||
docker pull --quiet ghcr.io/everest/build-kit-alpine:latest | ||
docker image tag ghcr.io/everest/build-kit-alpine:latest build-kit | ||
- name: Compile | ||
env: | ||
CMAKE_FLAGS_EXTRA: ${{ inputs.cmake_flags_extra || '-DEVC_ENABLE_CCACHE=1 -DISO15118_2_GENERATE_AND_INSTALL_CERTIFICATES=OFF' }} | ||
run: | | ||
docker run \ | ||
--volume "$(pwd):/ext" \ | ||
--name compile-container \ | ||
build-kit run-script compile | ||
- name: Unit tests and install | ||
- name: Unit tests | ||
if: ${{ inputs.unit_tests || false }} | ||
run: | | ||
docker commit compile-container build-image | ||
docker commit compile-container unit-tests-image | ||
docker run \ | ||
--volume "$(pwd):/ext" \ | ||
--name test-container \ | ||
build-image run-script test_and_install | ||
- name: Run integration tests | ||
unit-tests-image run-script unit_tests | ||
- name: Integration tests | ||
if: ${{ inputs.integration_tests || true }} | ||
run: | | ||
docker commit test-container integration-image | ||
pushd source/.ci/e2e | ||
docker-compose run e2e-test-server run-script tests | ||
docker commit compile-container integration-tests-image | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do the integration-tests rely on the unit-tests somehow? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think so :) |
||
pushd .ci/e2e | ||
docker-compose run e2e-test-server run-script e2e-test-server | ||
- name: Store cache | ||
uses: actions/cache/save@v3 | ||
if: always() | ||
with: | ||
path: cache # keep in sync with "Restore cache" | ||
key: ${{ steps.restore-cache.outputs.cache-primary-key }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,4 @@ | ||
*build | ||
*build-cross | ||
/build*/ | ||
.cache/ | ||
workspace.yaml | ||
.vscode/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@andistorm Do you know a solution for this? Otherwise, I would just set it to
main
for now.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest to not checkout the ci repo in the workflow, but to use custom github actions. There are multiple options for github actions:
composite actions
that are similiar toreusable workflows
,Javascript actions
andDocker container actions
. The last one would be an option to include additional filesto the action.I could also imagine a solution with having the bash scripts in each repo with "custom" commands and cmake flags..