Self hosted runner(s)

[j-l-s]

Hi there, instead of exchanging in control-toolbox/OptimalControl.jl#247, we can share about self-hosted runners here....

You will find a new ready to use "organisation runner" here: https://github.com/organizations/control-toolbox/settings/actions/runners/65, which is ia 4 cores/8GB X86 slave on Inria's CI infrastructure (ubuntu-24.04 LTS).

You may publish here your ssh pubkeys to add them to the ci account of this runner...

julia is not installed yet on this runner, we have to decide:

• how to install/manage julia versions (juliaup ?)
• how to manage julia cache/env (we may need a weekly job to remove/reinstall all dependancies and start from fresh from time to time)

PS: if we share private informations here, we may need to protect this discussion (if possible)

[jbcaillau]

@j-l-s @srcansiz @ocots @PierreMartinon posting here the links shared this morning (zoom chat):

• https://github.com/julia-actions/install-juliaup/blob/main/README.md
• https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions
• https://stackoverflow.com/questions/78062700/how-can-i-get-a-github-actions-job-to-use-a-different-runner-if-the-first-runner
• https://github.com/BenjaminMichaelis/get-soonest-available-runner/blob/main/action.yml
• https://github.com/actions/runner-images

[jbcaillau]

@j-l-s @srcansiz New meeting today 14:00: @ocots @PierreMartinon same zoom link
https://univ-cotedazur.zoom.us/my/caillau

[jbcaillau]

For the record: ct org settings for self-hosted runners:
https://github.com/organizations/control-toolbox/settings/actions/runners

[jbcaillau]

@j-l-s size of GitHub runners (paying ones, default are smaller)

[jbcaillau]

thanks @srcansiz and @j-l-s for the self-hosted runner setup 👍🏽 we come back to you within 2 weeks about

• ease of use (and performance)
• checking that builds are reduced
• checking that resets (every Monday 3:21) are OK
• relevance of adding a second (bigger) slave

[PierreMartinon]

Boink.
There seems to be an issue for the TagBot workflow (the one that automatically creates a release on the github repo after each new release of the Julia package): missing docker ? Or maybe this particular workflow should be kept on github runners ?

https://github.com/control-toolbox/CTDirect.jl/actions/runs/10562569660/job/29260837620

[srcansiz]

Hi @PierreMartinon,

I installed the docker on the slave. Could you please re-run the workflow to verify if it was the issue.

[PierreMartinon]

Yup, TagBot is fine now, thanks !

[ocots]

@srcansiz Moi j'ai toujours un soucis avec le TagBot : https://github.com/control-toolbox/OptimalControl.jl/actions/runs/10578837519/job/29309857743

Une idée ?

[srcansiz]

Hi @ocots,
C'est une issue de permission. J'ai ajouté l'utilisateur dans le groupe de docker mais j'ai oublié de redémarrer le « GitHub Actions Runner ». Je pense que c'était le problème. Pourais-tu refaire l'essai, s'il te plaît ?

[ocots]

Hi @srcansiz
I think its ok now. Thanks!

[ocots]

@j-l-s Pour pouvoir ajouter des commentaires par le script CommentPR.yml, j'aurais besoin de la commande unzip sur le runner.

https://github.com/control-toolbox/OptimalControl.jl/actions/runs/10563389756/job/29263601820#step:3:5

[srcansiz]

Hi @ocots,

Unzip is installed too.

[jbcaillau]

@j-l-s @srcansiz a typical PR launches three (or more) actions, simultaneously. At least for CI (test) + doc. So probably makes sense to:

• have 2 runners
• each of them possibly bigger than the default GitHub runners (typically 8 cores + 32 Go RAM)

To be discussed further.

[ocots]

[jbcaillau]

Point du 11/12/2024

bilan rapide

• le mécanisme de "backup" enter runner Inria et github est OK

runners

augmenter le nombre

• ajouter 3 runners supplémentaires
• cloner ct-ubuntu-24.ci.inria.fr (Ubuntu 24.04.1 LTS)
• minimun 8 coeurs, 16 si 4 * 16 OK -> upgrader le slave existant
• minimum 8 Gb, disque OK
• en best effort

taille des runners

(vérifier sur cloudstack)

• taille actuelle: 4 * 2 Ghz / 8 Gb

centralisation des actions github

• définition : fichier actions.yml

name: unit-tests
description: Action to execute unit tests

inputs:
  test-dir:
    description: The directory where the test files are kept
    required: true
    default: tests

outputs:
  coverage: # id of the output
    description: Final test coverage
    value: ${{ steps.coverage-rate.outputs.coverage }}

runs:
  using: "composite"
  steps:
    - name: Install researcher dependencies
      run: |
        source ~/.bashrc
        conda clean -i -y
        ./scripts/configure_conda researcher
      shell: bash -l {0}

etc....

• utilisation :

      - name: Run unit tests
        id: unit-test
        uses: ./.github/actions/unit-tests
        if: steps.filter.outputs.module == 'true'
        with:
          test-dir: tests

• gérer globalement les versions des github-actions

  • piste: utiliser une variable d'organisation

au lieu de

- uses: actions/upload-artifact@v3

utiliser

- uses: actions/upload-artifact@${ UPLOAD_ARTIFACT_VERSION }

• workflow templates : plutôt template d'aide à la création que centralisation

https://docs.github.com/en/actions/sharing-automations/creating-workflow-templates-for-your-organization

https://docs.github.com/en/actions/writing-workflows/using-workflow-templates

• reusing workflows :

https://docs.github.com/en/actions/sharing-automations/reusing-workflows

misc

• à terme : dispo de runners sur gpu (simple, voire double précision) pour les tests