Convert Azure pipeline to GitHub Actions #4

mouismail · 2024-06-03T11:56:40Z

Related to #3

Converts the Azure pipeline to GitHub Actions, implementing the specified jobs and variables as outlined in the issue.

CI Workflow Setup: Adds a new GitHub Actions workflow in .github/workflows/ci.yml that mirrors the Azure pipeline's functionality. This includes setting up environment variables build_pool and linux_image, and configuring the workflow to trigger manually or on a schedule.
GenerateMatrix Job: Implements the GenerateMatrix job to list agents to inspect and calculates a matrix for execution, closely following the Azure pipeline's steps.
Inspect Job: Adds the Inspect job that depends on GenerateMatrix, utilizing a matrix strategy for parallel execution. It includes steps to verify the accessibility of the Artifact Staging Directory and to publish artifacts, adapting Azure pipeline commands to GitHub Actions syntax.
CD Workflow Placeholder: Introduces a placeholder GitHub Actions workflow in .github/workflows/cd.yml for future deployment processes, with comments outlining intended steps based on the Azure pipeline conversion.

For more details, open the Copilot Workspace session.

mouismail · 2024-06-03T11:57:42Z

In the file .github/workflows/ci.yml at lines 23-30, there is a loop that is creating a new array but doesn't handle an empty list of agents. If there are no agents, the script could break or produce unexpected results. It is recommended to add a check at the start of the loop to handle the case where AgentsToInspect is empty or undefined. ci.yml:23-30

@@ -0,0 +1,67 @@
[-1,1]+name: CI
[-1,2]+
[-1,3]+on:
[-1,4]+  workflow_dispatch:
[-1,5]+  schedule:
[-1,6]+    - cron: '0 0 * * *' # Scheduled trigger as per Azure pipeline
[-1,7]+
[-1,8]+env:
[-1,9]+  build_pool: "JUP-DEERL-IPS-VS-HMI-RT-LBA-1"
[-1,10]+  linux_image: "captain.rtf.siemens.net:8443/rtng_unified/wincc-bullseye:11.0.58"
[-1,11]+
[-1,12]+jobs:
[-1,13]+  GenerateMatrix:
[-1,14]+    name: list agents to inspect
[-1,15]+    runs-on: ${{ env.build_pool }}
[-1,16]+    container:
[-1,17]+      image: ${{ env.linux_image }}
[-1,18]+      options: --endpoint=docker-registry
[-1,19]+    timeout-minutes: 5
[-1,20]+    steps:
[-1,21]+      - name: Calculate matrix to execute
[-1,22]+        run: |
[-1,23]+          set -euo pipefail
[-1,24]+          [[ -z "${AGENT_DIAGNOSTIC+x}" ]] || set -x # enable diagnostics for inline script
[-1,25]+          declare -a legs=()
[-1,26]+          for agent in $(AgentsToInspect); do
[-1,27]+              legs+=("'':{'agent':''}")
[-1,28]+          done
[-1,29]+          echo "legs=${legs[*]}" >> 
[-1,30]+
[-1,31]+  Inspect:
[-1,32]+    name: inspect
[-1,33]+    needs: GenerateMatrix
[-1,34]+    runs-on: ${{ env.build_pool }}
[-1,35]+    strategy:
[-1,36]+      matrix: ${{ fromJson(needs.GenerateMatrix.outputs.legs) }}
[-1,37]+      max-parallel: 999
[-1,38]+    container:
[-1,39]+      image: ${{ env.linux_image }}
[-1,40]+      options: --endpoint=docker-registry
[-1,41]+    timeout-minutes: 2
[-1,42]+    steps:
[-1,43]+      - name: verify Artifact Staging Directory is accessible
[-1,44]+        run: |
[-1,45]+          set -euo pipefail
[-1,46]+          warn() {
[-1,47]+              echo "::error::$*"
[-1,48]+              echo "::endgroup::"
[-1,49]+          }
[-1,50]+          die() {
[-1,51]+              warn "$@"
[-1,52]+              exit 1
[-1,53]+          }
[-1,54]+          date
[-1,55]+          set -x # always show details of execution.
[-1,56]+          [[ -d "$(Agent.WorkFolder)" ]] || warn "Cannot access the agent Workfolder!"
[-1,57]+          ls -la  "$(Agent.WorkFolder)" || warn "Cannot list content of the agent Workfolder!"
[-1,58]+          [[ -d "$(Build.ArtifactStagingDirectory)" ]] || warn "Cannot access the Artifact Staging Directory!"
[-1,59]+          ls -la "$(Build.ArtifactStagingDirectory)" || warn "Cannot list content of Artifact Staging Directory!"
[-1,60]+          touch "$(Build.ArtifactStagingDirectory)/$(agent).log" || die "Cannot create a log file in Artifact Staging Directory!"
[-1,61]+          ls -la "$(Build.ArtifactStagingDirectory)"
[-1,62]+        if: always()
[-1,63]+      - name: Publish artifacts
[-1,64]+        uses: actions/upload-artifact@v2
[-1,65]+        with:
[-1,66]+          name: ArtifactStagingDirectory
[-1,67]+          path: ${{ env.Build.ArtifactStagingDirectory }}

Also in the .github/workflows/ci.yml file at lines 45-61, error handling is done using custom functions warn and die which are defined within the script. Using custom error handling can lead to inconsistency and confusion. It is recommended to replace these functions with standard error handling mechanisms of the shell or, if the functions are required across multiple scripts, to move them into a script that can be sourced when needed. ci.yml:45-61

@@ -0,0 +1,67 @@
[-1,1]+name: CI
[-1,2]+
[-1,3]+on:
[-1,4]+  workflow_dispatch:
[-1,5]+  schedule:
[-1,6]+    - cron: '0 0 * * *' # Scheduled trigger as per Azure pipeline
[-1,7]+
[-1,8]+env:
[-1,9]+  build_pool: "JUP-DEERL-IPS-VS-HMI-RT-LBA-1"
[-1,10]+  linux_image: "captain.rtf.siemens.net:8443/rtng_unified/wincc-bullseye:11.0.58"
[-1,11]+
[-1,12]+jobs:
[-1,13]+  GenerateMatrix:
[-1,14]+    name: list agents to inspect
[-1,15]+    runs-on: ${{ env.build_pool }}
[-1,16]+    container:
[-1,17]+      image: ${{ env.linux_image }}
[-1,18]+      options: --endpoint=docker-registry
[-1,19]+    timeout-minutes: 5
[-1,20]+    steps:
[-1,21]+      - name: Calculate matrix to execute
[-1,22]+        run: |
[-1,23]+          set -euo pipefail
[-1,24]+          [[ -z "${AGENT_DIAGNOSTIC+x}" ]] || set -x # enable diagnostics for inline script
[-1,25]+          declare -a legs=()
[-1,26]+          for agent in $(AgentsToInspect); do
[-1,27]+              legs+=("'':{'agent':''}")
[-1,28]+          done
[-1,29]+          echo "legs=${legs[*]}" >> 
[-1,30]+
[-1,31]+  Inspect:
[-1,32]+    name: inspect
[-1,33]+    needs: GenerateMatrix
[-1,34]+    runs-on: ${{ env.build_pool }}
[-1,35]+    strategy:
[-1,36]+      matrix: ${{ fromJson(needs.GenerateMatrix.outputs.legs) }}
[-1,37]+      max-parallel: 999
[-1,38]+    container:
[-1,39]+      image: ${{ env.linux_image }}
[-1,40]+      options: --endpoint=docker-registry
[-1,41]+    timeout-minutes: 2
[-1,42]+    steps:
[-1,43]+      - name: verify Artifact Staging Directory is accessible
[-1,44]+        run: |
[-1,45]+          set -euo pipefail
[-1,46]+          warn() {
[-1,47]+              echo "::error::$*"
[-1,48]+              echo "::endgroup::"
[-1,49]+          }
[-1,50]+          die() {
[-1,51]+              warn "$@"
[-1,52]+              exit 1
[-1,53]+          }
[-1,54]+          date
[-1,55]+          set -x # always show details of execution.
[-1,56]+          [[ -d "$(Agent.WorkFolder)" ]] || warn "Cannot access the agent Workfolder!"
[-1,57]+          ls -la  "$(Agent.WorkFolder)" || warn "Cannot list content of the agent Workfolder!"
[-1,58]+          [[ -d "$(Build.ArtifactStagingDirectory)" ]] || warn "Cannot access the Artifact Staging Directory!"
[-1,59]+          ls -la "$(Build.ArtifactStagingDirectory)" || warn "Cannot list content of Artifact Staging Directory!"
[-1,60]+          touch "$(Build.ArtifactStagingDirectory)/$(agent).log" || die "Cannot create a log file in Artifact Staging Directory!"
[-1,61]+          ls -la "$(Build.ArtifactStagingDirectory)"
[-1,62]+        if: always()
[-1,63]+      - name: Publish artifacts
[-1,64]+        uses: actions/upload-artifact@v2
[-1,65]+        with:
[-1,66]+          name: ArtifactStagingDirectory
[-1,67]+          path: ${{ env.Build.ArtifactStagingDirectory }}

In the .github/workflows/ci.yml file at lines 57-60, there are checks to ensure that certain directories exist and are accessible, but these checks are not comprehensive. For instance, there are no checks on the permissions of these directories. It is recommended to add additional checks to ensure that these directories are not only accessible but also have the necessary permissions. ci.yml:57-60

@@ -0,0 +1,67 @@
[-1,1]+name: CI
[-1,2]+
[-1,3]+on:
[-1,4]+  workflow_dispatch:
[-1,5]+  schedule:
[-1,6]+    - cron: '0 0 * * *' # Scheduled trigger as per Azure pipeline
[-1,7]+
[-1,8]+env:
[-1,9]+  build_pool: "JUP-DEERL-IPS-VS-HMI-RT-LBA-1"
[-1,10]+  linux_image: "captain.rtf.siemens.net:8443/rtng_unified/wincc-bullseye:11.0.58"
[-1,11]+
[-1,12]+jobs:
[-1,13]+  GenerateMatrix:
[-1,14]+    name: list agents to inspect
[-1,15]+    runs-on: ${{ env.build_pool }}
[-1,16]+    container:
[-1,17]+      image: ${{ env.linux_image }}
[-1,18]+      options: --endpoint=docker-registry
[-1,19]+    timeout-minutes: 5
[-1,20]+    steps:
[-1,21]+      - name: Calculate matrix to execute
[-1,22]+        run: |
[-1,23]+          set -euo pipefail
[-1,24]+          [[ -z "${AGENT_DIAGNOSTIC+x}" ]] || set -x # enable diagnostics for inline script
[-1,25]+          declare -a legs=()
[-1,26]+          for agent in $(AgentsToInspect); do
[-1,27]+              legs+=("'':{'agent':''}")
[-1,28]+          done
[-1,29]+          echo "legs=${legs[*]}" >> 
[-1,30]+
[-1,31]+  Inspect:
[-1,32]+    name: inspect
[-1,33]+    needs: GenerateMatrix
[-1,34]+    runs-on: ${{ env.build_pool }}
[-1,35]+    strategy:
[-1,36]+      matrix: ${{ fromJson(needs.GenerateMatrix.outputs.legs) }}
[-1,37]+      max-parallel: 999
[-1,38]+    container:
[-1,39]+      image: ${{ env.linux_image }}
[-1,40]+      options: --endpoint=docker-registry
[-1,41]+    timeout-minutes: 2
[-1,42]+    steps:
[-1,43]+      - name: verify Artifact Staging Directory is accessible
[-1,44]+        run: |
[-1,45]+          set -euo pipefail
[-1,46]+          warn() {
[-1,47]+              echo "::error::$*"
[-1,48]+              echo "::endgroup::"
[-1,49]+          }
[-1,50]+          die() {
[-1,51]+              warn "$@"
[-1,52]+              exit 1
[-1,53]+          }
[-1,54]+          date
[-1,55]+          set -x # always show details of execution.
[-1,56]+          [[ -d "$(Agent.WorkFolder)" ]] || warn "Cannot access the agent Workfolder!"
[-1,57]+          ls -la  "$(Agent.WorkFolder)" || warn "Cannot list content of the agent Workfolder!"
[-1,58]+          [[ -d "$(Build.ArtifactStagingDirectory)" ]] || warn "Cannot access the Artifact Staging Directory!"
[-1,59]+          ls -la "$(Build.ArtifactStagingDirectory)" || warn "Cannot list content of Artifact Staging Directory!"
[-1,60]+          touch "$(Build.ArtifactStagingDirectory)/$(agent).log" || die "Cannot create a log file in Artifact Staging Directory!"
[-1,61]+          ls -la "$(Build.ArtifactStagingDirectory)"
[-1,62]+        if: always()
[-1,63]+      - name: Publish artifacts
[-1,64]+        uses: actions/upload-artifact@v2
[-1,65]+        with:
[-1,66]+          name: ArtifactStagingDirectory
[-1,67]+          path: ${{ env.Build.ArtifactStagingDirectory }}

The .github/workflows/cd.yml file is currently a placeholder and does not contain any real deployment steps. While this is not a problem per se, it is important to remember to update this file with the actual deployment steps when they are ready. cd.yml:1-6

@@ -0,0 +1,6 @@
[-1,1]+# This is a placeholder for future deployment processes
[-1,2]+# The following steps are intended based on Azure pipeline conversion:
[-1,3]+# - Define environment variables for deployment
[-1,4]+# - Setup deployment environment
[-1,5]+# - Execute deployment scripts
[-1,6]+# - Verify deployment status

In the .github/workflows/ci.yml file at lines 14-30, the job GenerateMatrix is generating a matrix for parallel execution. However, the generation of the matrix is hard-coded in the job and doesn't account for changes in environment variables or agent lists. It is recommended to externalize this logic into a separate script or use an action that can generate the matrix dynamically based on the current state of the environment. ci.yml:14-30

@@ -0,0 +1,67 @@
[-1,1]+name: CI
[-1,2]+
[-1,3]+on:
[-1,4]+  workflow_dispatch:
[-1,5]+  schedule:
[-1,6]+    - cron: '0 0 * * *' # Scheduled trigger as per Azure pipeline
[-1,7]+
[-1,8]+env:
[-1,9]+  build_pool: "JUP-DEERL-IPS-VS-HMI-RT-LBA-1"
[-1,10]+  linux_image: "captain.rtf.siemens.net:8443/rtng_unified/wincc-bullseye:11.0.58"
[-1,11]+
[-1,12]+jobs:
[-1,13]+  GenerateMatrix:
[-1,14]+    name: list agents to inspect
[-1,15]+    runs-on: ${{ env.build_pool }}
[-1,16]+    container:
[-1,17]+      image: ${{ env.linux_image }}
[-1,18]+      options: --endpoint=docker-registry
[-1,19]+    timeout-minutes: 5
[-1,20]+    steps:
[-1,21]+      - name: Calculate matrix to execute
[-1,22]+        run: |
[-1,23]+          set -euo pipefail
[-1,24]+          [[ -z "${AGENT_DIAGNOSTIC+x}" ]] || set -x # enable diagnostics for inline script
[-1,25]+          declare -a legs=()
[-1,26]+          for agent in $(AgentsToInspect); do
[-1,27]+              legs+=("'':{'agent':''}")
[-1,28]+          done
[-1,29]+          echo "legs=${legs[*]}" >> 
[-1,30]+
[-1,31]+  Inspect:
[-1,32]+    name: inspect
[-1,33]+    needs: GenerateMatrix
[-1,34]+    runs-on: ${{ env.build_pool }}
[-1,35]+    strategy:
[-1,36]+      matrix: ${{ fromJson(needs.GenerateMatrix.outputs.legs) }}
[-1,37]+      max-parallel: 999
[-1,38]+    container:
[-1,39]+      image: ${{ env.linux_image }}
[-1,40]+      options: --endpoint=docker-registry
[-1,41]+    timeout-minutes: 2
[-1,42]+    steps:
[-1,43]+      - name: verify Artifact Staging Directory is accessible
[-1,44]+        run: |
[-1,45]+          set -euo pipefail
[-1,46]+          warn() {
[-1,47]+              echo "::error::$*"
[-1,48]+              echo "::endgroup::"
[-1,49]+          }
[-1,50]+          die() {
[-1,51]+              warn "$@"
[-1,52]+              exit 1
[-1,53]+          }
[-1,54]+          date
[-1,55]+          set -x # always show details of execution.
[-1,56]+          [[ -d "$(Agent.WorkFolder)" ]] || warn "Cannot access the agent Workfolder!"
[-1,57]+          ls -la  "$(Agent.WorkFolder)" || warn "Cannot list content of the agent Workfolder!"
[-1,58]+          [[ -d "$(Build.ArtifactStagingDirectory)" ]] || warn "Cannot access the Artifact Staging Directory!"
[-1,59]+          ls -la "$(Build.ArtifactStagingDirectory)" || warn "Cannot list content of Artifact Staging Directory!"
[-1,60]+          touch "$(Build.ArtifactStagingDirectory)/$(agent).log" || die "Cannot create a log file in Artifact Staging Directory!"
[-1,61]+          ls -la "$(Build.ArtifactStagingDirectory)"
[-1,62]+        if: always()
[-1,63]+      - name: Publish artifacts
[-1,64]+        uses: actions/upload-artifact@v2
[-1,65]+        with:
[-1,66]+          name: ArtifactStagingDirectory
[-1,67]+          path: ${{ env.Build.ArtifactStagingDirectory }}

Convert Azure pipeline to GitHub Actions

70535b7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert Azure pipeline to GitHub Actions #4

Convert Azure pipeline to GitHub Actions #4

mouismail commented Jun 3, 2024 •

edited

Loading

mouismail commented Jun 3, 2024

Convert Azure pipeline to GitHub Actions #4

Are you sure you want to change the base?

Convert Azure pipeline to GitHub Actions #4

Conversation

mouismail commented Jun 3, 2024 • edited Loading

mouismail commented Jun 3, 2024

mouismail commented Jun 3, 2024 •

edited

Loading