PDA-325 Shared GHA for all repos [ruby] (#120)

* initial

* split build from test

* fix go-test, add eks deploy action

* fix go-test

* fix eks-deploy

* update descriptions + step names

* WIP rspec action

* Adds first draft of rspec action

* Remove setup step

* Adds linting action

* added Conors SQ action

* remove not needed steps from SQ action

* Adds one-for-all action

* removed ununsed input from SQ action

* Adds shell: bash lines

* Add shell: bash to one-for-all

* improve names/comments

* Add rubocop results download to sonarqube scan

* added ruby topic+schema validation job

* Fix test output naming

* use inputs

* small fix

* small fix

* small fix

* fix shell missing

* bump actions/cache version

* fix timeout

* add validation action

* seperate topic+schema validation and shared setup

* WIP

* added setup to custom steps

* fix ruby test

* reorganizing deploy and cache for eks

* added steps

* Changed to using go mod for go build and go setup

* Missed saving file

* Updated sonarscanner setup and passed the repo token down to setup go

* Added github token to buf

* Changed files around again

* Fixed cache naming

* Made caching optional

* remove optional cache

* fixed comment

* Added tag update

* Fixing name

* ugh no autosave

* Changed to v0

* Adjusting sudo command

* updating to newer uterm code

* Cleaned up dist

* Added back dist or everything fails

* changed name to ssh

* Moving back to original action but removing sudo

* Try again

* Added sudo back in

* fixing sudo

* Continued refactor

* remove tidy and use download

* remove tidy and use download

* rename setup

* tab on copy/paste

* Reverting all js changes

* added back licenses

* merging in 15% ruby

* Fixed setup and create

* try sonarscan official

* fixed sonar command

* with args

* Move to bundler deployment

* Refactor

* let's try with package

* change permissions

* changing permissions to not be optional

* static paths

* Auto install missing rubocop

* actually restore cache

* use lint results out

* Updated patterns to match go

* updating patterns again

* try moving continue-on-error

* Move continue on error to caller

* conditional check to set continue on error

* reordered env

* Fixing spacing

* removed staging reference

* Try chown instead of 777

* Change permissions back

* Add bundle package back

* Running it all

* remove bundle deployment

* Added trace

* back to normal

* Fixed typos

* Does this work?

* Match github example

* exit code

* check if this is happening correctly

* No quotes

* Need to make sure we don't exit early

* Fixed spacing issue

* Added wait for db to actions

* remove wait for db

* remove input since we only validate on prod topics

* removed exit code hackery

* Added some readme comments

---------

Co-authored-by: chriskarlin <[email protected]>
Co-authored-by: Conor McLaughlin <[email protected]>

README.md

@@ -2,6 +2,10 @@
 
 This is a GitHub Action to be used in workflows at Flipp.
 
+## Versions
+
+The current major version tag is `v0`. When using actions in this repo you should generally use this tag. The repo will continue adding patch versions `v0.x` and advancing the `v0` tag to the latest patch version for all non-breaking changes.
+
 ## Usage
 
 This action should be included in every job in every workflow. It provides tools that should be useful across all workflows, particularly in the pre/post steps. Having all this in one action should reduce noise in your config files and allow you to get new updates "for free" in the future.
@@ -10,7 +14,7 @@ A sample configuration may look like this:
 
 ```yaml
     - name: Flipp global
-      uses: wishabi/github-actions@v0.5
+      uses: wishabi/github-actions@v0
       timeout-minutes: 10
       env:
         SLACK_BOT_TOKEN: "${{ secrets.SLACK_BOT_TOKEN }}"
@@ -53,3 +57,5 @@ ssh_always|No|If set, SSH sessions will always be created, even on success.
 ## Contributing
 
 Please make sure to run `npm run prepare` before committing your files! You should probably add this to `.git/hooks/pre-commit`.
+
+After creating a new minor patch version (ie `v0.x`), remember to update the major version tag to point to the latest patch.

deploy/lambda/action.yml

@@ -6,7 +6,6 @@ inputs:
     description: Application environment (staging or production)
     required: true
 
-
 runs:
   using: 'composite'
   steps:

ruby/deps/action.yml

@@ -0,0 +1,24 @@
+name: 'Ruby bundle install and caching'
+description: 'custom GitHub action that runs to setup ruby bundle install and caching'
+
+runs:
+  using: "composite"
+  steps:
+    - name: Bundle cache
+      id: bundle-cache
+      uses: actions/cache@v3
+      with:
+        path: vendor/bundle
+        key: rails-${{ hashFiles('Gemfile.lock') }}
+        restore-keys: rails-${{ hashFiles('Gemfile.lock') }}
+    - name: Bundle install
+      if: steps.bundle-cache.outputs.cache-hit != 'true'
+      shell: bash
+      run: |
+        sudo chown circleci:circleci -R /home/circleci/.bundle
+        bundle install
+      env: 
+        BUNDLE_JOBS: 4
+        BUNDLE_DEPLOYMENT: true
+    - name: Caching workspace for deploy jobs
+      uses: wishabi/github-actions/cache@v0

ruby/lint/action.yml

@@ -0,0 +1,20 @@
+name: 'Lint - rubocop'
+description: 'Lints using rubocop'
+
+runs:
+  using: "composite"
+  steps:
+    - name: Execute rubocop
+      id: rubocop
+      continue-on-error: true
+      shell: bash
+      run: |
+        bundle exec rubocop --format progress --format json --out rubocop/rubocop.json
+      env: 
+        BUNDLE_DEPLOYMENT: true
+    - name: 'Upload rubocop results'
+      uses: actions/upload-artifact@v3
+      with:
+        name: "${{ github.sha }}-lint-results.out"
+        path: ./rubocop/rubocop.json
+        retention-days: 1

ruby/one-for-all/action.yml

@@ -0,0 +1,20 @@
+name: 'one-for-all'
+description: 'Runs the one-for-all script'
+
+runs:
+  using: "composite"
+  steps: 
+    - name: Download test result
+      uses: actions/download-artifact@v3
+      with:
+        name: ${{ github.sha }}-test-report.out
+        path: ./result/
+    - name: execute one-for-all
+      shell: bash
+      run: |
+        aws s3 cp s3://one-for-all/start-one-for-all.sh start-one-for-all.sh
+        chmod +x ./start-one-for-all.sh
+        ENFORCER_LANGUAGE="ruby" ./start-one-for-all.sh
+      env:
+        CIRCLE_BRANCH: ${{ github.head_ref || github.ref_name }} 
+        CIRCLE_PROJECT_REPONAME: ${{ github.event.repository.name }}

ruby/test/action.yml

@@ -0,0 +1,49 @@
+name: 'Tests - rspec'
+description: 'Runs tests using rspec'
+inputs:
+  DB_HOST:
+      description: Database host
+      required: false
+  DB_PORT:
+    description: Database port
+    required: false
+
+runs:
+  using: "composite"
+  steps:
+    - name: Execute rspec tests
+      if: ${{ inputs.DB_HOST != '' && inputs.DB_PORT != '' }}
+      shell: bash
+      run: |
+           bundle exec rails db:create db:migrate
+      env:
+        BUNDLE_DEPLOYMENT: true
+        DB_HOST: ${{ inputs.DB_HOST }}
+        DB_PORT: ${{ inputs.DB_PORT }}
+        RAILS_ENV: test
+    - name: Execute rspec tests
+      shell: bash
+      run: |
+           bundle exec rspec --format progress --format RspecJunitFormatter -o result/rspec.xml
+      env:
+        BUNDLE_DEPLOYMENT: true
+        DB_HOST: ${{ inputs.DB_HOST }}
+        DB_PORT: ${{ inputs.DB_PORT }}
+        RAILS_ENV: test
+    - name: Rename code coverage paths to be found by Sonarqube
+      working-directory: ./coverage
+      shell: bash
+      run: |
+        sed -i 's@'$GITHUB_WORKSPACE'@/github/workspace/@g' coverage.json
+    - name: 'Upload coverage'
+      uses: actions/upload-artifact@v3
+      with:
+        name: ${{ github.sha }}-coverage.out
+        path: ./coverage/coverage.json
+        retention-days: 1
+    - name: 'Upload test results'
+      uses: actions/upload-artifact@v3
+      with:
+        name: ${{ github.sha }}-test-report.out
+        path: ./result/rspec.xml
+        retention-days: 1

ruby/validation/schema/action.yml

@@ -0,0 +1,22 @@
+name: 'Kafka Schema validation'
+description: 'Custom GitHub action that runs kafka schema validation'
+
+inputs:
+  ENV:
+    description: Application environment (staging or production)
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: Generate schema validation json
+      shell: bash
+      run: |
+        bundle exec rails g flipp_ruby_kafka:schema_validation
+        sudo chown circleci:circleci /var/run/docker.sock
+      env: 
+        BUNDLE_DEPLOYMENT: true
+    - name: Schema validation 
+      shell: bash
+      run: |
+        chmod +x ./deploy/build.sh && ./deploy/build.sh service schema_validation --env=${{ inputs.ENV }}

ruby/validation/topic/action.yml

@@ -0,0 +1,22 @@
+name: 'Kafka Topic validation'
+description: 'custom GitHub action that runs kafka topic validation'
+
+inputs:
+  CONSUMER_SERVICE:
+    description: Name of the consumer service
+    required: true
+  PRODUCER_SERVICE:
+    description: Name of the producer service
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+    - name: Staging Topic validation
+      shell: bash
+      run: |
+        bundle exec rails g flipp_ruby_kafka:topic_config --producer_services=${{ inputs.PRODUCER_SERVICE }} --consumer_services=${{ inputs.CONSUMER_SERVICE }}
+        sudo chown circleci:circleci /var/run/docker.sock
+        chmod +x ./deploy/build.sh && ./deploy/build.sh service topic_validation --env=production --config-file=./platform.yml --config-file=./topic_validation.yaml
+      env: 
+        BUNDLE_DEPLOYMENT: true

sonarqube-scan/action.yml

@@ -1,4 +1,4 @@
-name: 'SonarQube Quality Scan'
+name: 'SonarQube Scan'
 description: 'Runs various quality and security checks against changes'
 inputs:
   SONAR_HOST_URL:
@@ -7,27 +7,34 @@ inputs:
   SONAR_TOKEN:
     description: secret value
     required: true
+  CHECK_LINTER:
+    description: whether to download and analyze linter results
+    default: false
+
 runs:
   using: "composite"
   steps:
     - uses: actions/checkout@v3
       with:
         fetch-depth: 0
-    - name: Setting up sonarqube
-      uses: warchant/setup-sonar-scanner@v4
     - name: Downloading test report
       uses: actions/download-artifact@v3
       with:
         name: "${{ github.sha }}-test-report.out"
-    - name: Downloading coverage
+    - name: Downloading coverage report
       uses: actions/download-artifact@v3
       with:
         name: "${{ github.sha }}-coverage.out"
+    - name: Download linter results
+      if: ${{ inputs.CHECK_LINTER == 'true' }}
+      uses: actions/download-artifact@v3
+      with:
+        name: "${{ github.sha }}-lint-results.out"
     - name: SonarQube Scan
-      shell: bash
-      run: |
-          export PATH=/home/circleci/_work/sonar-scanner/bin:$PATH
-          sonar-scanner \
-            -Dsonar.projectVersion=${GITHUB_SHA::7} \
-            -Dsonar.host.url=${{ inputs.SONAR_HOST_URL }} \
-            -Dsonar.token=${{ inputs.SONAR_TOKEN }}
+      uses: sonarsource/sonarqube-scan-action@master
+      env:
+        SONAR_TOKEN: ${{ inputs.SONAR_TOKEN }}
+        SONAR_HOST_URL: ${{ inputs.SONAR_HOST_URL }}
+      with:
+        args: >
+          -Dsonar.projectVersion=${GITHUB_SHA::7}