feat: add daily CLI smoke test pipeline

[Raina451]

Summary

• Adds a scheduled GitHub Actions workflow that runs the full coded app lifecycle (pack → publish → deploy → health check) daily at 9 AM IST
• Two parallel jobs: stable CLI (npm @latest) and alpha CLI (GH Packages) — both deploy coded app against alpha env
• Posts to #coded-apps-alerts Slack channel on failure only

[claude]

These three values are hardcoded in the workflow file and will be committed to git history. ORG and TENANT expose internal alpha-environment infrastructure details; FOLDER_KEY is a GUID that could be used to make API calls in the wrong environment if reused elsewhere.

Use repository variables instead so they can be rotated/changed without a code commit:

Suggested change

	ORG: popoc
	TENANT: adetenant
	FOLDER_KEY: 8645d674-92d8-4281-9aef-43f3e3608ded
	ORG: ${{ vars.SMOKE_TEST_ORG }}
	TENANT: ${{ vars.SMOKE_TEST_TENANT }}
	FOLDER_KEY: ${{ vars.SMOKE_TEST_FOLDER_KEY }}

[claude]

The workflow deploys an app to the alpha environment on every run but never cleans it up. Daily runs will accumulate deployed versions indefinitely. Add an explicit cleanup step that runs even on failure — either a dedicated uip codedapp undeploy step after health-check, or an if: always() cleanup job that runs after the test jobs.

Example (add as the last step in each job):

      - name: Cleanup
        if: always()
        run: |
          uip codedapp undeploy \
            --name "$APP_NAME" \
            --folder-key "${{ env.FOLDER_KEY }}" || true

[claude]

The alpha-cli job is a verbatim copy of stable-cli — 5 of 6 steps are identical. Only the "Install CLI" step differs. Any future change to Pack, Publish, Deploy, or Health-check steps must be made in two places, which is an ongoing maintenance hazard.

Refactor using a matrix strategy:

jobs:
  smoke:
    name: ${{ matrix.label }}
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        include:
          - label: Stable CLI (npm @latest)
            app_name: cli-smoke-stable
            install_from_gh_packages: false
          - label: Alpha CLI (GH Packages)
            app_name: cli-smoke-alpha
            install_from_gh_packages: true
    env:
      APP_NAME: ${{ matrix.app_name }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Install CLI
        run: |
          if [ "${{ matrix.install_from_gh_packages }}" = "true" ]; then
            echo "//npm.pkg.github.com/:_authToken=${{ secrets.GH_PACKAGES_TOKEN }}" >> ~/.npmrc
            echo "@uipath:registry=https://npm.pkg.github.com" >> ~/.npmrc
            npm i -g @uipath/cli
          else
            npm i -g @uipath/cli@latest
          fi
          uip tools update
      # ... shared Pack / Publish / Deploy / Health-check / Cleanup steps

This also naturally handles the packages: read permission — it can be set at the job level with if: matrix.install_from_gh_packages or simply granted for all matrix legs since the stable leg never hits npm.pkg.github.com.

[claude]

Review summary

Four issues found this run:

1. Hardcoded ORG/TENANT/FOLDER_KEY — internal alpha environment details committed to git history; should be repository variables (vars.*)
2. Version collision on same-day re-run — date +%Y%m%d granularity means a manual re-trigger publishes an already-existing version; use +%Y%m%d%H%M%S
3. No cleanup step — daily runs accumulate deployed app versions in the alpha environment indefinitely; add an if: always() undeploy step
4. ~70 lines of duplicated job steps — alpha-cli is a verbatim copy of stable-cli with one step different; refactor to a matrix strategy

[claude]

This trigger is correctly flagged as "temporary" in the inline comment, but it lives in the file that's being merged. If this PR is approved and merged without removing these lines first, the feat/cli-smoke-test branch will no longer exist — so there's no immediate harm — but if anyone creates a branch with that same name in the future they'll trigger full alpha deploys on every push.

More importantly, the # temporary: remove before merging to main comment is easy to miss during a busy merge. Consider removing it now (in this PR) rather than relying on remembering to do it later:

Suggested change

branches: [feat/cli-smoke-test] # temporary: remove before merging to main

(Remove the push: trigger block entirely — schedule: + workflow_dispatch: are the two triggers this workflow needs long-term.)

[claude]

The deploy step doesn't pin --version "$VERSION", so the CLI will deploy whatever it considers the "latest" published version. Today this is harmless because both jobs publish the same date-based version string. But once the version-collision fix (seconds-precision, separate open thread) lands, each job will publish a unique version. If a second run's Publish step completes between this run's Publish and Deploy steps, this job silently deploys the wrong version — and the health-check passes, masking the regression the pipeline is supposed to catch.

Pin the version to match what was just published:

Suggested change

	--folder-key "${{ env.FOLDER_KEY }}"
	--folder-key "${{ env.FOLDER_KEY }}" \
	--version "$VERSION"

Same fix is needed in the alpha-cli job's Deploy step.

[claude]

Review summary

Two new findings this run (four prior threads remain open):

1. Push trigger should be removed now (line 9) — the push: branches: [feat/cli-smoke-test] trigger is marked temporary but included in the PR. Removing it in this PR avoids a merge-time oversight risk.

2. Deploy missing --version (line 70, and the mirror in alpha-cli) — the deploy step doesn't pass --version "$VERSION", so once the seconds-precision version fix (existing thread) lands, a concurrent run could cause this job to deploy a different version than the one it just published, defeating the regression-detection purpose of the pipeline.

[claude]

Neither stable-cli nor alpha-cli sets timeout-minutes. GitHub's default is 6 hours, so a hung uip codedapp deploy, uip codedapp publish, or network stall will silently occupy a runner until the run is force-cancelled — which could pile up for a daily workflow. Add a reasonable upper bound (30 minutes covers the full lifecycle with headroom):

Suggested change

	runs-on: ubuntu-latest
	runs-on: ubuntu-latest
	timeout-minutes: 30

Same addition needed for the alpha-cli job at its runs-on line.

[claude]

Review summary

Two new findings this run (six prior threads remain open):

1. Token mismatch for GH Packages auth (line 112) — The alpha-cli job uses the built-in GITHUB_TOKEN to authenticate against npm.pkg.github.com, but the PR description lists GH_PACKAGES_TOKEN (a PAT) as a required secret. If the @uipath/cli package is private, GITHUB_TOKEN will get a 401. The token used in code and the setup table in the PR description need to agree.

2. No timeout-minutes on jobs (line 24, and the mirror in alpha-cli) — Without it, a hung CLI step will occupy a GitHub-hosted runner for 6 hours. A 30-minute cap is a safe upper bound for the full pack → publish → deploy → health-check lifecycle.

[claude]

Review summary

One new finding this run (eight prior threads remain open):

if: failure() misses cancelled jobs (line 166) — When timeout-minutes is added (per the open thread on line 24), a hung step produces a cancelled result, not failure. The notify job's if: failure() condition evaluates to false for cancelled results, so no Slack alert fires. The fix is if: failure() || contains(needs.*.result, 'cancelled'), and the inner = "failure" string comparisons at lines 179 and 182 also need to include the cancelled case.

[claude]

Same concern as the push trigger on line 9 — this if: always() is marked "temporary" but will be merged as-is.

With if: always(), the notify job runs on every run, including successful ones. On a success run FAILED_JOBS will be empty, so Slack receives:

:red_circle: *CLI Smoke Test Failed*
Jobs: 
Run: https://...

That's a misleading "CLI Smoke Test Failed" alert with no failed jobs listed, firing every day even when everything is green.

The PR description explicitly says "Posts to #coded-apps-alerts via Slack webhook on failure only". Apply the fix from the open cancelled-jobs thread and set the final condition now:

Suggested change

	if: failure()
	if: failure() || contains(needs.*.result, 'cancelled')

This removes the "temporary" state before merge and also incorporates the cancelled-job fix the other open thread requests.

[claude]

Two problems here:

1. Secret name mismatch with the PR description

The setup table in the PR description documents SLACK_WEBHOOK_URL (an incoming webhook), but the implementation uses SLACK_BOT_TOKEN (a bot token for the chat.postMessage API method). These are entirely different Slack authentication methods. If the repo only has SLACK_WEBHOOK_URL configured (as the docs say), SLACK_BOT_TOKEN will resolve to an empty string and the step will fail with an authentication error. The PR description's setup table needs to document SLACK_BOT_TOKEN (not SLACK_WEBHOOK_URL).

2. "Gracefully skipped if missing" is not implemented

The PR description says: "optional, gracefully skipped if missing". But there is no if: condition on this step — if SLACK_BOT_TOKEN is absent, the step runs and fails. Add a guard to match the documented behavior:

Suggested change

	token: ${{ secrets.SLACK_BOT_TOKEN }}
	- name: Post to Slack
	if: ${{ secrets.SLACK_BOT_TOKEN != '' }}
	uses: slackapi/slack-github-action@v3
	with:
	method: chat.postMessage
	token: ${{ secrets.SLACK_BOT_TOKEN }}

(And update the setup table to list SLACK_BOT_TOKEN, not SLACK_WEBHOOK_URL.)

[claude]

Review summary

Two new findings this run (nine prior threads remain open):

1. if: always() on notify job fires on every run (line 166) — This "temporary" condition (like the push trigger on line 9) is being merged as-is. With if: always(), Slack receives a misleading "CLI Smoke Test Failed" alert with empty FAILED_JOBS on every successful daily run. Should be changed to if: failure() || contains(needs.*.result, 'cancelled') before merge (which also incorporates the fix requested by the open cancelled-jobs thread).

2. SLACK_BOT_TOKEN vs SLACK_WEBHOOK_URL / missing graceful-skip guard (line 187) — The PR setup table documents SLACK_WEBHOOK_URL (incoming webhook) but the code uses secrets.SLACK_BOT_TOKEN (bot token for chat.postMessage). If the repo has only SLACK_WEBHOOK_URL configured, the step fails with an auth error. Additionally, the PR description says Slack is "optional, gracefully skipped if missing" but there is no if: guard on the step.

[claude]

Review summary

One new finding this run (ten prior threads remain open):

Slack secret type mismatch (line 187) — The PR description's setup table lists SLACK_WEBHOOK_URL (incoming webhook) as the required secret, but the implementation uses secrets.SLACK_BOT_TOKEN with method: chat.postMessage. These are different Slack auth mechanisms and are not interchangeable. The setup table and code need to agree on which approach is used. The "gracefully skipped if missing" claim in the PR description is also not implemented — a missing token causes a step failure, not a skip.

[claude]

The Slack channel ID is hardcoded. Like ORG, TENANT, and FOLDER_KEY (see open thread on line 16), this is internal infrastructure detail committed to the repo. If the channel is archived or the workspace migrates, this silently breaks all failure alerts.

Move it to a repository variable:

Suggested change

	channel: "C0AMP09RXH7"
	channel: "${{ vars.SMOKE_TEST_SLACK_CHANNEL_ID }}"

Then add SMOKE_TEST_SLACK_CHANNEL_ID to the setup table in the PR description alongside the other required variables.

[claude]

Review summary

One new finding this run (twelve prior threads remain open):

Hardcoded Slack channel ID (line 187) — channel: "C0AMP09RXH7" is an internal infrastructure detail committed to the repo. If the channel is archived or the workspace changes, all failure alerts break silently. Should be ${{ vars.SMOKE_TEST_SLACK_CHANNEL_ID }} — consistent with the open request to move ORG/TENANT/FOLDER_KEY to repository variables.

[claude]

Review summary

Four previously-resolved threads were reopened because the requested changes were not applied in the current diff:

1. Hardcoded ORG/TENANT/FOLDER_KEY — ORG: popoc, TENANT: adetenant, FOLDER\_KEY: 8645d674-... are still hardcoded on lines 13–15. The requested fix (move to vars.* repository variables) was not applied.

2. Duplicate job steps — alpha-cli remains a near-verbatim copy of stable-cli with only the "Install CLI" step differing. The matrix strategy refactor was not applied.

3. Missing timeout-minutes — Neither stable-cli nor alpha-cli sets timeout-minutes. GitHub's 6-hour default still applies.

4. Missing graceful-skip guard on Slack step — The Post to Slack step has no if: ${{ secrets.SLACK_BOT_TOKEN != '' }} guard. A missing token causes a step failure rather than a silent skip, contradicting the documented "optional" behavior.

Items that were fixed and stay resolved: version collision (now uses hardcoded 1.0.0 + unique APP_NAME per run_number), failure() || cancelled condition on the notify job, and the GH_PACKAGES_TOKEN / GITHUB_TOKEN documentation alignment.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud