Skip to content

Commit

Permalink
update Fail workflow if changes are found to clairfy that exitZeroOnC…
Browse files Browse the repository at this point in the history
…hanges is set to true by default
  • Loading branch information
winkerVSbecks committed Dec 4, 2024
1 parent e33302c commit cbd200b
Showing 1 changed file with 28 additions and 30 deletions.
58 changes: 28 additions & 30 deletions src/content/ci/github-actions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -464,33 +464,9 @@ Read the official [Storybook environment variable's documentation](https://story

</div>

### Recommended configuration for build events

GitHub's Actions, like other CI systems, can run based on any type of event. Our recommendation is to run Chromatic's step on `push` events.
### Fail workflow if changes are found

While the `pull_request` event also works, it can cause Chromatic's baselines to be lost in certain scenarios or cause Chromatic to use an unexpected baseline from the `main` branch.

GitHub allows workflows to execute on commits pushed to a branch in a pull request or on "merge" commits between that branch and the base branch (main). These merge commits do not persist in your repository's history. When a GitHub action is triggered on `pull_request`, GitHub creates an ephemeral branch and merges the PR branch into the latest from `main`. For details, refer to the [pull request workflow docs](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request) to understand what `GITHUB_SHA` and `GITHUB_REF` are set to.

This means the latest build on main will be an ancestor of this ephemeral branch, even though it is not an ancestor of the actual PR branch. Therefore, it'll use the incorrect baseline for comparison. That's why we recommend running Chromatic using the `push` event.

If you decide to use the `pull_request` event, we recommend creating a separate workflow for Chromatic using the following strategy for the checkout step:

```yml
- uses: actions/checkout@v3
with:
fetch-depth: 0
# 👇 Tells the checkout which commit hash to reference
ref: ${{ github.event.pull_request.head.ref }}
```

### UI Test and UI Review

[UI Tests](/docs/test) and [UI Review](/docs/review) rely on [branch and baseline](/docs/branching-and-baselines) detection to keep track of [snapshots](/docs/snapshots). We recommend the following configuration.

#### Command exit code for "required" checks

If you are using pull request statuses as required checks before merging, you may not want your workflow step to fail if test snapshots render without errors (but with changes). To achieve this, pass the option `exitZeroOnChanges` to the `with` clause, and your workflow will continue in such cases. For example:
If you are using pull request statuses as required checks before merging, you may want your workflow step to fail if test snapshots show changes. By default, Chromatic exits with code `0` if changes are detected. We expect users to review these changes and decide whether to accept or reject them. However, if you prefer the workflow to fail when changes are found, you can use the `exitZeroOnChanges` option. For example:

```yml title=".github/workflows/chromatic.yml"
# Other necessary configuration
Expand All @@ -504,7 +480,7 @@ jobs:
uses: chromaui/action@latest
with:
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
exitZeroOnChanges: true # 👈 Option to prevent the workflow from failing
exitZeroOnChanges: false # 👈 Fail workflow if changes are found
```

<div class="aside">
Expand All @@ -513,14 +489,36 @@ Read about the [available options](/docs/configure/#options).

</div>

When using `exitZeroOnChanges`, your workflow will still stop and fail if your Storybook contains stories that error.

#### Re-run failed builds after verifying UI test results

Builds that contain visual changes need to be [verified](/docs/test#verify-ui-changes). They will fail if you are not using the `exitZeroOnChanges` option. Once you accept all the changes, re-run the workflow, and the `chromatic` job will pass.
Builds that contain visual changes need to be [verified](/docs/test#verify-ui-changes). The task will fail if `exitZeroOnChanges` option is set to false. Once you accept all the changes, re-run the workflow, and the `chromatic` job will pass.

If you deny any change, you will need to make the necessary code changes to fix the test (and thus start a new run) to get Chromatic to pass again.

### Recommended configuration for build events

GitHub's Actions, like other CI systems, can run based on any type of event. Our recommendation is to run Chromatic's step on `push` events.

While the `pull_request` event also works, it can cause Chromatic's baselines to be lost in certain scenarios or cause Chromatic to use an unexpected baseline from the `main` branch.

GitHub allows workflows to execute on commits pushed to a branch in a pull request or on "merge" commits between that branch and the base branch (main). These merge commits do not persist in your repository's history. When a GitHub action is triggered on `pull_request`, GitHub creates an ephemeral branch and merges the PR branch into the latest from `main`. For details, refer to the [pull request workflow docs](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request) to understand what `GITHUB_SHA` and `GITHUB_REF` are set to.

This means the latest build on main will be an ancestor of this ephemeral branch, even though it is not an ancestor of the actual PR branch. Therefore, it'll use the incorrect baseline for comparison. That's why we recommend running Chromatic using the `push` event.

If you decide to use the `pull_request` event, we recommend creating a separate workflow for Chromatic using the following strategy for the checkout step:

```yml
- uses: actions/checkout@v3
with:
fetch-depth: 0
# 👇 Tells the checkout which commit hash to reference
ref: ${{ github.event.pull_request.head.ref }}
```

### UI Test and UI Review

[UI Tests](/docs/test) and [UI Review](/docs/review) rely on [branch and baseline](/docs/branching-and-baselines) detection to keep track of [snapshots](/docs/snapshots). We recommend the following configuration.

#### Maintain a clean "main" branch

A clean `main` branch is a development **best practice** and **highly recommended** for Chromatic. This means testing your `main` branch to ensure builds are passing. It's important to note that baselines will not persist through branching and merging unless you test your `main` branch.
Expand Down

0 comments on commit cbd200b

Please sign in to comment.