Switch to Docker's build-publish-action in GitHub Actions #2331
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updated action to the latest version.
Added a step printing disk space after the cache is restored. We had problems with executor running out of space. With this step we can debug disk usage after the cache is restored. We assume that available disk space for `/` before cache restore to be at ~20 GB.
We experienced some problems with `satackey/action-docker-layer-caching` that caused workflow runners to get out of a disk space. This is related to the way the action stores layer incrementaly along with the previously cached ones. See: satackey/action-docker-layer-caching#55 There is also a solution from Docker that we can use for building images but also on later stage of RFC-18 implementation to publish images to registires. See: https://github.com/marketplace/actions/build-and-push-docker-images With the new solution we can also use caching and hopefully it won't cause problems we had before.
We don't need to include branch name in the cache key as GitHub cache handles the logic for accesing cachess between branches. Cache matches are performed first for the current branch according to `key` and `restore-keys` and in case of no much it checks caches on the parent and upstream branches. See: [LINK 1] Additionally caches are isolated for siblings branches, so cache from feature branch can't be accessed by another feature branch. See [LINK 2] LINK 1:https://docs.github.com/en/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key LINK 2:https://docs.github.com/en/actions/guides/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
As we expect the arguments to be updating, especially REVISION to be set to the current commit hash we need to define them closer to the place they will be used. Currently if we update a value for REVISION it causes all subsequent layers to be executed and not fetched from cache. There's actualy nothing changin besides the commit hash we pass to go build command, so all the layers can be reused.
…uild-publish-action
michalinacienciala
previously approved these changes
Feb 15, 2021
Initally we added ignore filter for branches with rfc-18 prefix which sounds like a good idea taking into account that we want to run GitHub Actions for such branches, so duplication is not necessary. It turned out that we require CircleCI job to be executed as a check before merging to master. That's why we removed the filter and will execute the job anywas. It shouldn't be much of a problem as the duplication will take place only for workflows on rfc-18 branches as we filter them in GiHub Actions configuration.
michalinacienciala
approved these changes
Feb 15, 2021
michalinacienciala
deleted the
rfc-18/use-docker-build-publish-action
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR we are switching the Go building workflow to Docker's official Build and Publish Action.
We were experiencing some issues with running out of disk space while using
satackey/action-docker-layer-caching. It was happening because the cache size was incremented with each execution. During testing, it grew up pretty quickly to 7 GB. There is an issue describing such a situation satackey/action-docker-layer-caching#55. I tried the proposed workarounds but was not satisfied with them.According to documentation caches will be cleaned up when their total size reaches 5 GB, but in this case, we had just one cache of 7 GB.
I noticed that Docker released an action set that may suit our needs. It also supports layers caching. Additionally, we can use it later for image publication to registries. During testing the cache size was around 1 GB.
What is worth mentioning is a clever way how GH's cache action handles access to caches between branches. Long story short: