Feature to run extra-steps

[treezio]

I don't know if this is totally out of the updatecli scope, but I'm facing an scenario in our organization that might fit with updatecli purpose.

Currently updatecli is automatically running as a GitHub Actions workflow taking care of upgrading our system dependencies for Ruby projects, which means bumping:

• Ruby version for Docker base image
• bumping .ruby-version file for rbenv local env config
• bumping ruby version for the project in Gemfile

Updating Gemfile requires re-generating Gemfile.lock in order to keep dependencies consistency. This is currently being manually assessed, which means checking out the updatecli branch locally, running bundle update --ruby command and commiting changes.

I was thinking that maybe it could be possible that somehow updatecli runs this required extra steps so all changes required for an update to be completed are automated on the apply process.

I could do this as extra steps in a github actions workflow but as the PR is already created by updatecli before manipulating the "extra-files" it might be a bit complex to implement.

I hope this makes sense somehow,
Thank you!

[dduportal]

Quick question : is it possible to run this « extra step » as part of your shell target?
(Moving the command to a script for instance)

long term, it means a native ruby provider in updatecli would help, to take care of both files (a bit like golang)

[treezio]

Yes, I think that might fit my use case. Currently taking a look at @olblak proposal 👇🏻

[olblak]

Hi @treezio,

It's definitely possible to use the shell plugin, I have many examples using that approach.

For example the npm autodiscovery plugin fallbacks to running npm command if it detects a package-lock.json

name: Bump "exec-bin" package version
sources:
    npm:
        name: Get "exec-bin" package version
        kind: npm
        spec:
            name: exec-bin
            versionfilter:
                kind: semver
                pattern: ^1.0.0
targets:
    package-lock.json:
        name: Bump "exec-bin" package version to {{ source "npm" }}
        kind: shell
        spec:
            changedif:
                kind: file/checksum
                spec:
                    files:
                        - package-lock.json
                        - package.json
            command: npm install --package-lock-only --dry-run=$DRY_RUN exec-bin@{{ source "npm" }}
            environments:
                - name: PATH
            workdir: .
        disablesourceinput: true

The changedif is trick here which allow to decide if the target was changed are not by monitoring the files package-lock.json and package.json. They are other changedif kind if needed.
Shame on me I never documented that part

[treezio]

This looks pretty good, not sure if there is a similar approach for bundler, I'll take a look and try to implement something similar!

Otherwise I'd need Gemfile to be updated with the new ruby version before running the bundle check command, which takes place in the targets runtime.

Is it possible to use the output file from an updated target in another target?

something like:

-> update Gemfile with $source_ruby_version
-> update Gemfile.lock running bundle update --ruby over the Gemfile updated in the previous target

[olblak]

Otherwise as @dduportal suggested another option would be to run an embedded script like on
https://github.com/olblak/updatecli-policy-docusaurus/blob/main/policy/updatecli.d/docusaurus.yaml

But IMHO it would be nice to better understand your usecase because maybe we could add a ruby integration.

Like a new ruby plugin that integrate with https://guides.rubygems.org/rubygems-org-api/ and a new ruby autodiscovery plugin that suggest update for each entries from a Gemfile

[treezio]

Well, let me know if I can help somehow, I'm happy to answer any questions :)

Although as of today we delegate most of Gemfile dependencies management to dependabot, updatecli takes care of updating ruby versions, which requires updating Gemfile and Gemfile.lock

Snippet from a Gemfile:

source "https://rubygems.org"
ruby "3.2.2"
gem "rails", "~> 6.1.7.6"

snippet from the related Gemfile.lock

RUBY VERSION
   ruby 3.2.2p53

Gemfile.lock is managed by bundler meaning it should not be manually updated, that's why I want to implement this "Extra Step"

[olblak]

I just check and apparently some documentation is missing so a few things that may help you writing the right pipeline.

Using dependson, you can specify that one target must be executed after another one.
When combined with dependsonchange the second target will only be executed if the first one is in a "changed" state

Example

updatecli/e2e/updatecli.d/success.d/command.yaml

Line 90 in ceea46e

6:

And finally you can have conditional targets using conditionsid with yet another example on https://github.com/updatecli/updatecli/blob/main/e2e/updatecli.d/success.d/conditionalTarget.yaml

[treezio]

Thank you for this information, this comes really handy 🙏🏻

[treezio]

I'll need to rethink the approach for this scenario.
There is a dependency with the ruby version installed when running bundler actions regarding ruby version updates in the Gemfile.

This means that in case i want to update for instance from 3.2.2 to 3.2.3, I would:

1. Update ruby version in Gemfile lock
2. Install/Set Ruby version 3.2.3 locally
3. Run bundle ruby --update to update ruby version in Gemfile.lock

The tricky thing here is that I'd need to install the ruby version to be updated (step 2) so I don't get this error when running bundle ruby --update:

bundle update --ruby
Your Ruby version is 3.2.2, but your Gemfile specified 3.2.3

This means that I'd need to know the ruby version to be updated beforehand in order to create the environment in the CI System, and that's what I haven't been able to figure out how to do it.

GH Actions workflow using the 3.2.2 to 3.2.3 example:

1. Setup Ruby 3.2.3 version Environment -> Which I need to figure out how to discover this value before running updatecli.
2. Run updatecli

• Bump Gemfile
• Run bundle update --ruby (using 3.2.3 runtime version)

Another option could be doing this in different Pull Requests, but I don't like that solution.

Thank you again for all your support guys, you are awesome!

[olblak]

This means that I'd need to know the ruby version to be updated beforehand in order to create the environment in the CI System, and that's what I haven't been able to figure out how to do it.

I faced similar issues on some project, where the update policy also needed to update version used by CI.
For example this policy: ghcr.io/updatecli/policies/golang/version:0.1.0 bump the Golang version and the one specificied in GitHub action workflow. You can run updatecli manifest show ghcr.io/updatecli/policies/golang/version:0.1.0 to see the content of the policy.

Also to retrieve the latest ruby version, there are different ways to retrieve it but I am aware of this manifest on the jenkins project

• https://github.com/jenkins-infra/jenkins.io/blob/326a039811b58f85b0415fb836127e8a397e9a8f/updatecli/updatecli.d/ruby.yaml#L43

Because you can't use the githubrelease plugin because version published there don't follow semantic versioning and the ruby project maintain multiple release line at the same time so nothing guarantee you that the "latest" one is the one you are looking for.

Imho it would make sense to have a specific Updatecli ruby plugin to retrieve the latest version, similarly to Golang or Jenkins so we can also retrieve changelogs

[olblak]

Also today I discovered this yaml file that contain Ruby versions https://raw.githubusercontent.com/ruby/www.ruby-lang.org/master/_data/downloads.yml

So you could use something like https://gist.github.com/olblak/ecd251a3d13b0db4652e2ca5eecf7cf9 to retrieve the stable version for the previous release line