Switch to using opam-ci-check to generate the list of revdeps #403
Conversation
I've pulled out the changes to allow using the default opam root into #404. This PR can be reviewed once that has been merged. |
punchagan
force-pushed
the
opam-ci-check-revdeps
branch
2 times, most recently
from
d3fc664 to
9c9cb32
Compare
There was a problem hiding this comment.
This change looks great. Thanks for the tiny diff!
In the tests (which need to be updated), we now lose visibility of what revdeps are going to be tested. Do you think it would be easy enough to rig up a cram test for the the revdeps listing of opam-ci-check to go along with this change, so we can ensure we are inferring the revdeps we think we should?
punchagan
force-pushed
the
opam-ci-check-revdeps
branch
from
9c9cb32 to
fb23028
Compare
test/specs.expected
Outdated
| @@ -1106,7 +1106,8 @@ list-revdeps: debian-12-ocaml-4.14/amd64 opam-dev | |||
| COPY --chown=1000:1000 . opam-repository/ | |||
| RUN opam repository set-url --strict default opam-repository/ | |||
| RUN opam update --depexts || true | |||
| RUN echo '@@@OUTPUT' && opam list -s --color=never --depends-on 'a.0.0.1' --coinstallable-with 'a.0.0.1' --all-versions --depopts && opam list -s --color=never --depends-on 'a.0.0.1' --coinstallable-with 'a.0.0.1' --all-versions --recursive && opam list -s --color=never --depends-on 'a.0.0.1' --coinstallable-with 'a.0.0.1' --all-versions --with-test --depopts && echo '@@@OUTPUT' | |||
| RUN opam pin add -k git -y opam-ci-check git+https://github.com/ocurrent/opam-repo-ci.git#live | |||
There was a problem hiding this comment.
This step could get cached and in a Dockerfile we could use the ADD command to work around the caching:
ADD https://api.github.com/repos/punchagan/opam-repo-ci/git/refs/heads/revdeps-exp version.jsonbut obuilder spec doesn't seem to support ADD and I'm not sure there's a nice way to do this pinning with commit hashes since we'd want to pin to the commit after the merge.
There was a problem hiding this comment.
We discussed three ideas in our co-working meeting:
- Extending obuilder with
add - Some way to ensure that previous actions in docker container would invalidate the cache when needed
- Deploying a docker image alongside the other deployment with
opam-ci-checkand using that as the base
Instead of using all_known_packages to list availability, we can simply use available packages since we are currently only testing on a single platform and we try to install the packages to test that the build works. For unavailable packages, the installation wouldn't work anyway.
We add an opam pin to install opam-ci-check in the worker running the revdeps listing, and use opam-ci-check to list the reverse dependencies.
punchagan
force-pushed
the
opam-ci-check-revdeps
branch
from
fb23028 to
07ac60a
Compare
When running `opam-ci-check` in workers, we need to ensure we are running the right version of that tool. We cannot simply pin it to the `live` branch, because obuilder/docker might cache a previous install from the command, and fail to pick up an update. On the other hand, we do want caching, so we don't want to always install fresh. As a workaround to meet these requirements given our current code organization and architecture, this adds a build rule that depends on the `opam-ci-check` executable (so should only update when that changes), and will embed the commit hash of repo as a value, which we then thread thru for when we pin the executable.
Closes https://github.com/tarides/infrastructure/issues/422
This PR switches to using opam-ci-check to list reverse dependencies from the 3 command opam list invocation we previously had. The output is slightly different from the previous output in that it contains only available packages instead of all packages. The PR also changes
opam-ci-checkCLI to accept an additionaluse-default-rootargument to prevent creating a new opam root for running it's commands.