[2.14 main] Removing dapper from the build process

[crobby]

Part of: https://github.com/rancher/rancher-security/issues/1381
rancher/rancher#49201
Removing dapper from our build processes in favor of docker buildx staged builds to facilitate both local and CI needs.

[tomleb]

I know it's a draft and not ready to review yet but before I forget. One reason why we want to move away from dapper is that we want to also move the binary building inside of the package/Dockerfile. Right now, that Dockerfile is doing a copy of a binary built with make build, but really, we'd want the Dockerfile to build the binary.

To avoid having two separate go build command (since people will want to build the binary too), we can go the similar route as rancher/rancher where it has a step to build the binary and another to build the image. (eg: server build target: https://github.com/rancher/rancher/blob/main/package/Dockerfile#L381 which builds the binary, then either the binary is copied to scratch (to get it out to the host), or it's copied in the rancher server target.

Anyway, we can discuss further if you want.

[crobby]

I know it's a draft and not ready to review yet but before I forget. One reason why we want to move away from dapper is that we want to also move the binary building inside of the package/Dockerfile. Right now, that Dockerfile is doing a copy of a binary built with make build, but really, we'd want the Dockerfile to build the binary.

To avoid having two separate go build command (since people will want to build the binary too), we can go the similar route as rancher/rancher where it has a step to build the binary and another to build the image. (eg: server build target: https://github.com/rancher/rancher/blob/main/package/Dockerfile#L381 which builds the binary, then either the binary is copied to scratch (to get it out to the host), or it's copied in the rancher server target.

Anyway, we can discuss further if you want.

I think that makes good sense. Thanks for the early feedback.

[tomleb]

Sorry for this constant back-and-forth lol, but I think it's starting to look real good 👍

Some more comments.

We might benefit from input from someone doing webhook work too, ensure they're still able to do what they used to do. Maybe check with Peter?

[crobby]

@ericpromislow or @pmatseykanets Can one or both of you give this a whirl on your mac? We want to make sure that this refactoring of our build process doesn't adversely affect anyone. Thanks.

[pmatseykanets]

@crobby Seems to be working on my Mac.

[crobby]

@crobby Seems to be working on my Mac.

Thanks for checking

[tomleb]

A couple of nits, but the most important is .github/workflows/release.yaml will probably fail because the docker build step expects VERSION and COMMIT given to it to build the binary, but we're not passing it to it. Right?

[crobby]

A couple of nits, but the most important is .github/workflows/release.yaml will probably fail because the docker build step expects VERSION and COMMIT given to it to build the binary, but we're not passing it to it. Right?

The bigger problem is that release.yaml still calls the old scripts/build. I'll have to adapt that step to the new setup, which will include correct VERSION/COMMIT getting set.

[tomleb]

A couple of nits, but the most important is .github/workflows/release.yaml will probably fail because the docker build step expects VERSION and COMMIT given to it to build the binary, but we're not passing it to it. Right?

The bigger problem is that release.yaml still calls the old scripts/build. I'll have to adapt that step to the new setup, which will include correct VERSION/COMMIT getting set.

Ah right I didn't even see that. Yeah that should be moved over to the "new way" before merging 👍 I'll approve once that's done

[tomleb]

LGTM

nit: make image should probably remain make package to not break devs workflow but heh 🤷