Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ubuntu setup #432

Merged
merged 8 commits into from
Nov 19, 2024
Merged

Conversation

cpuguy83
Copy link
Member

Extracts all the jammy logic into reusable components.
See individual commits for more details.

@cpuguy83 cpuguy83 requested a review from a team as a code owner November 13, 2024 00:54
@cpuguy83 cpuguy83 added this to the v0.11.0 milestone Nov 13, 2024
@cpuguy83 cpuguy83 force-pushed the refactor_ubuntu_setup branch 2 times, most recently from 7e8bfd1 to 15a8906 Compare November 13, 2024 00:59
apt autoclean -y

apt update
apt install -y $@
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
apt install -y $@
apt install -y "$@"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quoting here would turn a list of packages into one single argument to apt which I don't think we want.

Copy link
Contributor

@pmengelbert pmengelbert Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$@ has special behavior when double-quoted, see the manual

From the manual:

That is, "$@" is equivalent to "$1" "$2" [... and so on]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL

Copy link
Contributor

@pmengelbert pmengelbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work overall. But I really think we need to find a solution for installing packages without introducing implicit parameter ordering.

@cpuguy83
Copy link
Member Author

All updated.

@pmengelbert
Copy link
Contributor

Needs a rebase and it's failing a test, but I'll approve once that's sorted.

@cpuguy83
Copy link
Member Author

This is rebased.
The failure was a network flake.

Copy link
Contributor

@adamperlin adamperlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One typo that needs to be fixed in the install-with-constraints script, and a few other small comments.

@cpuguy83
Copy link
Member Author

Rebased and comments addressed.

This does actually end up changing the ordering, which I think is more
correct.
If a client has supplied a context with an image name then we should
assume we need to install the distro's base build packages into it,
unlike when the worker context ref is used where we expect that the
provided image is the actual worker.

Signed-off-by: Brian Goff <[email protected]>
This was always a bit janky.
We really can't guarentee that all implementations package repos from a
worker to install images into a target, and this work-around was just to
make the test pass.
Instead the test is now just deleted since it doesn't make sense.

While it is technically possible to do this on azlinux people should be
using the custom repo support add extra repos.

Signed-off-by: Brian Goff <[email protected]>
This allows us to have a common implementation of a bunch of the things
that underpins building a deb package.

Signed-off-by: Brian Goff <[email protected]>
This moves the rest of the logic from the jammy package into the distro
config.

Signed-off-by: Brian Goff <[email protected]>
Do not require correct ordering of constraints options for apt/deb
functions.

Signed-off-by: Brian Goff <[email protected]>
@cpuguy83
Copy link
Member Author

Rebased again.

Copy link
Contributor

@adamperlin adamperlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good to me -- the way the functionality is encapsulated seems very clean and reusable. Personally, I am not too concerned about the parameter ordering issue. To me it makes sense that in the Install run options the constraints attached to the exec op will be applied to the child operations first, after which user specified constraints will be applied to potentially override

tmp := llb.Scratch().File(llb.Mkfile("tmp", 0o644, nil), opts...)
// Warning: HACK here
// The base ubuntu image has this `excludes` config file which prevents
// installation of a lot of thigns, including doc files.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// installation of a lot of thigns, including doc files.
// installation of a lot of things, including doc files.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

if err != nil {
t.Fatal(err)
}
solveT(ctx, t, gwc, sr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: why is this deleted, the res.SingleRef() and err handling? Shouldn't we still do that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's necessary.
If there's a problem with the build solveT will get it.
We don't need to check the actual content of the resulting image since this test is no longer making the assumption that the worker repos will be available at install time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mentioned in the commit where the change is at, btw.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh ok thanks, yeah, I reviewed the whole PR

Copy link
Contributor

@DannyBrito DannyBrito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@pmengelbert pmengelbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pmengelbert pmengelbert merged commit 189f268 into Azure:main Nov 19, 2024
9 checks passed
@cpuguy83 cpuguy83 deleted the refactor_ubuntu_setup branch November 19, 2024 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants