Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add image builder dependency version tracker #2695

Merged
merged 3 commits into from
Jan 9, 2024

Conversation

jaxesn
Copy link
Member

@jaxesn jaxesn commented Nov 28, 2023

Issue #, if available:

Description of changes:

We have had a number of users have trouble getting all the image-builder dependencies correct. I suspect this is due to users using the same admin machine across multiple versions of eks-a/image-builder. For example, in 0.18 we had to start requiring a newer ansible, which also requires python 3.9. If a user starts on a fresh machine, image builder will properly install packer/ansible/plugins for them and this is not an issue. However, say they had created an image with 0.17 which would have installed an older version of packer than 0.18 requires, well the image-builder ensure-packer script only looks for the executable, not the specific version.

The code changes here are simple. The target that image-builder cli calls (ex: local-build-nutanix-redhat-8) first calls dep-<format> which will run all the upstream scripts to install the deps and then runs our new script to actually ensure the versions match what we expect.

This also adds a new target to the build_targets that will regenerate and compare this new yaml file so that when we bump image-builder git_tag we keep the yaml up to date with the proper versions.

I think in our world its better if we get strict here and error out (with a bypass) so that users arent spinning their wheels dealing with bad configurations.

This PR also adds output of the currently install tools for us to use when debugging with people, it looks like this:

************************************************
The following are key dependencies and their current versions
Please include this output in any support requests
Python /usr/bin/python3 Python 3.10.12
Packer ./image-builder/images/capi/.local/bin/packer 1.9.4
Packer Plugins:
/home/jackson/.packer.d/plugins/github.com/nutanix-cloud-native/nutanix/packer-plugin-nutanix_v0.8.0_x5.0_linux_amd64
/home/jackson/.packer.d/plugins/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64
/home/jackson/.packer.d/plugins/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.1_x5.0_linux_amd64
Ansible:
ansible [core 2.15.3]
  config file = /home/jackson/eks-anywhere-build-tooling/projects/kubernetes-sigs/image-builder/ansible.cfg
  configured module search path = ['/home/jackson/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jackson/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/jackson/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jackson/.local/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True
Ansible Galaxy Collection:

# /home/jackson/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
amazon.aws        5.5.0  
ansible.posix     1.5.4  
ansible.windows   2.1.0  
community.general 8.0.2  
community.windows 2.0.0  
************************************************

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@eks-distro-bot eks-distro-bot added do-not-merge/work-in-progress size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 28, 2023
@jaxesn
Copy link
Member Author

jaxesn commented Nov 29, 2023

/retest

2 similar comments
@jaxesn
Copy link
Member Author

jaxesn commented Nov 29, 2023

/retest

@jaxesn
Copy link
Member Author

jaxesn commented Nov 29, 2023

/retest

@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from 7822fb3 to 68d144e Compare November 29, 2023 17:09
@eks-distro-bot eks-distro-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 29, 2023
@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from e13944b to fbbb436 Compare November 30, 2023 21:14
@jaxesn jaxesn changed the title WIP: add image builder dependency version tracker add image builder dependency version tracker Nov 30, 2023
@jaxesn
Copy link
Member Author

jaxesn commented Nov 30, 2023

/retest

@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from fbbb436 to 299a323 Compare December 1, 2023 22:53
@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from 299a323 to 1467cc2 Compare December 20, 2023 18:03
@eks-distro-bot eks-distro-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 20, 2023
@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from 2da14f3 to 2758330 Compare January 4, 2024 19:38
@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from 2758330 to c85b672 Compare January 4, 2024 19:49
@jaxesn
Copy link
Member Author

jaxesn commented Jan 4, 2024

/test all

@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch 2 times, most recently from b90b344 to 53393d9 Compare January 4, 2024 22:29
@jaxesn
Copy link
Member Author

jaxesn commented Jan 5, 2024

/test all

1 similar comment
@jaxesn
Copy link
Member Author

jaxesn commented Jan 5, 2024

/test all

@abhay-krishna
Copy link
Member

/retest

@eks-distro-bot
Copy link
Collaborator

eks-distro-bot commented Jan 5, 2024

@jaxesn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
imagebuilder-1-24-presubmit 2da14f3 link true /test imagebuilder-1-24-presubmit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from 53393d9 to 4deebdb Compare January 5, 2024 19:17
@jaxesn jaxesn force-pushed the jgw/add-dep-versions branch from 4deebdb to 0ced2ab Compare January 8, 2024 22:21
@abhay-krishna
Copy link
Member

/lgtm
/approve

@eks-distro-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhay-krishna

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@eks-distro-bot eks-distro-bot merged commit 6833188 into aws:main Jan 9, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants