Creating a new launch template version can fail if there are over 200 existing versions #2131

matiaskorhonen · 2024-07-31T11:03:43Z

Summary

If there are over 200 launch template versions and the source_version is old enough, creating a new launch template version will fail due to the maximum number of versions that is returned by default in Boto3 (MaxResults defaults to the maximum, which is 200)

Issue Type

Bug Report

Component Name

ec2_launch_template

Ansible Version

$ ansible --version
ansible [core 2.17.2]
  config file = /home/runner/work/<redacted>/<redacted>/ansible.cfg
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/ansible
  ansible collection location = /home/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/hostedtoolcache/Python/3.11.9/x64/bin/ansible
  python version = 3.11.9 (main, Jul 15 2024, 21:50:21) [GCC 11.4.0] (/opt/hostedtoolcache/Python/3.11.9/x64/bin/python)
  jinja version = 3.1.4
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
# /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               8.1.0  
ansible.netcommon                        6.1.3  
ansible.posix                            1.5.4  
ansible.utils                            4.1.0  
ansible.windows                          2.4.0  
arista.eos                               9.0.0  
awx.awx                                  24.6.1 
azure.azcollection                       2.6.0  
check_point.mgmt                         5.2.3  
chocolatey.chocolatey                    1.5.1  
cisco.aci                                2.10.1 
cisco.asa                                5.0.1  
cisco.dnac                               6.16.0 
cisco.intersight                         2.0.9  
cisco.ios                                8.0.0  
cisco.iosxr                              9.0.0  
cisco.ise                                2.9.2  
cisco.meraki                             2.18.1 
cisco.mso                                2.8.0  
cisco.nxos                               8.1.0  
cisco.ucs                                1.10.0 
cloud.common                             3.0.0  
cloudscale_ch.cloud                      2.3.1  
community.aws                            8.0.0  
community.ciscosmb                       1.0.9  
community.crypto                         2.21.0 
community.digitalocean                   1.26.0 
community.dns                            3.0.2  
community.docker                         3.11.0 
community.general                        9.2.0  
community.grafana                        1.9.1  
community.hashi_vault                    6.2.0  
community.hrobot                         2.0.1  
community.library_inventory_filtering_v1 1.0.1  
community.libvirt                        1.3.0  
community.mongodb                        1.7.5  
community.mysql                          3.9.0  
community.network                        5.0.3  
community.okd                            3.0.1  
community.postgresql                     3.4.1  
community.proxysql                       1.6.0  
community.rabbitmq                       1.3.0  
community.routeros                       2.17.0 
community.sap_libs                       1.4.2  
community.sops                           1.8.0  
community.vmware                         4.5.0  
community.windows                        2.2.0  
community.zabbix                         2.5.1  
containers.podman                        1.15.4 
cyberark.conjur                          1.3.0  
cyberark.pas                             1.0.25 
dellemc.enterprise_sonic                 2.4.0  
dellemc.openmanage                       9.4.0  
dellemc.powerflex                        2.5.0  
dellemc.unity                            2.0.0  
f5networks.f5_modules                    1.29.0 
fortinet.fortimanager                    2.5.0  
fortinet.fortios                         2.3.7  
frr.frr                                  2.0.2  
google.cloud                             1.3.0  
grafana.grafana                          5.3.0  
hetzner.hcloud                           3.1.1  
ibm.qradar                               3.0.0  
ibm.spectrum_virtualize                  2.0.0  
ibm.storage_virtualize                   2.4.1  
ieisystem.inmanage                       2.0.0  
infinidat.infinibox                      1.4.5  
infoblox.nios_modules                    1.6.1  
inspur.ispim                             2.2.3  
inspur.sm                                2.3.0  
junipernetworks.junos                    8.0.0  
kaytus.ksmanage                          1.2.2  
kubernetes.core                          3.2.0  
kubevirt.core                            1.5.0  
lowlydba.sqlserver                       2.3.3  
microsoft.ad                             1.6.0  
netapp.cloudmanager                      21.22.1
netapp.ontap                             22.11.0
netapp.storagegrid                       21.12.0
netapp_eseries.santricity                1.4.0  
netbox.netbox                            3.19.1 
ngine_io.cloudstack                      2.3.0  
ngine_io.exoscale                        1.1.0  
openstack.cloud                          2.2.0  
openvswitch.openvswitch                  2.1.1  
ovirt.ovirt                              3.2.0  
purestorage.flasharray                   1.30.0 
purestorage.flashblade                   1.17.0 
sensu.sensu_go                           1.14.0 
splunk.es                                3.0.0  
t_systems_mms.icinga_director            2.0.1  
telekom_mms.icinga_director              2.1.2  
theforeman.foreman                       4.0.0  
vmware.vmware                            1.3.0  
vmware.vmware_rest                       3.0.1  
vultr.cloud                              1.13.0 
vyos.vyos                                4.1.0  
wti.remote                               1.0.5

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.34.151
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.34.151
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /home/runner/work/<redacted>/<redacted>/ansible.cfg
LOCALHOST_WARNING(/home/runner/work/<redacted>/<redacted>/ansible.cfg) = False

OS / Environment

The target OS is irrelevant here

Steps to Reproduce

Use the ec2_launch_template to create a new version of a launch template, repeat until there are over 200 versions of the launch template

- name: Create new launch template version for the defined AMI
  register: created_launch_template
  ec2_launch_template:
    template_id: "lt-123"
    image_id: "ami-123"
    version_description: "Some new version description {{ now(utc=true,fmt='%Y-%m-%d %H:%M:%S') }}"
    source_version: 1
    default_version: 1

Expected Results

The new version of the launch template should be created instead of failing

Actual Results

One the pagination limit has been reached, the version creation will fail with:

fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": { … snip ...  }
    },
    "msg": "source_version does not exist, got \"1\""
}

This happens even though the version does exist on AWS.

ec2.describe_launch_template_versions only returns the last 200 versions in:

community.aws/plugins/modules/ec2_launch_template.py

Lines 505 to 507 in 97131ec

    
           ec2.describe_launch_template_versions(LaunchTemplateId=template_id, aws_retry=True)[ 
        
               "LaunchTemplateVersions" 
        
           ],

Which causes the version number validation to erroneously fail here:

community.aws/plugins/modules/ec2_launch_template.py

Lines 633 to 636 in 97131ec

    
           if source_version is None: 
        
               module.fail_json( 
        
                   msg=f"source_version does not exist, got \"{module.params.get('source_version')}\"" 
        
               )

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

markuman · 2024-08-01T05:56:21Z

Sounds like pagination with NextToken must be implemented here.
@matiaskorhonen do you have time to implement it?
There are already some implementations like that https://github.com/search?q=repo%3Aansible-collections%2Fcommunity.aws%20NextToken&type=code

abikouo · 2024-10-03T15:29:50Z

@matiaskorhonen This will be addressed with with the refactoring of the module by #2164 and ansible-collections/amazon.aws#2319

…#2164) SUMMARY Depends-On: ansible-collections/amazon.aws#2319 Add some type hint for the module Use shared code from amazon.aws.plugins.module_utils.ec2 Add the possibility to delete specific version of a launch template Add support for tagging for launch template resource (Closes ansible-collections#176) Add the possibility to tag specific resources, not always instance and volume (Closes [ansible-collections#48](ansible-collections#48, Closes ansible-collections#2083) Support EBS Throughput (Closes ansible-collections#1944) Fix issue occurring when launch template contains more than 200 versions (Closes ansible-collections#2131) ISSUE TYPE Feature Pull Request COMPONENT NAME ec2_launch_template Reviewed-by: Alina Buzachis Reviewed-by: Bikouo Aubin Reviewed-by: GomathiselviS <[email protected]>

matiaskorhonen changed the title ~~Creating a new launch template version can fail of there are over 200 existing versions~~ Creating a new launch template version can fail if there are over 200 existing versions Jul 31, 2024

matiaskorhonen mentioned this issue Aug 1, 2024

Pagination for describe_launch_template_versions #2133

Closed

abikouo added the has_pr label Oct 3, 2024

abikouo mentioned this issue Oct 7, 2024

Prepare module ec2_launch_template for promotion #2164

Merged

softwarefactory-project-zuul bot closed this as completed in #2164 Oct 18, 2024

softwarefactory-project-zuul bot closed this as completed in 40d61f0 Oct 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Creating a new launch template version can fail if there are over 200 existing versions #2131

Creating a new launch template version can fail if there are over 200 existing versions #2131

matiaskorhonen commented Jul 31, 2024 •

edited

Loading

markuman commented Aug 1, 2024

abikouo commented Oct 3, 2024

Creating a new launch template version can fail if there are over 200 existing versions #2131

Creating a new launch template version can fail if there are over 200 existing versions #2131

Comments

matiaskorhonen commented Jul 31, 2024 • edited Loading

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

markuman commented Aug 1, 2024

abikouo commented Oct 3, 2024

matiaskorhonen commented Jul 31, 2024 •

edited

Loading