Error retrieving resource group xxxxxxxx - API version 2022-09-01 does not have operation group 'resource_groups'

[vijayreddiar]

SUMMARY

Modules under azure.azcollection v3.0.0 fails when resource_group is required. The same is working without any issue when downgraded to v2.7.0

ISSUE TYPE

• Bug Report

COMPONENT NAME

azure_rm_virtualmachine

ANSIBLE VERSION

ansible-playbook [core 2.15.12]
  config file = None
  configured module search path = ['/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/requirements_collections:/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.9.19 (main, Jul 18 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)

COLLECTION VERSION

Collection         Version
------------------ -------
azure.azcollection 3.0.0

CONFIGURATION

CONFIG_FILE() = None

OS / ENVIRONMENT

RHEL 8.8 / AWX v24.6.1 (Installed and configured using awx-on-k3s)

STEPS TO REPRODUCE

Install latest version (v3.0.0) of azure.azcollection and attempt to start or stop an Azure VM using the module azure_rm_virtualmachine

- name: Stop Azure VM
  azure.azcollection.azure_rm_virtualmachine:
    auth_source: "{{ azure_auth_source }}"
    profile: "{{ azure_profile }}"
    resource_group: "{{ azure_resource_group }}"
    name: "{{ vm_name }}"
    started: false
    allocated: "{{ is_allocated }}"

EXPECTED RESULTS

Azure VM should be stopped and the same is working as expected with azure.azcollection v2.7.0

ACTUAL RESULTS

msg: >-
  Error retrieving resource group xxxxxxxx - API version 2022-09-01 does not
  have operation group 'resource_groups'
exception: |2
    File "/tmp/ansible_azure.azcollection.azure_rm_virtualmachine_payload_ypjuda1f/ansible_azure.azcollection.azure_rm_virtualmachine_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 603, in get_resource_group
      return self.rm_client.resource_groups.get(resource_group)
    File "/usr/local/lib/python3.9/site-packages/azure/mgmt/resource/resources/_resource_management_client.py", line 434, in resource_groups
      raise ValueError("API version {} does not have operation group 'resource_groups'".format(api_version))
invocation:
  module_args:
    auth_source: env
    profile: default
    resource_group: xxxxxxxx
    name: xxxxxxxxxxxxxxxxxx
    started: false
    allocated: false
    cloud_environment: AzureCloud
    api_profile: latest
    disable_instance_discovery: false
    append_tags: true
    state: present
    max_price: -1
    ssh_password_enabled: true
    storage_container_name: vhds
    os_type: Linux
    public_ip_allocation_method: Static
    remove_on_absent:
      - all
    created_nsg: true
    restarted: false
    force: false
    generalized: false
    accept_terms: false
    subscription_id: null
    client_id: null
    secret: null
    tenant: null
    ad_user: null
    password: null
    cert_validation_mode: null
    adfs_authority_url: null
    log_mode: null
    log_path: null
    x509_certificate_path: null
    thumbprint: null
    tags: null
    custom_data: null
    location: null
    short_hostname: null
    vm_size: null
    priority: null
    eviction_policy: null
    admin_username: null
    admin_password: null
    ssh_public_keys: null
    image: null
    availability_set: null
    storage_account_name: null
    storage_blob_name: null
    os_disk_caching: null
    os_disk_size_gb: null
    os_disk_encryption_set: null
    managed_disk_type: null
    os_disk_name: null
    proximity_placement_group: null
    capacity_reservation_group: null
    open_ports: null
    network_interface_names: null
    virtual_network_resource_group: null
    virtual_network_name: null
    subnet_name: null
    swap_os_disk: null
    data_disks: null
    plan: null
    zones: null
    license_type: null
    vm_identity: null
    winrm: null
    boot_diagnostics: null
    ephemeral_os_disk: null
    windows_config: null
    linux_config: null
    security_profile: null
    additional_capabilities: null
_ansible_no_log: false
changed: false
_ansible_delegated_vars:
  ansible_host: localhost
  ansible_port: null
  ansible_user: root
  ansible_connection: local

[Fred-sun]

@vijayreddiar Welcome you submit your problems, because we upgraded the half cup v3.0.0 and also upgraded azure-mgmt-resource to version v23.1.0(refer to PR #1712), can you check the version of azure-mgmt-resource? Thank you!

[vijayreddiar]

@Fred-sun We are using AWX running on k3s and the project containing collections/requirements.yml will have below line to install latest version azure.azcollection whenever the project is synced. Hence the expectation is that all the dependencies will be updated automatically with the required version.

collections:
  - name: azure.azcollection

For the specific version of azure-mgmt-resource (which is not controlled with explicit commands), could you please let me know how we can accurately verify whether we have the expected version available.

We follow the installation procedure from this repo which results into 4 different pods - awx-operator-controller-manager, awx-postgres, awx-web & awx-task. We have logged into awx-ee container of awx-task pod and upgraded all dependencies including azure-mgmt-resource from requirements.txt in the collection, however the issue persists.

bash-5.1$ pip install -r /runner/.ansible/collections/ansible_collections/azure/azcollection/requirements.txt
....
....
....
Successfully installed azure-cli-core-2.64.0 azure-common-1.1.28 azure-containerregistry-1.2.0 azure-core-1.31.0 azure-identity-1.19.0 azure-mgmt-apimanagement-4.0.1 azure-mgmt-authorization-4.0.0 azure-mgmt-automation-1.1.0b3 azure-mgmt-batch-17.3.0 azure-mgmt-compute-33.0.0 azure-mgmt-containerinstance-10.1.0 azure-mgmt-containerregistry-10.3.0 azure-mgmt-containerservice-32.1.0 azure-mgmt-cosmosdb-10.0.0b3 azure-mgmt-datafactory-9.0.0 azure-mgmt-devtestlabs-10.0.0b1 azure-mgmt-dns-8.1.0 azure-mgmt-eventhub-11.1.0 azure-mgmt-keyvault-10.3.1 azure-mgmt-loganalytics-13.0.0b6 azure-mgmt-managedservices-7.0.0b1 azure-mgmt-managementgroups-1.1.0b1 azure-mgmt-marketplaceordering-1.2.0b1 azure-mgmt-monitor-6.0.2 azure-mgmt-network-26.0.0 azure-mgmt-notificationhubs-8.1.0b1 azure-mgmt-nspkg-3.0.2 azure-mgmt-privatedns-1.1.0 azure-mgmt-rdbms-10.2.0b17 azure-mgmt-recoveryservices-3.0.0 azure-mgmt-recoveryservicesbackup-9.1.0 azure-mgmt-redis-14.4.0 azure-mgmt-resource-23.1.0 azure-mgmt-search-9.2.0b2 azure-mgmt-servicebus-8.2.0 azure-mgmt-sql-4.0.0b19 azure-mgmt-storage-21.2.1 azure-mgmt-trafficmanager-1.1.0 azure-mgmt-web-7.3.1 azure-nspkg-3.0.2 azure-storage-blob-12.23.0b1 microsoft-kiota-abstractions-1.6.2 microsoft-kiota-serialization-form-1.6.2 microsoft-kiota-serialization-json-1.6.2 microsoft-kiota-serialization-multipart-1.6.2 msal-1.30.0 msal-extensions-1.2.0 msgraph-sdk-1.6.0 opentelemetry-api-1.28.2 opentelemetry-sdk-1.28.2 opentelemetry-semantic-conventions-0.49b2 std-uritemplate-2.0.1
bash-5.1$
bash-5.1$ pip list | grep azure-mgmt-resource
azure-mgmt-resource                     23.1.0
bash-5.1$

So, I doubt whether it uses the above package and runner depends something else or the issue might be existing even with azure-mgmt-resource v23.1.0

[Fred-sun]

@vijayreddiar I confirmed v23.1.0 this time(Relate link

azure/requirements.txt

Line 28 in 051dc18

azure-mgmt-resource==23.1.0

) , the 'resource_groups' function exists, thank you!

[vijayreddiar]

@Fred-sun We are using AWX running on k3s and the project containing collections/requirements.yml will have below line to install latest version azure.azcollection whenever the project is synced. Hence the expectation is that all the dependencies will be updated automatically with the required version.

collections:
  - name: azure.azcollection

For the specific version of azure-mgmt-resource (which is not controlled with explicit commands), could you please let me know how we can accurately verify whether we have the expected version available.

We follow the installation procedure from this repo which results into 4 different pods - awx-operator-controller-manager, awx-postgres, awx-web & awx-task. We have logged into awx-ee container of awx-task pod and upgraded all dependencies including azure-mgmt-resource from requirements.txt in the collection, however the issue persists.

bash-5.1$ pip install -r /runner/.ansible/collections/ansible_collections/azure/azcollection/requirements.txt
....
....
....
Successfully installed azure-cli-core-2.64.0 azure-common-1.1.28 azure-containerregistry-1.2.0 azure-core-1.31.0 azure-identity-1.19.0 azure-mgmt-apimanagement-4.0.1 azure-mgmt-authorization-4.0.0 azure-mgmt-automation-1.1.0b3 azure-mgmt-batch-17.3.0 azure-mgmt-compute-33.0.0 azure-mgmt-containerinstance-10.1.0 azure-mgmt-containerregistry-10.3.0 azure-mgmt-containerservice-32.1.0 azure-mgmt-cosmosdb-10.0.0b3 azure-mgmt-datafactory-9.0.0 azure-mgmt-devtestlabs-10.0.0b1 azure-mgmt-dns-8.1.0 azure-mgmt-eventhub-11.1.0 azure-mgmt-keyvault-10.3.1 azure-mgmt-loganalytics-13.0.0b6 azure-mgmt-managedservices-7.0.0b1 azure-mgmt-managementgroups-1.1.0b1 azure-mgmt-marketplaceordering-1.2.0b1 azure-mgmt-monitor-6.0.2 azure-mgmt-network-26.0.0 azure-mgmt-notificationhubs-8.1.0b1 azure-mgmt-nspkg-3.0.2 azure-mgmt-privatedns-1.1.0 azure-mgmt-rdbms-10.2.0b17 azure-mgmt-recoveryservices-3.0.0 azure-mgmt-recoveryservicesbackup-9.1.0 azure-mgmt-redis-14.4.0 azure-mgmt-resource-23.1.0 azure-mgmt-search-9.2.0b2 azure-mgmt-servicebus-8.2.0 azure-mgmt-sql-4.0.0b19 azure-mgmt-storage-21.2.1 azure-mgmt-trafficmanager-1.1.0 azure-mgmt-web-7.3.1 azure-nspkg-3.0.2 azure-storage-blob-12.23.0b1 microsoft-kiota-abstractions-1.6.2 microsoft-kiota-serialization-form-1.6.2 microsoft-kiota-serialization-json-1.6.2 microsoft-kiota-serialization-multipart-1.6.2 msal-1.30.0 msal-extensions-1.2.0 msgraph-sdk-1.6.0 opentelemetry-api-1.28.2 opentelemetry-sdk-1.28.2 opentelemetry-semantic-conventions-0.49b2 std-uritemplate-2.0.1
bash-5.1$
bash-5.1$ pip list | grep azure-mgmt-resource
azure-mgmt-resource                     23.1.0
bash-5.1$

So, I doubt whether it uses the above package and runner depends something else or the issue might be existing even with azure-mgmt-resource v23.1.0

@Fred-sun The output shared in my previous comment also confirms azure-mgmt-resource v23.1.0 exists in Execution Environment, however the issue persists. Could you please check the same behavior using latest version of AWX on K3S from here

[Fred-sun]

@vijayreddiar I'm very sorry to reply you so late. But I checked that the 'resource_groups' of API 'v2022-09-01 exists, is it because your installation environment and execution environment are inconsistent? Could you confirm that? Thank you very much!