feat!: (PSKD-348) Dependency version update

[riragh]

Changes:

This PR updates all the Terraform modules, providers, and other dependencies versions to latest. Also cleans up any old, unused code.

Notes:

• The impact on variable change was kept minimal but please make sure to use the updated example files.
• As this a major version update with breaking changes direct upgrade is not recommended, please remove your old infrastructure and then use the newer version.

Version Update summary:

• terraform 1.8.2 -> 1.9.6
• azure cli 2.59.0 -> 2.64.0 — Breaking change (Alpine base image support dropped)
  • Starting from the version 2.64.0 of Azure CLI, the base Linux distribution of Azure CLI is now Azure Linux. See Release notes here: https://learn.microsoft.com/en-us/cli/azure/release-notes-azure-cli#packaging-2

Provider version updates

• azurerm 3.92.0 -> 4.3.0  — Major version update with breaking changes
  • The major version upgrade documentation can be found here: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/4.0-upgrade-guide
• azuread 2.47 -> 2.53
• local 2.4 -> 2.5
• kubernetes 2.25 -> 2.32

Tests:

Verifying following scenarios:

Scenario	Task	Provider	Cadence	Deploy Method	Notes
1	OOTB, default, monitoring log enabled	Azure	fast:2020	Terraform	See additional verification note below
2	OOTB, host encryption enabled	Azure	stable:2024.06	Terraform
3	OOTB, Microsoft Entra authentication with Kubernetes RBAC	Azure	lts:2023.03	Terraform
4	OOTB, Storage_type: HA	Azure	fast:2020	Terraform
5	OOTB, external PG	Azure	-	Docker

Additional Verification:

• Verified public ip sku is set to standard
• Verified autoscaling is enabled
• Verified docker_bridge_cidr is removed
• Verified api_server_access_profile range is set as expected
• Verified node taints were removed
• Verified VM node acceleration is enabled

[joboon]

There's a lot here, and I am unfamiliar with most of it. However, the changes look consistent and match with the discoveries detailed in PSKD-348, so I'll toss my approval. I think David and/or Chris should also review before merging.

[dhoucgitter]

LGTM