Understanding the usage of SSH in the module

[felipe-loka]

Hey!

I'm trying to create a VM using Ubuntu ISO but I'm facing issues to attach the disk to node. This is the error I'm facing:

│ Error: creating custom disk: unable to authenticate user "root" over SSH to "10.10.10.1:22". Please verify that ssh-agent is correctly loaded with an authorized key via 'ssh-add -L' (NOTE: configurations in ~/.ssh/config are not considered by the provider): failed to dial 10.10.10.1:22: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain

I know this is related to SSH/networks but I don't understand exactly why I'm facing this issue.

I tried the following approaches:

• Ran terraform from ProxMox machine
• Ran terraform from a machine in the same network as the ProxMox

None of these approaches work. How this needs to be set up in order to attach this hard drive correctly? I can provide the values/manifests I'm using if needed.

[bpg]

Hey @felipe-loka 👋🏼

You may wan't to take a look at SSH Connection section in the provider docs.

In general the provider is using Proxmox VE REST API for most of its functionality. However, certain features are not exposed via APIs, and only available as cmd-line tools on the PVE host itself. So SSH is used as a workaround, to circumvent these limitations and performs the required actions. Like attaching an existing disk image to a VM.

However, I'm not sure attaching an ISO image as a disk to a VM will work. It has to be connected as a cdrom drive, and VM should be booted from it to start an installed.
You probably need a specifically build cloud image instead. There are some examples here: https://registry.terraform.io/providers/bpg/proxmox/latest/docs/guides/cloud-image

[14traits]

@bpg I am having a similar issue; I am working on a Talos deployment to a three-node proxmox cluster following this guide: https://olav.ninja/talos-cluster-on-proxmox-with-terraform

For simplicity's sake, currently, I am only deploying VMs to my second node; however, my Proxmox API is the primary node. I want to deploy across all three nodes, and this SSH error seems to be node-dependent. Am I missing something, as the documentation for the provider on SSH keys doesn't seem to cover this use case?

Also, I can start a new thread if this should be outside of this one.

[bpg]

Hi @14traits 👋🏼

Depending on your nodes' network setup, the provider may not be able to auto-detect your node IP addresses, or using a wrong IP. In that case, you'll need to define a map of node names to node IPs as described here.

Would you mind sharing more details about your cluster setup, nodes' network & SSH settings, and the provider's configuration? That way, I can give you more specific advice.

[windowsrefund]

The SSH side of this provider continues to be a complete nightmare to work with. Above, this documentation was referenced and yet, it doesn't even work.

Plan: 3 to add, 0 to change, 0 to destroy.
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Creating...
proxmox_virtual_environment_vm.this: Creating...
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [10s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [20s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Still creating... [30s elapsed]
proxmox_virtual_environment_download_file.ubuntu_cloud_image: Creation complete after 30s [id=local:iso/jammy-server-cloudimg-amd64.img]
proxmox_virtual_environment_vm.ubuntu_vm: Creating...
proxmox_virtual_environment_vm.ubuntu_vm: Still creating... [10s elapsed]
proxmox_virtual_environment_vm.ubuntu_vm: Creation complete after 11s [id=100]

Error: error waiting for VM creation: task "UPID:pve:000159D2:0038E96A:66CFE314:qmcreate:9000:terraform-prov@pve!terraform:" failed to complete with exit code: unable to create VM 9000 - no such logical volume pve/base-9000-disk-0

So it looks like the provider's SSH connection isn't even being used now despite the properly configured provider block

provider "proxmox" {
  endpoint = "https://pve:8006"
  insecure = true
  ssh {
    node {
      name    = "pve"
      address = "192.168.1.200"
    }
    username = "root"
    agent    = true
  }
}

Update: The linked documentation does work with no changes made to the mentioned provider block. I think I may have overlooked disk.file_id.

[svengreb]

Without SSH usage you would still be doing ClickOps so nightmare seems like a harsh description for a great future that allows to automate almost everything in PVE.

Have you read the documentation for setting up SSH for multiple nodes? Looking at your example the endpoint = "https://pve:8006" makes me suspicious because pve must be resolvable through DNS and I guess this is not the case. If you enabled the agent because pve is a hostname in your ~/.ssh/config file you should read the documentation about the SSH agent because it…

[…] does not use OS-specific SSH configuration files, such as ~/.ssh/config.

which is also shown in the error message the author of this discussion posted.

In general I also recommend to do a "split provider" setup like this:

provider "proxmox" {
  endpoint = "https://192.168.1.200:8006"
  insecure = true
  #…
}

provider "proxmox" {
  alias = "ssh
  endpoint = "https://192.168.1.200:8006"
  insecure = true
  ssh {
    agent = true
    node {
      name    = "pve"
      address = "192.168.1.200"
    }
  }
  #…
}

Now the dedicated SSH provider can be used for resources and modules:

resource "proxmox_virtual_environment_vm" "ubuntu_vm" {
  provider = proxmox.ssh
  name = "ubuntu"
  #…
}

# For modules multiple providers can be added which are passed down to the underlying source module.
module "my-custom-vms" {
  source = "../../shared/modules/my/custom/vms"
  providers = [
    proxmox.ssh
  ]

  name = "ubuntu"
  #…

A great side-effect is that this makes the whole problem of "knowing when SSH is required and when not" go away because now it can be easily determined by the aliased provider in the resource/module.

[windowsrefund]

Sorry for the noise. Must have been a problem on my end. While I'm here, I do want to say I'm not a fan of hard-coding an aliased provider in the way mentioned above as all that does is bake an assumption into things better left to be determined programmatically.

[bpg]

@svengreb great idea with provider aliases, we should probably document it as the recommended approach.