add support for VZNetworkBlockDeviceStorageDeviceAttachmentDelegate

[lstocchi]

this adds support for the VZNetworkBlockDeviceStorageDeviceAttachmentDelegate to being able to listen to changes to a network block device attachment.

Which issue(s) this PR fixes:

Fixes #166

Additional documentation

I was supposed to add tests but then I noticed that you only run them on macOS 13 because macOS 14, 15 does not support build on nested virtualization. So I'm gonna mark the PR as ready to review. NBD support is only available from macOS 14+ . Please let me know if I should do something else.

My test use case below:
In the short vide I have a remote nbd server.
Then I create a vm connecting to the nbd server and works fine. The connection is made and the delegate receive the wasConnected message.
At this point, I stop the nbd server and I see the nbd client start receiving errors from the delegate as the connection dropped.
Then I start the nbd server again and my client gets reconnected.

test_nbd.mov

[Code-Hex]

@lstocchi Thanks for sending PR!
I will check in today after I finished today's my work 🙏

Btw, Could you add unit test for this? Or even at hand, I'd appreciate it if you could tell me how to check.

[lstocchi]

@lstocchi Thanks for sending PR!
I will check in today after I finished today's my work 🙏

Thanks @Code-Hex

Btw, Could you add unit test for this? Or even at hand, I'd appreciate it if you could tell me how to check.

Updating the macOS example 👍

[lstocchi]

@Code-Hex I tried to run the macOS example (from the main branch) using the nbd-url but I'm not able to make it work. Not sure what I'm missing.
I run make all -> virtualization -install -> virtualization -nbd-url nbd+unix:///export?socket=nbd.sock but I keep getting Invalid virtual machine configuration. The storage device attachment is invalid bc the attachment is nil 🤔 I followed the indication from #156 to set up the nbd-server and run the macOS sample

I tried writing the code to cover the delegate part 07aef59 and I also pushed it so you can give it a look but I was not able to try it for the reason above. Any hints?

BTW, my manual way to test it is using vfkit.
I create a VM that is responsible to run the NBD server.
I use fedora cloud 40 as OS.
The seed.img is the cloud-init configuration to enable a new user

./vfkit --cpus 2 --memory 2048 --bootloader efi,variable-store=efi-variable-store,create --device virtio-blk,path=fedora-40.raw --device virtio-blk,path=seed.img  --device virtio-net,nat,mac=<Mac_addr>

then I ssh in it and run
qemu-nbd -f raw -x export --cache=none -p 11111 disk.img

Now I create a new vfkit binary using this branch https://github.com/crc-org/vfkit/pull/212 and I launch another VM to act as the nbd-client

./vfkit --cpus 2 --memory 2048 --bootloader efi,variable-store=efi-variable-store,create --device virtio-blk,path=fedora-40.raw --device virtio-blk,path=seed.img --device nbd,uri=nbd://<ip_first_vm>:11111/export  --device virtio-net,nat,mac=<Mac_addr>

<Mac_addr> and <ip_first_vm> taken by looking at /var/db/dhcpd_leases

I can provide the seed.img and also provide a detailed step-by-step procedure if you want to do exactly what I do

[Code-Hex]

@lstocchi Thanks for your explanation and given an example to work them!

hmm, I didn't try it on macOS but I found the ndb server works on macOS too

https://gitlab.com/nbdkit/nbdkit/-/blob/master/README.md#macos

So I think you can check with install the ndbkit and run it on your macOS guest. I will confirm it too.

I tried writing the code to cover the delegate part 07aef59 and I also pushed it so you can give it a look but I was not able to try it for the reason above. Any hints?

[Code-Hex]

I got panic 😇

./virtualization
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x10293e024]

goroutine 1 [running]:
github.com/Code-Hex/vz/v3.(*baseStorageDeviceConfiguration).Attachment(...)
        /Users/codehex/Desktop/vz/storage.go:165
main.listenNetworkBlockDevice(0x14000110050?)
        /Users/codehex/Desktop/vz/example/macos/main.go:341 +0x54
main.runVM({0x14000100990?, 0x140000021c0?})
        /Users/codehex/Desktop/vz/example/macos/main.go:82 +0x110
main.run({0x102a4bf68?, 0x102c57b60?})
        /Users/codehex/Desktop/vz/example/macos/main.go:36 +0x40
main.main()
        /Users/codehex/Desktop/vz/example/macos/main.go:26 +0x74

[Code-Hex]

I'm sorry, this is going to be a bit of a long review, let me check around channel a bit more too.

[lstocchi]

np, I'm investigating too. Trying to make it work from the main branch first.

[Code-Hex]

I made some code modifications and was able to verify the connection to the NBD server from the macOS guest using the following steps.

macOS host

$ hdiutil create -size 1g -fs HFS+ -volname "SharedDisk" shared.dmg
$ scp -O shared.dmg [email protected]:/home/ubuntu

multipass ubuntu guest

ubuntu@UBUNTU:~$ cat /etc/nbd-server/config
[generic]
# If you want to run everything as root rather than the nbd user, you
# may either say "root" in the two following lines, or remove them
# altogether. Do not remove the [generic] section, however.
        user = nbd
        group = nbd
        includedir = /etc/nbd-server/conf.d

# What follows are export definitions. You may create as much of them as
# you want, but the section header has to be unique.
[disk0]
        exportname = /home/ubuntu/shared.dmg
ubuntu@UBUNTU:~$ sudo systemctl restart nbd-server

example code

./virtualization -nbd-url=nbd://192.168.105.2:10809/disk0
2024/11/02 00:23:07 start VM is running
2024/11/02 00:23:07 Successfully connected to NBD server

I will review this tomorrow as it is already time to sleep Japan time.

[Code-Hex]

@lstocchi Hi 👋

I created a PR to propose some changes to your PR. This update is designed to ensure that the functionality can be resumed on the VM side when the NBD server recovers, rather than handling it as a one-time process.

Additionally, in Go, it’s common to leave the creation of goroutines to the user. This is because the implementations of methods and functions provided by libraries are generally hidden from the user’s view, so delegating this responsibility clears up these issues as well.

#168

If there are no issues with the following content, I will merge PR #168 and close this PR. Of course, all of your previous commits will also be included.

[lstocchi]

closing this in favor of #168