Talos PXE question (RPi4)

[BryceAshey]

I'm trying to work through a method for running Talos via netboot on normal Raspberry PI4s. I've been working through Sidero but it has a lot complexities due to the way RPI's boot. In the end it would be easier to just flash the cards with Talos to begin with. I'm also not super keen on running a separate cluster just for Sidero.

I'm happy to and capable of configuring a separate PXE server but ran into some challenges mapping the (6?) partitions in the talos image. Does anyone have any pointers there? How does Sidero do that? I crawled through their GitHub but I'll be danged if I can find the code. I'm happy to RTFM but have sort of hit a wall after several hours of Googling and experimenting.

Based on this question I'm also curious if netboot would still require an SD in the PI?

Any pointers are greatly appreciated.

[frezbo]

Yes, talos would need an SD card or a storage disk in order for kubernetes to store data (Talos operating system itself runs in memory, but the kubernetes need storage). If you want to just set an external PXE server you can use the initrd and kernel images for ARM64 from the releases page

[BryceAshey]

Thanks - that got me a lot further. I'm getting right up to the reboot and then hanging on the rainbow screen. I'm using the UEFI boot (1.32) similar to described on Sidero. I then copied over initramfs-arm64 and vmlinuz-arm64. I decompressed initramfs but left the kernel compressed.

Here's the dir:

pi@raspberrypi:/srv/tftpboot/e8673058 $ ls -al
total 111176
drwxrwxrwx 4 root root     4096 Feb 21 00:33 .
drwxrwxrwx 6 root root     4096 Feb 20 19:10 ..
-rw-r--r-- 1 pi   pi      49829 Feb 20 19:12 bcm2711-rpi-4-b.dtb
-rw-r--r-- 1 pi   pi        233 Feb 21 00:13 config.txt
drwxr-xr-x 3 pi   pi       4096 Feb 20 19:12 firmware
-rw-r--r-- 1 pi   pi       5351 Feb 20 19:12 fixup4.dat
-rwxrwxrwx 1 pi   pi   53359104 Feb 11 20:10 initramfs-arm64
drwxr-xr-x 2 pi   pi       4096 Feb 20 22:50 overlays
-rw-r--r-- 1 pi   pi    2031616 Feb 20 19:51 RPI_EFI.fd
-rw-r--r-- 1 pi   pi    2243232 Feb 20 19:12 start4.elf
-rwxrwxrwx 1 pi   pi   56201728 Feb 11 20:11 vmlinuz-arm64

Here is the config.txt:

pi@raspberrypi:/srv/tftpboot/e8673058 $ cat config.txt
arm_64bit=1
arm_boost=1
enable_uart=1
uart_2ndstage=1
dtoverlay=disable-bt
enable_gic=1
armstub=RPI_EFI.fd
disable_commandline_tags=1
disable_overscan=1
kernel=vmlinuz-arm64
ramfsfile=initramfs-arm64
hdmi_force_hotplug=1

I do see the files going across in /var/log/daemon.log on the tftp node - using dnsmasq. I do see some errors there which I'm troubleshooting but the files do eventually send:

Feb 21 00:38:27 raspberrypi dnsmasq-tftp[1243]: sent /srv/tftpboot/e8673058/RPI_EFI.fd to 192.168.192.19
Feb 21 00:38:28 raspberrypi dnsmasq-tftp[1243]: error 0 Early terminate received from 192.168.192.19
Feb 21 00:38:28 raspberrypi dnsmasq-tftp[1243]: failed sending /srv/tftpboot/e8673058/vmlinuz-arm64 to 192.168.192.19
Feb 21 00:38:40 raspberrypi dnsmasq-tftp[1243]: sent /srv/tftpboot/e8673058/vmlinuz-arm64 to 192.168.192.19

Any ideas why it might not be booting into the kernel? This is my first time using ramfs (instead of nfs) so I'm sure I'm screwing something up. :)

[frezbo]

the kernel needs to be uncompressed for network boot if I remember that correctly for ARM64 (i believe the released arm64 kernel is uncompressed anyways). There's no need to decompress the initramfs (well at least when using ipxe).

[BryceAshey]

@frezbo thank you for the kind pointers. I was able to get it up and running! Really looking forward to digging into this - thanks again.