On disk boot loader for booting from local pool on PXE failure

[nshalman]

From one of the conversations yesterday. Adding to backlog for posterity.

Issue Description

In theory if we create our pool correctly and install a boot loader like GRUB2 or the FreeBSD loader, either of which should be able to read from arbitrary ZFS pools we could store the last booted kernel and ramdisk images in the pool (or whatever is the latest upgrade shipped to the cluster) so that if PXE fails we can fall back to booting from disk. This would allow for completely unattended recovery of a cluster with none of the machines having an original boot CD or USB stick installed.

Acceptance Criteria (How do we know when this is done?)

Shut down a MistifyOS machine. Disconnect network cable. Boot.
MistifyOS should be loaded and booted from disk.
In particular this is complete when that can be done from a pool containing RAIDZ vdevs.

Environment (physical, VM, cloud provider, etc.):

Definitely physical. Might smooth booting in a cloud provider?

Other Relevant Info:

This is connected to distributing updates throughout the cluster for making sure that when machines reboot they get the latest platform image.