Instructions on how to use this to make a Hetzner Cloud snapshot (includes moving /old-root/nixos to /)

[nh2]

This describes how you can use nixos-in-place to create a NixOS Snapshot on Hetzner Cloud, from which you can then boot more NixOS machines.

I am currently working on implementing NixOps support for snapshots created that way.

Maybe we want to put it into a file or wiki or something @jeaye, let me know what you think.

Step 1: Running nixos-in-place on an Ubuntu 16.04 Hetzner Cloud machine

You can copy-paste this all in one go and paste it into the Ubuntu root shell.

apt-get install -y squashfs-tools git
git clone https://github.com/jeaye/nixos-in-place.git
cd nixos-in-place
git checkout ac8fdff901f0ff03875f9839a74b9d9c20cd58d5 # just for reproducibility, you can skip this to use the latest version
./install
y
reboot

Step 2: Moving /old-root/nixos to /

To turn a Hetzner Cloud machine converted with nixos-in-place into a normal NixOS installation, moving the /nixos directory to be the new root:

You need to replace the users.extraUsers.root.openssh.authorizedKeys.keys with your SSH pubkey in the below so you can SSH into the machine.

Put the machine into Hetzner rescue mode and run on it (you can copy-paste this all in one go after replacing the pubkey string):

mount /dev/sda1 /mnt/
cd /mnt

shopt -s extglob
rm -r --one-file-system !(nixos)
rmdir nixos/old-root/
mv nixos/* .
rmdir nixos

mount -t proc proc proc/
mount -t sysfs sys sys/
mount -o bind /dev dev/
chroot . /nix/store/ddqnyrjdm9la6dwvdx50w65vmq712lbm-bash-4.3-p46/bin/bash
export PATH=/nix/store/cvvl412nrzp47mp3f8lim8vmpiskfn8r-system-path/bin:$PATH

cat > /etc/nixos/nixos-in-place.nix <<EOF
{ config, pkgs, ... }:
{
  boot.kernelParams = ["boot.shell_on_fail"];
  boot.loader.grub.device = "/dev/sda";
  boot.loader.grub.storePath = "/nix/store";
  boot.initrd.supportedFilesystems = [ "ext4" ];
  fileSystems = {
    "/" = {
      device = "/dev/sda1";
      fsType = "ext4";
    };
  };
  users.extraUsers.root.password = "nixos";
  services.openssh.enable = true;
  users.extraUsers.root.openssh.authorizedKeys.keys = [
    "ssh-rsa  ......................................................................."
  ];
}
EOF

nixos-rebuild switch

grub-install /dev/sda

exit  # exit chroot

apt-get install -y zerofree
zerofree -v /dev/sda1

reboot  # reboot out of rescue mode; or `poweroff` instead if you want to make a snapshot

Now you can SSH into the machine with the key you've put in above.

You can also use Hetzner Cloud's web terminal, where the username is root and the password is nixos (this password obviously works only via the web terminal and not via SSH).

[nh2]

Also note this is very similar to #37; though I only found that issue after I had already written the above.

Probably something in there can be used to remove the two hardcodes I have made above:

chroot . /nix/store/ddqnyrjdm9la6dwvdx50w65vmq712lbm-bash-4.3-p46/bin/bash
export PATH=/nix/store/cvvl412nrzp47mp3f8lim8vmpiskfn8r-system-path/bin:$PATH

Would be great if somebody could figure that out!

[jeaye]

This is excellent info, @nh2! We're not currently using a wiki or any other doc collection, so keeping it as an issue and referencing it from the README, which I have just done, should be fine. Thanks so much for the detailed breakdown.

As for the hard-coding, you can probably just glob it up. /nix/store/*-bash-* should get you bash. If it returns multiple, just | head -1 to take one of them. What do you think?

[alexanderkjeldaas]

Why does the machine have to be placed into rescue mode?

Also, would it be possible to attach the existing NixOS 18.03 ISO using the Hetzner API? https://docs.hetzner.cloud/#resources-server-actions-post-13

If the ISO is attached like that, it should be immediately available, and maybe installation can be faster so we don't need to pay for a snapshot - which is quite expensive on hetzner (relative to a server at least).

[hlolli]

Worked for me, note to future internet users. I would not leave out the checkout sum, remember to reboot from webconsole after turning on rescue mode.

[frankbo]

When I did nixos-rebuild switch I got an error that the folder /run/user/0. After creating it with mkdir -p /run/user/0 the installation process worked fine.

[hlolli]

Note to my future self (or others who bump into this):

If getting boot problems from @nh2 's method

• enable rescue mode
  mount /dev/sda1 /mnt/

vim /mnt/boot/grub/grub.cfg

change:
linux ($drive2)/nixos/nix/store/x58d7k8lczvh4qsqaj4jky1hzpc788b4-linux-4.4.23/bzImage
to:
linux ($drive2)/nix/store/x58d7k8lczvh4qsqaj4jky1hzpc788b4-linux-4.4.23/bzImage

and change:
initrd ($drive2)/nixos/nix/store/3ca7cs5gm595872icxa9kidzpy78qmi3-initrd/initrd
to:
initrd ($drive2)/nix/store/3ca7cs5gm595872icxa9kidzpy78qmi3-initrd/initrd

Or if nixos-rebuild switch fails do mkdir -p /run/user/0 or depending on the user id of the rescue mode (do it before nixos-rebuild switch, given that chroot is /mnt).

[srid]

@nh2 Can nixos-in-place be used to install on top of Ubuntu install on hetzner dedicated server? (AX101 in particular)?

[nh2]

@srid There should most likely be the possibility, however for Hetzner and many other hosters, doing a clean (not -in-place hooked on another OS) installation got much easier, and I open-sourced scripts for that at https://github.com/nix-community/nixos-install-scripts.

I think nixos-in-place is best used in places where normal install methods aren't an option.