Qubes and Hybrid

This guide is for two different concerns which intersect: Developing Sandstorm apps on QubesOS, and creating SPK (raw) apps that can also be built with Vagrant SPK. I can't guarantee this will be everything you need to do. You may need to play around with it. But it'll take care of a lot of what will go wrong.

Given popular demand they can be separated. Dan the author does both at once, so in the interest of time he made one guide. Please be careful if you're doing raw development on something other than a dedicated Qubes VM, since it alters your system. I wrote this guide with a dedicated Qubes VM in mind.

Qubes

One nice benefit of Qubes is that you can keep everything separate. As such you may consider creating a new VM for each Sandstorm project. Though, perhaps this isn't necessary if you use a normal AppVM which wipes the system (everything but home directory) for each restart anyway. I guess this is up to you.

If you do use an AppVM, you'll need to install Sandstorm every time. You'll also need to set up your Qubes user (usually literally called user) to be able to work with it:

curl https://install.sandstorm.io | bash

sudo usermod -a -G sandstorm user

Now, for whatever reason your Qubes VM won't have the user's group associations with Sandstorm on a normal bash session (if someone can fix this please put it in here! and maybe in a more current version of Qubes they fix it?). So you'll get an error if you try to run spk dev. To fix this, do:

sudo su user
spk dev

🤷 At this point you'll start a new shell with your same user, but you'll have the sandstorm group. (You may need this for other commands as well, I forget.)

Hybrid

You might want to start by bootstrapping a Vagrant-SPK project. Maybe even on another machine, or maybe copy another project or something. (I haven't thought about how to do it on Qubes as such).

Scripts

Vagrant SPK has a handful of scripts and config files under .sandstorm/ directory.

Vagrantfile specifies the OS and other Vagrant stuff. As the comments in the file says, you shouldn't edit it. However for your SPK setup, you should probably make sure to have the same version of Debian on the system you're developing on (such as your Qubes VM) as the one mentioned in this file.

build.sh and sudo setup.sh are run automatically by Vagrant SPK during the build process. When developing with SPK, assuming you set everything up correctly (particularly paths, mentioned below), you can just run them manually. Put all your setup and dependencies etc into these files, so that it works for Vagrant SPK developers as well. Qubes users: just keep in mind that if these scripts install anything on the system, you'll have to run this every time you start up your VM.

launch.sh, sandstorm-pkgdef.capnp, and sandstorm-files.list also work straightforwardly, again so long as you have the paths set up properly below.

Paths

Here's the real secret. It tripped me up a lot trying to go back and forth between vagrant-spk and spk, getting different stuff in sandstorm-files.list for either one. You want your app set up at /opt/app. However, this isn't a very convenient place to do your development. If you're on a Qubes AppVM, this directory will get wiped out every restart. You could probably do this with a Qubes Standalone VM, but maybe you don't want to have a full system just for this.

So I recommend putting your project in your home directory as usual. Can you symlink /opt/app to your repo? Turns out Sandstorm doesn't like that. What you can do instead is something called a bind mount.

sudo mkdir /opt/app
sudo mount --bind /home/user/my-cool-app /opt/app

Again, run this every time you start up.

User

I think that the user account name when building (not running) under Vagrant is vagrant. This can be used to your advantage for the rare case where you need to do something special in one case and not the other. Best of course to minimize such things.

Just make sure your SPK environment doesn't happen to also use a username vagrant. (Again in Qubes it'll be user so you're set).

A startup script

Putting the Qubes and Hybrid concerns all together you might want a startup script that looks something like this:

#!/bin/bash

set -exuo pipefail

# don't install twice
id -u sandstorm || curl https://install.sandstorm.io | bash

sudo usermod -a -G sandstorm user

# This part requires that you checked out the repo first
ls /opt/app/ || sudo mkdir /opt/app
ls /opt/app/.sandstorm || sudo mount --bind /home/user/my-cool-app /opt/app

cd /opt/app/.sandstorm/
sudo setup.sh

Note that I tried to make it idempotent so that it could be re-run after any changes to setup.sh. (You might add build if that has sudo stuff in there as well).