PostUp and PreDown commands

[dani-corie]

It would be nice to be able to run scripts when a new namespace got pulled up, and before it's shut down. One thing I'd use this for is to spin up a dnscrypt-proxy instance handling the vpn connection.

[jamesmcm]

Sure, what would the aim be? To provide an executable script/binary as arguments for each when using vopono exec?

I don't think it makes sense to parse these from the wg-quick config files, because usually we have already done whatever they would do anyway (i.e. DNS or killswitch configuration).

[dani-corie]

It would be for spinning up and shutting down local services that serve the namespace. I for example like to use dnscrypt-proxy, because I kinda enjoy having the added peace of mind of routing through independent DNS servers over encrypted channels, with DNSSEC enabled.

The same logic would also serve shadowsocks, privoxy, tor node routed through vpn, etc.. Note that these aren't services you want to run through the vpn - they are services serving your local vpn session. Your own Firefox instance in the namespace will use privoxy and dnscrypt-proxy. They have to be there when the user application starts, and should be shut down along with the namespace when no longer needed by the user application.

Also note the fact that whether I use Swedish or Tokelauan vpn servers have zero bearing on these local services. They are part of my own system configuration, not of the vpn connection configuration. So even if we ran scripts from wg-quick configs, that wouldn't solve it. There's a TON of those files - for Mullvad alone, we're talking over a hundred files, often randomly selected when you filter for an exit country for example.

So yes, primarily I'd say these should be specifiable as parameters to vopnono... But ultimately, I feel like vopono is missing a crucial component - a user level config file for vopono in general. If I only use mullvad, there's no good reason to have to specify --provider mullvad every single time...

So it's not really part of this enhancement proposal, but there should be a ~/.vopono.conf, where I can specify postup and predown scripts, defaults for --provider and --server, and similar stuff.

[jamesmcm]

That makes sense, could you please provide examples of both scripts, just to have something to test against?

[dani-corie]

postUp.sh:

#!/bin/bash

WORK_DIR="${HOME}/.vopono"
mkdir -p ${WORK_DIR}

SANDBOX_NAME=${1}
SVC_NAME="dnscrypt-proxy"

PROC_NAME="${SANDBOX_NAME}-${SVC_NAME}"

FILE_ROOT=${WORK_DIR}/${PROC_NAME}
LOG_FILE=${FILE_ROOT}.log
PID_FILE=${FILE_ROOT}.pid
LOCK_FILE=${FILE_ROOT}.lock

CFG_FILE="${WORK_DIR}/dnscrypt-proxy.toml"
SVC_COMMAND="/usr/sbin/dnscrypt-proxy -config ${CFG_FILE}"

sudo daemonize -a -c ${WORK_DIR} -e ${LOG_FILE} -p ${PID_FILE} -l ${LOCK_FILE} ${SVC_COMMAND}

preDown.sh:

#!/bin/bash

WORK_DIR="${HOME}/.vopono"

SANDBOX_NAME=${1}
SVC_NAME="dnscrypt-proxy"

PROC_NAME="${SANDBOX_NAME}-${SVC_NAME}"
FILE_ROOT=${WORK_DIR}/${PROC_NAME}
PID_FILE=${FILE_ROOT}.pid

sudo kill -s 15 $(cat $PID_FILE)

For now, a namespace can be pulled up with bash as the executed process, and the scripts run from there manually, providing the namespace name as argument.

[jamesmcm]

Hey, sorry it's been a while I've moved country and job 😂

I tried to sort out the defaults in configuration file idea in this PR: #72

I'll see if I can do the PostUp and PreDown scripts too (so they could be specified there or as command line arguments, as paths to any executable binary/script).

[jamesmcm]

Merged in #74 and in release 0.7.0 - please test it and see if it works for your use case.

[dani-corie]

Alright, I'll have a look!

[dani-corie]

Okay there is one thing... Could you set an environment variable inside the sandbox with the sandbox name, that's available to these scripts?

[jamesmcm]

Added VOPONO_NS in release 0.7.1 - https://github.com/jamesmcm/vopono/releases/tag/0.7.1

[dani-corie]

Something incredibly weird is happening.

dani@gorillo:~$ cat ~/bin/vpnbash
#!/bin/bash

POSTUP="${HOME}/bin/vpn_postup.sh"
PREDOWN="${HOME}/bin/vpn_predn.sh"

vopono -v exec --dns 127.0.0.16 --provider mullvad --server ${1} --postup ${POSTUP} --predown ${PREDOWN} "bash"
dani@gorillo:~$ cat ~/bin/vpn_postup.sh 
#!/bin/bash

WORK_DIR="${HOME}/.vopono"
mkdir -p ${WORK_DIR}

SANDBOX_NAME=${VOPONO_NS}

if [[ -z "${SANDBOX_NAME}" ]]
then
	echo "No sandbox specified"
	exit 1
fi

SVC_NAME="dnscrypt-proxy"

PROC_NAME="${SANDBOX_NAME}-${SVC_NAME}"

FILE_ROOT=${WORK_DIR}/${PROC_NAME}
LOG_FILE=${FILE_ROOT}.log
PID_FILE=${FILE_ROOT}.pid
LOCK_FILE=${FILE_ROOT}.lock

CFG_FILE="${WORK_DIR}/dnscrypt-proxy.toml"
SVC_COMMAND="/usr/sbin/dnscrypt-proxy -config ${CFG_FILE}"

sudo daemonize -a -c ${WORK_DIR} -e ${LOG_FILE} -p ${PID_FILE} -l ${LOCK_FILE} ${SVC_COMMAND}
dani@gorillo:~$ cat ~/bin/vpn_predn.sh 
#!/bin/bash

WORK_DIR="${HOME}/.vopono"

SANDBOX_NAME=${VOPONO_NS}

if [[ -z "${SANDBOX_NAME}" ]]
then
        echo "No sandbox specified"
        exit 1
fi

SVC_NAME="dnscrypt-proxy"

PROC_NAME="${SANDBOX_NAME}-${SVC_NAME}"
FILE_ROOT=${WORK_DIR}/${PROC_NAME}
PID_FILE=${FILE_ROOT}.pid

sudo kill -s 15 $(cat $PID_FILE)
dani@gorillo:~$ vpnbash sweden
 2021-04-16T14:19:28.952Z DEBUG vopono::pulseaudio > Setting PULSE_SERVER to /run/user/1000/pulse/native
 2021-04-16T14:19:28.952Z INFO  vopono::util       > Calling sudo for elevated privileges, current user will be used as default user
 2021-04-16T14:19:28.952Z DEBUG vopono::util       > Args: ["vopono", "-v", "exec", "--dns", "127.0.0.16", "--provider", "mullvad", "--server", "sweden", "--postup", "/home/dani/bin/vpn_postup.sh", "--predown", "/home/dani/bin/vpn_predn.sh", "bash"]
[sudo] password for dani: 
 2021-04-16T14:19:32.300Z DEBUG vopono::pulseaudio > Setting PULSE_SERVER to /run/user/1000/pulse/native
 2021-04-16T14:19:32.302Z DEBUG vopono::util       > Existing namespaces: []
 2021-04-16T14:19:32.302Z DEBUG vopono::exec       > vopono config.toml: configuration property "firewall" not found
 2021-04-16T14:19:32.302Z DEBUG vopono::exec       > vopono config.toml: configuration property "custom_config" not found
 2021-04-16T14:19:32.302Z DEBUG vopono::exec       > vopono config.toml: configuration property "user" not found
 2021-04-16T14:19:32.302Z DEBUG vopono::exec       > vopono config.toml: configuration property "protocol" not found
 2021-04-16T14:19:32.302Z DEBUG vopono::network_interface > ip addr
 2021-04-16T14:19:32.304Z DEBUG vopono::exec              > Interface: wlp61s0
 2021-04-16T14:19:32.305Z INFO  vopono::util              > Chosen config: /home/dani/.config/vopono/mv/wireguard/sweden-se18.conf
 2021-04-16T14:19:32.306Z DEBUG vopono::util              > Existing namespaces: []
 2021-04-16T14:19:32.306Z DEBUG vopono::util              > ip netns add vopono_mv_sweden
 2021-04-16T14:19:32.307Z INFO  vopono::netns             > Created new network namespace: vopono_mv_sweden
 2021-04-16T14:19:32.308Z DEBUG vopono::util              > Existing interfaces: 8: veth67d2d2c@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-12da57f1c6db state UP group default 
    link/ether 2e:bf:e2:64:a8:3d brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::2cbf:e2ff:fe64:a83d/64 scope link 
       valid_lft forever preferred_lft forever

 2021-04-16T14:19:32.309Z DEBUG vopono::util              > Assigned IPs: []
 2021-04-16T14:19:32.309Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip addr add 127.0.0.1/8 dev lo
 2021-04-16T14:19:32.311Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip link set lo up
 2021-04-16T14:19:32.313Z DEBUG vopono::veth_pair         > NetworkManager detected, adding mv_sweden_d to unmanaged devices
 2021-04-16T14:19:32.313Z DEBUG vopono::util              > nmcli connection reload
 2021-04-16T14:19:32.332Z DEBUG vopono::util              > ip link add mv_sweden_d type veth peer name mv_sweden_s
 2021-04-16T14:19:32.334Z DEBUG vopono::util              > ip link set mv_sweden_d up
 2021-04-16T14:19:32.335Z DEBUG vopono::util              > ip link set mv_sweden_s netns vopono_mv_sweden up
 2021-04-16T14:19:32.344Z DEBUG vopono::util              > ip addr add 10.200.1.1/24 dev mv_sweden_d
 2021-04-16T14:19:32.347Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip addr add 10.200.1.2/24 dev mv_sweden_s
 2021-04-16T14:19:32.354Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip route add default via 10.200.1.1 dev mv_sweden_s
 2021-04-16T14:19:32.357Z INFO  vopono::netns             > IP address of namespace as seen from host: 10.200.1.2
 2021-04-16T14:19:32.357Z INFO  vopono::netns             > IP address of host as seen from namespace: 10.200.1.1
 2021-04-16T14:19:32.357Z DEBUG vopono::util              > iptables -t nat -A POSTROUTING -s 10.200.1.0/24 -o wlp61s0 -j MASQUERADE
 2021-04-16T14:19:32.361Z DEBUG vopono::util              > iptables -I FORWARD -i mv_sweden_d -o wlp61s0 -j ACCEPT
 2021-04-16T14:19:32.371Z DEBUG vopono::util              > iptables -I FORWARD -o mv_sweden_d -i wlp61s0 -j ACCEPT
 2021-04-16T14:19:32.378Z DEBUG vopono::util              > sysctl -q net.ipv4.ip_forward=1
 2021-04-16T14:19:32.382Z DEBUG vopono::wireguard         > Deserializing: 193.138.218.74 to Vec<IpAddr>
 2021-04-16T14:19:32.382Z DEBUG vopono::wireguard         > TOML config: WireguardConfig { interface: WireguardInterface { private_key: "ACLmYnuCd5czuugVuyKHunw7ZJoTb+Pb+3TAzsYdBUM=", address: [10.65.233.4/32, fc00:bbbb:bbbb:bb01::2:e903/128], dns: [193.138.218.74] }, peer: WireguardPeer { public_key: "fZFAcd8vqWOBpRqlXifsjzGf16gMTg2GuwKyZtkG6UU=", allowed_ips: [0.0.0.0/0, ::/0], endpoint: 193.138.218.83:51820 } }
 2021-04-16T14:19:32.382Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip link add mv_sweden type wireguard
 2021-04-16T14:19:32.385Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden wg setconf mv_sweden /tmp/vopono_nft.conf
 2021-04-16T14:19:32.395Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -4 address add 10.65.233.4/32 dev mv_sweden
 2021-04-16T14:19:32.397Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -6 address add fc00:bbbb:bbbb:bb01::2:e903/128 dev mv_sweden
 2021-04-16T14:19:32.398Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip link set mtu 1420 up dev mv_sweden
 2021-04-16T14:19:32.412Z DEBUG vopono::dns_config        > Setting namespace vopono_mv_sweden DNS server to 127.0.0.16
 2021-04-16T14:19:32.412Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden wg set mv_sweden fwmark 51820
 2021-04-16T14:19:32.413Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -4 route add 0.0.0.0/0 dev mv_sweden table 51820
 2021-04-16T14:19:32.415Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -4 rule add not fwmark 51820 table 51820
 2021-04-16T14:19:32.418Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -4 rule add table main suppress_prefixlength 0
 2021-04-16T14:19:32.420Z DEBUG vopono::util              > sysctl -q net.ipv4.conf.all.src_valid_mark=1
 2021-04-16T14:19:32.420Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -6 route add ::/0 dev mv_sweden table 51820
 2021-04-16T14:19:32.423Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -6 rule add not fwmark 51820 table 51820
 2021-04-16T14:19:32.425Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip -6 rule add table main suppress_prefixlength 0
 2021-04-16T14:19:32.427Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden iptables -t raw -A PREROUTING ! -i mv_sweden -d 10.65.233.4/32 -m addrtype ! --src-type LOCAL -j DROP
 2021-04-16T14:19:32.452Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip6tables -t raw -A PREROUTING ! -i mv_sweden -d fc00:bbbb:bbbb:bb01::2:e903/128 -m addrtype ! --src-type LOCAL -j DROP
 2021-04-16T14:19:32.480Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden iptables -t mangle -A POSTROUTING -p udp -j MARK --set-mark 51820
 2021-04-16T14:19:32.511Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden iptables -t mangle -A PREROUTING -p udp -j CONNMARK --save-mark
 2021-04-16T14:19:32.532Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip6tables -t mangle -A POSTROUTING -p udp -j MARK --set-mark 51820
 2021-04-16T14:19:32.557Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip6tables -t mangle -A PREROUTING -p udp -j CONNMARK --save-mark
 2021-04-16T14:19:32.570Z DEBUG vopono::wireguard         > Setting Wireguard killswitch....
 2021-04-16T14:19:32.570Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden iptables -A OUTPUT ! -o mv_sweden -m mark ! --mark 51820 -m addrtype ! --dst-type LOCAL -j REJECT
 2021-04-16T14:19:32.589Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden ip6tables -A OUTPUT ! -o mv_sweden -m mark ! --mark 51820 -m addrtype ! --dst-type LOCAL -j REJECT
 2021-04-16T14:19:32.626Z DEBUG vopono::netns             > Writing lockfile: /home/dani/.config/vopono/locks/vopono_mv_sweden
 2021-04-16T14:19:32.626Z DEBUG vopono::netns             > Lockfile written: /home/dani/.config/vopono/locks/vopono_mv_sweden/102123
[sudo] password for dani:  2021-04-16T14:19:32.682Z DEBUG vopono::netns             > ip netns exec vopono_mv_sweden sudo -Eu dani bash
 2021-04-16T14:19:32.682Z INFO  vopono::exec              > Application bash launched in network namespace vopono_mv_sweden with pid 102191
dani@gorillo:~$ 

[1]+  Stopped                 vpnbash sweden
dani@gorillo:~$ Sorry, try again.

exit
dani@gorillo:~$ 
dani@gorillo:~$ 
dani@gorillo:~$ 
dani@gorillo:~$ exit
logout
There are stopped jobs.
dani@gorillo:~$ fg
vpnbash sweden
[sudo] password for dani:  2021-04-16T14:19:45.978Z INFO  vopono::netns             > Shutting down vopono namespace - as there are no processes left running inside
 2021-04-16T14:19:45.978Z DEBUG vopono::util              > ip link delete mv_sweden_d
cat: /home/dani/.vopono/vopono_mv_sweden-dnscrypt-proxy.pid: No such file or directory
 2021-04-16T14:19:45.989Z DEBUG vopono::util              > nmcli connection reload
[sudo] password for dani:  2021-04-16T14:19:46.006Z DEBUG vopono::util              > ip netns exec vopono_mv_sweden ip link del mv_sweden
 2021-04-16T14:19:46.069Z DEBUG vopono::util              > iptables -t nat -D POSTROUTING -s 10.200.1.0/24 -o wlp61s0 -j MASQUERADE
 2021-04-16T14:19:46.080Z DEBUG vopono::util              > iptables -D FORWARD -o mv_sweden_d -i wlp61s0 -j ACCEPT
 2021-04-16T14:19:46.091Z DEBUG vopono::util              > iptables -D FORWARD -i mv_sweden_d -o wlp61s0 -j ACCEPT
 2021-04-16T14:19:46.103Z DEBUG vopono::util              > ip netns delete vopono_mv_sweden
dani@gorillo:~$ 
sudo: unable to read password: Input/output error
sudo: 1 incorrect password attempt
l
sudo: unable to read password: Input/output error

dani@gorillo:~$ 

Something is REALLY messed up with bg/fg and all that. Note that no Ctrl+key combinations have been pressed, nothing strange was done, this is all happening by itself.

There is no issue unless --postup and --predown are used. So it's clearly those scripts that are causing the issue.

Are you using some kind of magic to call these scripts parallel or in the background? I don't think it's a good idea to do that. They should be executed in series with the startup at the appropriate point.

[dani-corie]

One clear and rather big issue, even if it didn't blow up, would be that the postup and predown script execution isn't even mentioned in the (verbose) log output.

[dani-corie]

Okay so I made a wrapper script that just allows me to kick off bash sessions with a set vpn connection and dnscrypt. I'll include it here (also as an example of the kind of behavior that would be ideal in my opinion):

vpnbash.sh:

#!/bin/bash
vopono -v exec --dns 127.0.0.16 --provider mullvad --server ${1} "${HOME}/bin/vpn_env_dnscrypt.sh bash ${1}"

vpn_env_dnscrypt.sh:

#!/bin/bash

WORK_DIR="${HOME}/.vopono"
mkdir -p ${WORK_DIR}

SANDBOX_NAME=${2}

if [[ -z "${SANDBOX_NAME}" ]]
then
	echo "No sandbox specified"
	exit 1
fi

SVC_NAME="dnscrypt-proxy"

PROC_NAME="${SANDBOX_NAME}-${SVC_NAME}"

FILE_ROOT=${WORK_DIR}/${PROC_NAME}
LOG_FILE=${FILE_ROOT}.log
PID_FILE=${FILE_ROOT}.pid
LOCK_FILE=${FILE_ROOT}.lock

CFG_FILE="${WORK_DIR}/dnscrypt-proxy.toml"
SVC_COMMAND="/usr/sbin/dnscrypt-proxy -config ${CFG_FILE}"

echo "Starting dnscrypt service with PID file ${PID_FILE}"
sudo daemonize -a -c ${WORK_DIR} -e ${LOG_FILE} -p ${PID_FILE} -l ${LOCK_FILE} ${SVC_COMMAND}

${1}

echo "Killing dnscrypt service at $(cat $PID_FILE)"
sudo kill -s 15 $(cat $PID_FILE)

This implements the postup/predown behavior automatically by wrapping the entire session in a bash script. When the child process (from the command passed as the first argument) terminates, the script resumes and kills the daemon, after which control is given back to vopono.

Honestly, I'm starting to think that probably this makes the need for a built-in postUp/preDn moot - in case you have limited resources, I'd be perfectly fine if you dropped the feature rather than spent time fixing it. With security-critical software like this, it's probably better to have a smaller set of functionality.