JupyterHub Pharma2-53

This is a custom Dockerspawner for JupyterHub that gives more choice to the users, permitting the admin allows it.

It restricts user resources to Docker containers, whilst still permitting them to use their conda environments.

Quick Start

A full breakdown of the installation process can be read in the ./COMPLETE_GUIDE.org, but if your OS is relatively modern (read: Arch, not: Ubuntu):

1. Clone the following repositories into the following directories

   sudo mkdir -p /opt/jupyterhub
   sudo chown $(whoami): /opt/jupyterhub
   git clone --depth 1 https://gitlab.com/mtekman/jupyterhub-pharma253 -b main /opt/jupyterhub/jupyterhub-pharma
   git clone --depth 1 https://github.com/mtekman/jupyterhub/ -b sysmon /opt/jupyterhub/jupyterhub-metrics
       

2. Generate a virtualenv of the Jupyterhub metrics repo and install it

   cd /opt/jupyterhub/jupyterhub-pharma
   python -m virtualenv venv_jupyter_metrics
   source venv_jupyter_metrics/bin/activate
   pip install ../jupyterhub-metrics
   pip install dockerspawner psutil configurable-http-proxy
       

3. Configure Jupyter for your system

   Please read the “Jupyter Config file” section in the ./COMPLETE_GUIDE.org, as well as finding the docker subnet, and allowing it within the firewall.

4. Setup the SystemD and Docker Limits
   • First configure the files in the system/etc/systemd/system directory, namely:
     • docker_limit.slice: Set the correct CPUQuota and MemoryHigh limits to match your system
     • proxy-tunnel.service: Set the correct user, ssh port, proxy binding port, proxyuser, and proxydevice
     • jupyterhub.service: Set the PATH to the virtual environment we installed above
   • Then, copy over all systemd files to /etc/systemd/system/
   • Modify the /etc/docker/daemon.json file to:

     {
         "storage-driver": "overlay2",
         "cgroup-parent": "docker_limit.slice"
     }
             

5. Run JupyterHub

   sudo systemctl start jupyterhub
       

   It should automatically spawn up the proxy, and docker limits by itself.

   You can verify that the limits are in place by invoking

   sudo systemd-cgtop
       

   and witnessing that all docker process are children of the docker-slice process.

Custom Docker Image

This needs to be built and added to the config file:

sudo docker buildx build -t bash-python-r docker-image

and then set

"bash-python-r": "docker.io/library/bash-python-r"

as an option in c.JupyterHub.spawner_class.docker_profiles

Administration

The main active forms of administration is adding users, stopping their docker containers if they get too greedy, or upgrading them if they need more resources.

If you should so wish to, you can also copy over the administration script at system/etc/profile.d/pharma_admin.sh into your /etc/profile.d/, which should print a welcome message of common administrative tasks for any users in the sudo group

For adding users, a convenience script is also provided system/usr/local/sbin/add_new_pharma_user.sh that you should copy over to /usr/local/sbin.

Troubleshooting

Please see the “Troubleshooting” section in the ./COMPLETE_GUIDE.org