Using pyenv

The pyenv utility allows you to install any and all versions of Python in parallel in your Home directory. It has some overlap with conda, another way to install multiple versions of Python into your home directory. In fact, though, conda (Miniconda/Anaconda) is contained in pyenv, so you can get the best of both systems by installing them withing pyenv.

Benefits of a pyenv-based workflow over a conda-based workflow

• pyenv allows to have multiple versions of Python active at the same time. A lot of projects using [tox][] for testing rely on this. In contrast, conda has a specific Python version for each environment, and while you can create as many environments as you want (each with a different version of Python), only one environment can be active at a time.
• Virtual project environments created with pyenv are much smaller than conda environments
• pyenv installs packages with pip exclusively, which means installing either the official pre-compiled wheels, or installing packages from source. Assuming you have all the necessary compilers and dependencies installed, this can avoid the crashes sometimes associated with binary incompatibilities in the (less standardized) conda packages. Current (2018) versions of QuTiP are a particular problem.

Benefits of a conda-based workflow over a pyenv-based workflow

• conda is a general package manager, not restricted to Python. You can install git, pandoc, npm and many other loosely related programs through conda. This is particularly useful for Jupyter Notebooks / JupyterLab, which have optional dependencies on these programs.
• Packages that do not provide wheels and are hard to install from source may provide conda packages, which would be easier to install. As wheels are becoming more and more common, this is likely to become less relevant in the future.
• The Anaconda distribution specifically guarantees compatibility of the packages it includes.

Installing pyenv

Follow pyenv's installation instructions

• git clone https://github.com/pyenv/pyenv.git ~/.pyenv

• git clone https://github.com/pyenv/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

• Modify your ~/.bashrc to set up the correct paths:

  • near the top of your .bashrc, put

    export PYENV_ROOT=$HOME/.pyenv
    export PATH=$PYENV_ROOT/bin:$PATH
    

  • near the bottom of your .bashrc, put

    if [ -d $PYENV_ROOT ]; then
        eval "$(pyenv init -)"
    fi
    export PATH=$HOME/bin:$PATH
    

• Log out and in again, check that pyenv versions shows system as the active environment

• Install all the Python versions that you might want to use

  pyenv install 2.7.16
  pyenv install 3.4.10
  pyenv install 3.5.7
  pyenv install 3.6.8
  pyenv install 3.7.3
  pyenv install 3.8-dev
  pyenv install miniconda3-latest
  

  You may use newer versions if they are available

• Activate the python versions in order of precedence

  pyenv global 3.7.3 3.6.8 3.5.7 3.4.10 2.7.16 miniconda-latest
  

  This means that going forward, the commands python and python3 will refer to Python 3.7.3, python2 will refer to Python 2.7.16, python2.7, python3.4, python3.5, python3.6, python3.7, and python3.8 will refer to the respective Python version, and conda will allow to create and manage Conda-environments based on Miniconda 3. All of these will be available in your shell at the same time, giving you the greatest flexibility.

At any time, you can now create project-specific virtual environments.

This can be either "named" environments that will be stored e.g. ~/.pyenv/versions or ~/anaconda3/envs, or an environment rooted at a specific path (the recommendation is the .venv subfolder of a particular project-folder).

• Creating a named environment ("myproject") with pyenv:

  pyenv virtualenv 3.6.8 myproject
  pyenv activate myproject
  

• Creating a named environment ("myproject") with conda:

  conda create -n myproject python=3.6.8
  conda activate myproject
  

• Creating an environment in the .venv folder of the current working directory, with pyenv:

  python3.6 -m venv .venv
  source .venv/bin/activate
  

• Creating an environment in the .venv folder of the current working directory, with conda:

  conda create -p .venv python=3.6.8
  source .venv/bin/activate
  

Virtual environments in the .venv subfolder of a project are recommended above named environments: They are nicely isolated and can easily created and managed with a Makefile in the project folder. Using the pyenv method will produce a much smaller .venv folder than the conda method.