Incense

Though automated logging of machine learning experiments results is crucial, it does not replace manual interpretation. Incense is a toolbox to facilitate manual interpretation of experiments that are logged using sacred. It lets you find and evaluate experiments directly in Jupyter notebooks. Incense lets you query the database for experiments by id, name or any hyperparmeter value. For each found experiment, configuration, artifacts and metrics can be displayed. The artifacts are rendered according to their type, e.g. a PNG image is displayed as an image, while a CSV file gets transformed to a pandas DataFrame. Metrics are by default transformed into pandas Series, which allows for flexible plotting. Together with sacred and incense, Jupyter notebooks offer the perfect solution for interpreting experiments as they allow for a combination of code that reproducibly displays the experiment’s results, as well as text that contains the interpretation.

Installation

Install the latest release

pip install incense

Or install the latest development version

pip install git+https://github.com/JarnoRFB/incense.git

Documentation

demo.ipynb demonstrates the basic functionality of incense. You can also try it out interactively on binder.

Contributing

We recommend using the VSCode devcontainer for development. It will automatically install all dependencies and start necessary services, such as mongoDB and JupyterLab. See .devcontainer/docker-compose.yml for details. If the output of id -u is something different than 1000 on your system, please add

export UID

to your .bashrc or .zshrc.

Building the container for the first time may take some time. Once in the container run

$ pre-commit install
$ python tests/example_experiment/conduct.py

to set up the pre-commit hooks and populate the example database.

Alternatively, you can use conda to set up your local development environment.

$ conda create -n incense-dev python=3.7
$ conda activate incense-dev
# virtualenv is required for the precommit environments.
$ conda install virtualenv
# tox-conda is required for using tox with conda.
$ pip install tox-conda
$ pip install -r requirements-dev.txt
$ pre-commit install