Pröspect image bot

This is the images processing bot behind Pröspect, an art project that reassembles images from McCord Museum's photo collection.

The bot consists in a Flask API relying on a few libraries to process images:

• Google's Cloud Vision API is used to find images similar to the one that is being processed
• OpenCV is used to extract and colorize features in the original image.
• pixelsort is used to create a pixel-sorted version of the original image
• Pillow is responsible for assembling and saving the final image
• pytracery generates random poems by picking verses in the text_rules.json file

Example

Here's an example of each processing step.

Original image

First, an image is picked randomly in the museum's collection.

Modified images

Then we generate a few modified vesions of the image:

1. We use OpenCV's grabCut method to extract the image's background and replace it with the current color
2. OpenCV's contours method let's us draw some features in the image, again using the current color. This image is also used to find similar images.
3. The third image is the pixel-sorted version

Final image

We then compose the final image by taking a slice of each version of the image, as well as a slice of 5 visually similar images found with Google's Cloud Vision API.

Requirements

• Python >=3.6.5
• OpenCV-Python
• pixelsort
• MongoDB

Install

$ python -m venv venv
$ . venv/bin/activate
$ pip install -r requirements.txt
$ apt-get install python-opencv

Setup

• Make sure you have an Vision API-authorized GCP service account key saved as ./service-account.json.
• Clone pixelsort in the project's root:

git clone https://github.com/satyarth/pixelsort.git

• Create a .env file based on the example file and set environment variables as needed

cp .env.example .env

Run in development

Start Flask app in debug mode.

$ ./dev

Run in production

Start Flask app with gunicorn.

$ ./prod

Freeze requirements

$ pip freeze > requirements.txt

Cleanup intermediary image files

$ rm static/(colorized|pixelsorted)/*.png

Usage

Once you have the bot up and running, process images by calling the /process-image endpoint with the image_url GET parameter, ie:

http://127.0.0.1:5000/process-image?image_url=https://i.imgur.com/ByVWQfM.jpg

Credits

• Isabelle Gagné - multimedia artist
• Paul Gascou-Vaillancourt - programmer
• Stéphane Archambault - poem