When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images

Mahmoud Afifi¹, Brian Price², Scott Cohen², and Michael S. Brown<sup>1

1</sup>York University    ²Adobe Research

Reference code for the paper When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images. Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown, CVPR 2019. If you use this code or our dataset, please cite our paper:

@inproceedings{afifi2019color,
  title={When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images},
  author={Afifi, Mahmoud and Price, Brian and Cohen, Scott and Brown, Michael S},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={1535--1544},
  year={2019}
}

The original source code of our paper was written in Matlab. We also provide a Python version of our code. We tried to make both versions identical. However, there is no guarantee that the Python version will give exactly the same results. The differences should be due to rounding errors when we converted our model to Python or differences between Matlab and OpenCV in reading compressed images.

Quick start

1. Matlab:

1. Run install_.m
2. Run demo.m to process a single image or demo_images.m to process all images in a directory.
3. Check evaluation_examples.m for examples of reporting errors using different evaluation metrics. Also, this code includes an example of how to hide the color chart for Set1 images.

2. Python:

1. Requirements: numpy, opencv-python, and skimage (skimage is required for evaluation code only).
2. Run demo.py to process a single image or demo_images.py to process all images in a directory.
3. Check evaluation_examples.py for examples of reporting errors using different evaluation metrics. Also, this code includes an example of how to hide the color chart for Set1 images.

Graphical user interface

We provide a Matlab GUI to help tuning our parameters in an interactive way. Please, check demo_GPU.m.

Code/GUI parameters and options

1. K: Number of nearest neighbors in the KNN search (Sec. 3.4 in the paper) -- change its value to enhance the results.
2. sigma: The fall-off factor for KNN blending (Eq. 8 in the paper) -- change its value to enhance the results.
3. device: GPU or CPU (provided for Matlab version only).
4. gamut_mapping: Mapping pixels in-gamut either using scaling (gamut_mapping= 1) or clipping (gamut_mapping= 2). In the paper, we used the clipping options to report our results, but the scaling option gives compelling results in some cases (esp., with high-saturated/vivid images).
5. upgraded_model and upgraded: To load our upgraded model, use upgraded_model=1 in Matlab or upgraded=1 in Python. The upgraded model has new training examples. In our paper results, we did not use this model. However, our online demo uses it.

Dataset

In the paper, we mentioned that our dataset contains over 65,000 images. We further added two additional sets of rendered images, for a total of 105,638 rendered images. You can download our dataset from here. You can also download the dataset from the following links:

Input images: Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 | Part8 | Part9 | Part10

Input images (without color chart pixels): Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 | Part8 | Part9 | Part10

Augmented images (without color chart pixels): Download (rendered with additional/rare color temperatures)

Ground-truth images: Download

Ground-truth images (without color chart pixels): Download

Metadata files: Input images | Ground-truth images

Folds: Download

Online demo

Try the interactive demo by uploading your photo or paste a URL for a photo from the web.

Project page

For more information, please visit our project page

Related Research Projects

• White-Balance Augmenter: Emulating white balance effects for color augmentation. It can improve the accuracy of image classification and image semantic segmentation methods (ICCV 2019).
• Color Temperature Tuning: A camera pipeline that allows accurate post-capture white balance editing (CIC best paper award, 2019).
• SIIE: A sensor-independent deep learning framework for illumination estimation (BMVC 2019).
• Deep White-Balance Editing: A multi-task deep learning model for post-capture white-balance editing (CVPR 2020).

Commercial Use

This software and the dataset are provided for research purposes only. A license must be obtained for any commercial application.