From c236a34daec5bcd054cbcb973207a8338c2f2cd1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Barz?= Date: Wed, 11 Dec 2019 14:55:09 +0100 Subject: [PATCH] Updated README and documentation --- CUB-Hierarchy/README.md | 13 ++++++++++++- CosineLoss.md | 31 ++++++++++++++++++------------- README.md | 3 ++- datasets/__init__.py | 1 + 4 files changed, 33 insertions(+), 15 deletions(-) diff --git a/CUB-Hierarchy/README.md b/CUB-Hierarchy/README.md index 2bf9312..23d4fd3 100644 --- a/CUB-Hierarchy/README.md +++ b/CUB-Hierarchy/README.md @@ -55,8 +55,19 @@ Known Issues - Class 110 (***Geococcyx***) is at the *genus* level, but there are only 2 possible species with minor visual differences. +Citation +-------- + +If you use this hierarchy for the CUB dataset in your work, please cite the following paper: + +> [**Deep Learning on Small Datasets without Pre-Training usine Cosine Loss.**][6] +> Björn Barz and Joachim Denzler. +> IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. + + [1]: http://www.vision.caltech.edu/visipedia/CUB-200-2011.html [2]: https://species.wikimedia.org/ [3]: https://www.wikidata.org/ [4]: https://en.wikipedia.org/ -[5]: https://tree.opentreeoflife.org/ \ No newline at end of file +[5]: https://tree.opentreeoflife.org/ +[6]: https://arxiv.org/pdf/1901.09054 \ No newline at end of file diff --git a/CosineLoss.md b/CosineLoss.md index 379d2c1..f49bcb6 100644 --- a/CosineLoss.md +++ b/CosineLoss.md @@ -4,18 +4,19 @@ This document explains how the code in this repository can be used to produce th > [**Deep Learning on Small Datasets without Pre-Training using Cosine Loss.**][1] > Björn Barz and Joachim Denzler. +> IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. ## 1. Results According to Table 2 in the paper: -| Loss Function | CUB | NAB | Cars | Flowers | CIFAR-100 | -|---------------------------------|----------:|----------:|----------:|----------:|----------:| -| cross entropy | 51.9% | 59.4% | 78.2% | 67.3% | 77.0% | -| cross entropy + label smoothing | 55.9% | 68.3% | 78.1% | 66.8% | **77.5%** | -| cosine loss | 67.6% | 71.7% | 84.3% | **71.1%** | 75.3% | -| cosine loss + cross entropy | **68.0%** | **71.9%** | **85.0%** | 70.6% | 76.4% | +| Loss Function | CUB | NAB | Cars | Flowers | MIT 67 Scenes | CIFAR-100 | +|---------------------------------|----------:|----------:|----------:|----------:|--------------:|----------:| +| cross entropy | 51.9% | 59.4% | 78.2% | 67.3% | 44.3% | 77.0% | +| cross entropy + label smoothing | 55.9% | 68.3% | 78.1% | 66.8% | 38.7% | **77.5%** | +| cosine loss | 67.6% | 71.7% | 84.3% | **71.1%** | 51.5% | 75.3% | +| cosine loss + cross entropy | **68.0%** | **71.9%** | **85.0%** | 70.6% | **52.7%** | 76.4% | ## 2. Requirements @@ -38,6 +39,7 @@ The following datasets have been used in the paper: - [North American Birds][3] (NAB-large) - [Stanford Cars][5] (Cars) - [Oxford Flowers-102][6] (Flowers) +- [MIT 67 Indoor Scenes][7] (MIT67Scenes) - [CIFAR-100][2] (CIFAR-100) The names in parentheses specify the dataset names that can be passed to the scripts mentioned below. @@ -104,17 +106,18 @@ done The following table lists the values for `--sgdr_max_lr` that led to the best results. -| Loss | CUB | NAB | Cars | Flowers | CIFAR-100 | -|----------------------------------------|------:|------:|-----:|--------:|----------:| -| cross entropy | 0.05 | 0.05 | 1.0 | 1.0 | 0.1 | -| cross entropy + label smoothing | 0.05 | 0.1 | 1.0 | 0.1 | 0.1 | -| cosine loss (one-hot) | 0.5 | 0.5 | 1.0 | 0.5 | 0.05 | -| cosine loss + cross entropy (one-hot) | 0.5 | 0.5 | 0.5 | 0.5 | 0.1 | +| Loss | CUB | NAB | Cars | Flowers | MIT 67 Scenes | CIFAR-100 | +|----------------------------------------|------:|------:|-----:|--------:|--------------:|----------:| +| cross entropy | 0.05 | 0.05 | 1.0 | 1.0 | 0.05 | 0.1 | +| cross entropy + label smoothing | 0.05 | 0.1 | 1.0 | 0.1 | 1.0 | 0.1 | +| cosine loss (one-hot) | 0.5 | 0.5 | 1.0 | 0.5 | 2.5 | 0.05 | +| cosine loss + cross entropy (one-hot) | 0.5 | 0.5 | 0.5 | 0.5 | 2.5 | 0.1 | ## 5. Sub-sampling CUB -To experiment with differently sized variants of the CUB dataset, copy the image list files from the directory [`CUB-splits`](CUB-splits/) into the root directory of your CUB dataset and specify the dataset name as `CUB-subX`, where `X` is the number of samples per class. +To experiment with differently sized variants of the CUB dataset, download the [modified image list files][8] and unzip the obtained archive into the root directory of your CUB dataset. +For training, specify the dataset name as `CUB-subX`, where `X` is the number of samples per class. ![Performance comparison for differently sub-sampled variants of the CUB dataset](https://user-images.githubusercontent.com/7915048/51765373-d67bb600-20d7-11e9-85a9-ec6f28cef39b.png) @@ -126,3 +129,5 @@ To experiment with differently sized variants of the CUB dataset, copy the image [4]: http://www.vision.caltech.edu/visipedia/CUB-200-2011.html [5]: https://ai.stanford.edu/~jkrause/cars/car_dataset.html [6]: http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html +[7]: http://web.mit.edu/torralba/www/indoor.html +[8]: https://github.com/cvjena/semantic-embeddings/releases/download/v1.2.0/cub-subsampled-splits.zip diff --git a/README.md b/README.md index a8b02ab..db8283e 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,9 @@ This repository contains the official source code used to produce the results re > [**Deep Learning on Small Datasets without Pre-Training usine Cosine Loss.**][23] > Björn Barz and Joachim Denzler. +> IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. -If you use this code, please cite one of those papers (the more appropriate one, depending on your application). +If you use this code, please cite one of those papers (the first one when you work with hierarchy-based semantic embeddings, the second one when you use the cosine loss for classification). The remainder of this ReadMe will focus on learning hierarchy-based semantic image embeddings (the first of the papers mentioned above). If you came here for more information on how we obtained the results reported in the paper about using the cosine loss for classification on small datasets (the second one), you can find those [here][24]. diff --git a/datasets/__init__.py b/datasets/__init__.py index 0da2c9e..19fdf50 100644 --- a/datasets/__init__.py +++ b/datasets/__init__.py @@ -35,6 +35,7 @@ def get_data_generator(dataset, data_root, classes = None): - "cub" - "cars" - "flowers" + - "mit67scenes" - "inat" / "inat2018" (optionally followed by an underscore and the name of a super-category) - "inat2019" - "UCMLU"