Skip to content

Latest commit

 

History

History
86 lines (56 loc) · 4.99 KB

README.md

File metadata and controls

86 lines (56 loc) · 4.99 KB

Parametric Classification for Generalized Category Discovery: A Baseline Study

Parametric Classification for Generalized Category Discovery: A Baseline Study (ICCV 2023)
By Xin Wen*, Bingchen Zhao*, and Xiaojuan Qi.

teaser

Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. Previous studies argued that parametric classifiers are prone to overfitting to seen categories, and endorsed using a non-parametric classifier formed with semi-supervised $k$-means.

However, in this study, we investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We demonstrate that two prediction biases exist: the classifier tends to predict seen classes more often, and produces an imbalanced distribution across seen and novel categories. Based on these findings, we propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers. We hope the investigation and proposed simple framework can serve as a strong baseline to facilitate future studies in this field.

Running

Dependencies

pip install -r requirements.txt

Config

Set paths to datasets and desired log directories in config.py

Datasets

We use fine-grained benchmarks in this paper, including:

We also use generic object recognition datasets, including:

Scripts

Train the model:

bash scripts/run_${DATASET_NAME}.sh

We found picking the model according to 'Old' class performance could lead to possible over-fitting, and since 'New' class labels on the held-out validation set should be assumed unavailable, we suggest not to perform model selection, and simply use the last-epoch model.

Results

Our results:

SourcePaper (3 runs) Current Github (5 runs)
DatasetAllOldNewAllOldNew
CIFAR1097.1±0.095.1±0.198.1±0.197.0±0.193.9±0.198.5±0.1
CIFAR10080.1±0.981.2±0.477.8±2.079.8±0.681.1±0.577.4±2.5
ImageNet-10083.0±1.293.1±0.277.9±1.983.6±1.492.4±0.179.1±2.2
ImageNet-1K57.1±0.177.3±0.146.9±0.257.0±0.477.1±0.146.9±0.5
CUB60.3±0.165.6±0.957.7±0.461.5±0.565.7±0.559.4±0.8
Stanford Cars53.8±2.271.9±1.745.0±2.453.4±1.671.5±1.644.6±1.7
FGVC-Aircraft54.2±1.959.1±1.251.8±2.354.3±0.759.4±0.451.7±1.2
Herbarium 1944.0±0.458.0±0.436.4±0.844.2±0.257.6±0.637.0±0.4

Citing this work

If you find this repo useful for your research, please consider citing our paper:

@inproceedings{wen2023simgcd,
    author    = {Wen, Xin and Zhao, Bingchen and Qi, Xiaojuan},
    title     = {Parametric Classification for Generalized Category Discovery: A Baseline Study},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year      = {2023},
    pages     = {16590-16600}
}

Acknowledgements

The codebase is largely built on this repo: https://github.com/sgvaze/generalized-category-discovery.

License

This project is licensed under the MIT License - see the LICENSE file for details.