This is the official code repository for the paper:
This codebase is originally forked from IDSIA/modern-srwm which we modified for continual learning (also including improved practical settings for self-referential weight matrices, e.g., better initialization strategy).
NB: this is research code with many sub-optimal implementations (search for NB: in main.py for various comments).
Our codebase also includes code from other public repositories, e.g.,
-
tristandeleu/pytorch-meta for standard few-shot learning data preparation/processing and data-loader implementations. (forked and slightly modified code can be found under
torchmeta_local) -
khurramjaved96/mrcl for the OML baseline (Table 3). Forked and modified code can be found under
oml_baseline_local. We downloaded their out-of-the-box Omniglot model from their Google drive from the same repository. -
GT-RIPL/Continual-Learning-Benchmark: this is not included here but we modified/used it to produce the results for the 2-task class-incremental setting (Table 3)
as well as other architectural implementations (currently not reported in the paper):
-
lucidrains/mlp-mixer-pytorch for MLP mixer.
-
yinboc/few-shot-meta-baseline for Res-12.
Please find LICENSE files/mentions in the corresponding directory/fileheaders.
The basic requirements are same as the original repository IDSIA/modern-srwm/supervised_learning.
We used PyTorch 1.10.2+cu102 or 1.11.0 in our experiments but newer versions should also work.
Example training and evaluation scripts are provided under scripts.
Our pre-trained model checkpoints can be downloaded from this Google drive link.
@article{irie2023automating,
title={Automating Continual Learning},
author={Irie, Kazuki and Csord{\'a}s, R{\'o}bert and Schmidhuber, J{\"u}rgen},
journal={Preprint arXiv:2312.00276},
year={2023}
}