Self-supervised learning of Video Object Segmentation using DINOSAUR and SAVi
Currently, results are being consolidated and a paper is in progress. For now, enjoy the source code.
Presentation Slides: https://1drv.ms/p/s!AgE9E4ZerfvahsdfaYUe8caynzF0iw
Models: https://1drv.ms/f/s!AgE9E4Zerfvahsd8TTjyDmHvikwY6g?e=o2n7EY
Note: https://hydra.cc/ is used to manage configuration, CLI and experiment running. See https://hydra.cc/docs/advanced/override_grammar/basic/ for CLI override grammar. Jump to Codebase Notes for more info.
# (Optional) Use conda for virtual environment instead. Poetry creates venv by default.
conda create -n dinosavi python=3.10
poetry installpython -m dinosavi mode=train hparam={hparam_file_name} exp_name={experiment_name}python -m dinosavi mode=eval resume={.ckpt_to_load} exp_name={results_name} device={cpu_or_cuda}- Hydra (built on OmegaConf) is used for configuration management, CLI and experiment running.
- For CLI options, you can temporarily refer to
dinosavi/cfg/main.yamlanddinosavi/cfg/modefor now. Proper CLI documentation is in progress. - Hyperparameter config files are stored in
dinosavi/cfg/hparam.- OmegaConf Variable Interpolation is heavily used to link hyperparameter values into relevant config files.
- Many parts of the code (i.e., models, datasets, trainers) are implemented as config files using Hydra's Instantiate API.
- For CLI options, you can temporarily refer to
- We use Black for formatting, isort for import sorting, and Google-style docstrings.
- Anything to do with Contrastive Random Walk (CRW) is leftover legacy code from earlier experiments.