Novel View Synthesis for Scenes in the Wild

Michal Neumayr, Tim Tomov
Technical University of Munich

Code | Report | Poster

This is the repository for the project "Novel View Synthesis for Scenes in the Wild" as part of the course "Advanced Deep Learning for Visual Computing" by Prof. Nießner's chair. We propose a novel combination of local as well as global conditioning modules with which we finetune Stable Diffusion on the ScanNet++ dataset.

1. Set up wandb (https://docs.wandb.ai/quickstart)

Sign up for a free account and login to your wandb account.

wandb login

Paste the API key from https://wandb.ai/authorize when prompted.

2. Download the dataset

Register on the website and follow the download script.

3. Set up the environment

Clone repository and set up virtual environment.

git clone https://github.com/michaelnoi/scene_nvs.git
cd scene_nvs

pyenv virtualenv 3.10.10 nvs
pyenv local nvs

Install packages with poetry.

poetry install