Implementing NeRF model from scratch
v1_coarse_fine | ||
---|---|---|
Chair | Ship | Hotdog |
|
|
|
v11 | ||
---|---|---|
Hotdog | ||
|
v16 | ||
---|---|---|
Lego | Hotdog | Materials |
Final Version (v18) (Special thanks to: [4]) | |
---|---|
- Capture a video (forward facing), place it in Image2Pose_NERF/customdataset folder.
- Open terminal in that folder and extract frames using []:
ffmpeg -i input.mp4 '%04d.png'
- Create a folder name 'extract' inside customdataset and place the extracted image in this folder.
- Reduce the number of frames using filterimage.py code present in Image2Pose_NERF/customdataset folder.
- Open COLMAP, click on new project -> select the customdataset for database and customdataset/images for image path -> save
- Now in COLMAP select Reconstruction option -> Auto Reconstruction -> Select workspace as customdataset -> Select image folder as customdataset/images -> select data type as video frames -> check shared intrinsics -> check sparse -> uncheck dense and now click Run. It will create Sparse name folder in customdataset.
- Now, in Image2Pose_NERF folder run image2pose.py, it will create poses_bounds.npy file []:
python image2pose.py customdataset/
- customdataset folder is final dataset.
Parameters | Values |
---|---|
Iteration | 200K |
Scheduler | Exponential Decay |
Scheduler Step | 160K approx. |
Rays Sample | 1024 |
Crop | 0.5 |
Pre Crop Iter | 50 |
Factor | 2 |
Near Plane | 2.0 |
Far Plane | 6.0 |
Height | 800 / factor |
Width | 800 / factor |
Downscale | 2 |
lr | 5e-4 |
lrsch_gamma | 0.1 |
Pos Enc Dim | 10 |
Dir Enc Dim | 4 |
Num Samples | 64 |
Num Samples Fine | 128 |
Net Dim | 256 |
Net Depth | 8 |
Inp Feat | 2*(num_channels*pos_enc_dim) + num_channels |
Dir Feat | 2*(num_channels*dir_enc_dim) + num_channels |
[1] Computer Graphics and Deep Learning with NeRF using TensorFlow and Keras []
[2] 3D volumetric rendering with NeRF []
[3] Nerf Official Colab Notebook []