how do this instant-ngp deal with nerf_llff_data?

[lishuwei0424]

how do this instant-ngp deal with nerf_llff_data? thanks you

[ashawkey]

Hi, I wrote a script to convert the LLFF format to NeRF-compatible format, by generating transforms.json from poses_bounds.npy:
https://github.com/ashawkey/torch-ngp/blob/main/llff2nerf.py
However, it doesn't convert rays to the NDC space, but more like running colmap2nerf.py on the original images.
The results for the fern:

[kwea123]

How is the performance?

[ashawkey]

I have not tested the exact performance. Honestly, this may not be a good solution, since the script fails for the flower scene (the cameras are almost parallel, so the script cannot estimate the scene center?), and needs a manual fix.

[qingxu-thu]

I've checked llff2nerf.py in your project and found it corresponding to the colmap2nerf and llff's implementation. When I try to take it into the real scenes, it fails (but the scene seems to have a little large FOV..) with many floaters. Hope to contribute a bad case for you to analyze what happened in such scenes...

[kwea123]

With a different coordinate conversion, my ngp_pl can now train successfully on every llff scene (including flower that doesn't work here). However the result is worse than original NeRF, possibly because here we don't use NDC, but with the simplified code I think it's worth a try.

[AIBluefisher]

Is this a validation image or training image? I found that using the provided dataloader the training is normal while the validation psnr is very low.

[AIBluefisher]

Is this a validation image or training image? I found that using the provided dataloader the training is normal while the validation psnr is very low.

I used nerfacc as my pipeline. Not sure if npg_pl has the same issue. See a related issue from nerfacc here