The training quality of Instruct-GS2GS is inferior to Instruct-Nrf2nerf

[cici19850]

Train using Instruct-GS2GS as follows:
First, execute the command: ns-train splatfacto --data data\farm-small --max-num-iterations 20000
Next, execute the command: ns-train igs2gs --data data\farm-small --load-dir outputs/farm-small/splatfacto/2024-10-24_180131/nerfstudio_models --pipeline.prompt "make it look like it just snowed" --pipeline.guidance-scale 12.5 --pipeline.image-guidance-scale 1.5 (Default training 7500 times)
The final effect is as follows:
https://github.com/user-attachments/assets/073f51c0-2534-4ed8-9e27-5caea571f0e2

Train using Instruct-Nrf2nerf as follows:
First, execute the command: ns-train nerfacto --data data\farm-small (Default training 30000 times)
Next, execute the command: ns-train in2n --data data\farm-small --load-dir outputs\farm-small\nerfacto\2024-11-14_115128\nerfstudio_models --pipeline.prompt "make it look like it just snowed" --pipeline.guidance-scale 7.5 --pipeline.image-guidance-scale 1.5 (Default training 15000 times)
The final effect is as follows:
https://github.com/user-attachments/assets/6ef1a59a-70b1-4b97-b690-d010822ec2ba

What is the reason why Instruct-GS2GS is less effective than Instruct-Nrf2nerf? Is there any way to solve it

[ScienceLi1125]

Hi, I met the same error. According to my inspection, the farm-small and campsite-small scenes in in2n dataset are OPENCV-FISHEYE type. But 3D Gaussian Splatting only supports pinhole camera models and cannot handle the distortion problem of fisheye cameras^[Use my own camera position and point cloud data #205]、^[Fisheye lens support? #199]. Therefore, unreconstructable artifacts appear in the reconstruction process, and the editing results are naturally unacceptable.

The above is my observation of the experimental results, I hope it can help you. However, if you have a good solution, I would be happy to ask you for advice.

[ScienceLi1125]

Btw, the instructions and results of my training scene are as follows:

1. ns-train nerfacto --data ../autodl-tmp/data/farm-small

farm-small0.mp4

2. ns-train splatfacto --data ../autodl-tmp/data/farm-small

farm-small.mp4