Feasibility of generating pointcloud data for export to something like Meshroom?

[InconsolableCellist]

It seems that the marching cubes algorithm doesn't do a great job at reconstructing meshes, and I was wondering if external tools could be leveraged for that purpose. Meshroom, for example, has a whole pipeline designed to create a point cloud and depth map from a Structure From Motion step. With instant-ngp, however, would it be possible to train the model and then export a point cloud, vertex color data, and a depth map? Perhaps then something like Meshroom's meshing node could do a good job reconstructing the geometry.

[InconsolableCellist]

I was looking into this to see if it'd be feasible, and I think maybe gen_vertices in marching_cubes.cu:268 shows how to index into the data the model creates to get raw density values. Perhaps it'd be possible to iterate over this density object, check against a threshold, and if it matches assign a vertex in an output data structure? I'm not sure what data format should be used, though; Meshroom seems to expect SfM data, which I think is concerned with camera positions.

[Echolink50]

Any news on this? I had the same thoughts when I discovered ngp isn't really made for creating a mesh. Is it possible to get a point cloud and vertex colors from instant ngp in a .ply to import and create a mesh in some other software?

[Tom94]

You can get the vertex positions of the marching cubes mesh as a numpy array via the python bindings, and you can get depth renderings through the built-in depth render mode.

Beyond that, my guess is unfortunately as good as yours -- I'm not familiar with Meshroom.

[BenEngelCovalence]

I used open3D to make a post processing script that cleans up some of the noise and re-meshes the clean point cloud. I import the transforms.json from colmap to get camera locations, then use open3D hidden_point_removal. Join all of those new meshes together, and you are pretty good. I also do some filtering to get just the largest mesh.

[Echolink50]

Any examples of the quality you were able to achieve with this technique? Also would.you be able to share the script? Thanks

[BenEngelCovalence]

Not sure if it will do what you are looking for, but here is what I did some quick work on a few days ago. Tuning the values at the top of the file helps a ton, also the better the initial mesh the better it will clean up. In here you can also do some things to simplify the mesh down however much you need.
https://drive.google.com/drive/folders/1WZncUbR-Bt524wBv6JFAiWzvBHxV_-Fg?usp=sharing

Note: this is a colored export, but since the mesh is more simplified it looks like trash, so you will need to come up with your own solution for making new UVs and projecting the images onto the new mesh.

[Echolink50]

Thank you my friend. I will look at it tomorrow when I get to my computer.

[benbeezy]

were you able to find any way of making better meshes and possibly texturing?

[Echolink50]

were you able to find any way of making better meshes and possibly texturing?
Not with Instant NGP. Seems everything points to using a different ai all together. nvdiffrec or some nerf ai seems to be the best bet for now. These are very new and I haven'[t even tried them yet so I am not even sure how well or how fast they work.