Add tanks and temples dataset loader (#653)

* Add files via upload

* Rename tanks_and_temples_loader.py to gtsfm/loader/tanks_and_temples_loader.py

* Add minimal Tanks and Temples Barn data files for unit tests

* Rename tests/data/Barn.json to tests/data/tanks_and_temples_barn/Barn.json

* Rename tests/data/Barn_trans.txt to tests/data/tanks_and_temples_barn/Barn_trans.txt

* Update tanks_and_temples_loader.py

* Rename tests/data/Barn_COLMAP_SfM.log to tests/data/tanks_and_temples_barn/Barn_COLMAP_SfM.log

* Create test_tanks_and_temples_loader.py

* Update test_tanks_and_temples_loader.py

* Update tanks_and_temples_loader.py

* Update tanks_and_temples_loader.py

* Update benchmark.yml

* Update execute_single_benchmark.sh

* Update download_single_benchmark.sh

* Update execute_single_benchmark.sh

* Update download_single_benchmark.sh

* Add correspondence generator for synthetic data

* run on CI

* run in CI

* run w/ lookahead of 4

* use 700 synthetic points

* Test vectorization

* Remove vectorized code

* flake8 fixes

* flake8 fixes

* style fix

* fix image order

* Fix config comment

* add unit test to make sure images are sorted as expected

* loransac 0.5 px, no 2-view BA

* measure 2 view errors w/ synthetic

* dont compute error for None two view geometry

* fix resolution error

* fix resolution error

* fix resolution error

* remove print statements

* fix resolution error

* remove print statements

* fix resolution to 1080

* Remove 2-view error computation from the synthetic corr generator

* fix shonan capitalization

* remove print statements from multiview opt

* Add SO(3) check

* Fix capitlization in keypt agg

* fix capitalization in keypoint agg base

* fix 2-view estimator capitalization

* Fix Sim(3) alignment loading in loader

* improve unit test

* flake8 fixes

* flake8 fixes

* Add option for unit testing to use only K of N images, e.g. 3 of 410 images in runner

* fix docstrings

* fix tanks and temples unit test

* make some parts of T & T loader optional for unit testing w/o large files

* flake8 fixes

* Clean up T & T unit tests

* flake8 cleanup

* back to 700 correspondences

* clean up test comments

* Increase mesh resolution, with alpha=0.1 in mesh reconstruction, instead of 0.5

* fix input type as str not Path (via cast) for T& T loader test

* add 3 tanks and temples images as test data

* save shonan input

* cast i1, i2 to int, as int64 not json serializable

* 200 3d landmarks

* run tanks & temples on wildcat

* Log other connected components

* fix docstring capitalization

* dont redownload T&T

* run with low Shonan sigma on CI

* turn off wildcat

* sample using poisson disk

* Update tanks_and_temples_loader.py

* poisson sampling

* add pose auc

* add pose AUC

* add test_mesh fn on T & T

* update T & T test paths

* flake8 fix

* transition away from deprecated micromamba command

* add image_filenames() method

* remove activate-environment arg to setup-micromamba

* fewer retrieval pairs and no_grad for netvald

* fix typo

* lookahead 3

* use logging instead of print

* run tanks and temples on self hosted

* run T&T on eagle

* remove rotation quaternion logging

* tune shonan sigma to 0.01

* decrease uncertainty on poses to 0.01

* use logger_utils instead of getLogger() since unregistered

* fix logger formatting

* shonan sigma 0.1

* Sample random 3d points. This sampling must occur only once, to avoid clusters from repeated sampling.

* fix typo

* fix typo

* use isinstance() instead of type()

* Style fixes

* astrovision account for 2 new return args, and style fixes

* Add metrics group / report for Retriever

* style fixes on retriever

* docstring fixes on open3d vis utils, and add fn to viz GT alongisde

* style fixes and use new scene data loader for colmap format data that supports bin

* flake8 fix

* Style fixes on images.py

* support bin and txt scene data, and read out point_cloud and rgb point values from colmap data

* Always save pose auc plots

* save pre-ba summary

* improve name of metric

* Remove custom shonan covariances

* use points instead of spheres for rendering speed

* remove stale print statement

* remove dataset download from self-hosted runner

* revert changes to self-hosted runner

* Fix formatting

* update data paths

* flake8 fixes

* fix duplicated imports

* add more visualization functionality

* fix typo

* python black reformat

* get intrinsics from EXIF

* add back CI path

* improve viz script

* run synthetic in CI

* update retriever to image_pairs_generator

* improve comments

* move algorithm outside of loader

* move algorithm outside of loader

* fix flake8

* fix flake8

* fix docs and missing arg

* add todo

* fix

* sequential retriever

* improve ValueError message

* update to not use netvlad in synthetic front end

* remove synthetic tanks and temples from the CI

* revert CI files

* clean up dead code

* remove unncessary function

* clean up config

* 5k pts

* reformat python black

---------

gtsfm/configs/synthetic_front_end.yaml

@@ -0,0 +1,93 @@
+# Synthetic front-end configuration specifically for the Tanks & Temples dataset.
+
+SceneOptimizer:
+  _target_: gtsfm.scene_optimizer.SceneOptimizer
+  save_gtsfm_data: True
+  save_two_view_correspondences_viz: False
+  save_3d_viz: True
+  pose_angular_error_thresh: 5 # degrees
+
+  image_pairs_generator:
+    _target_: gtsfm.retriever.image_pairs_generator.ImagePairsGenerator
+    global_descriptor:
+      _target_: gtsfm.frontend.cacher.global_descriptor_cacher.GlobalDescriptorCacher
+      global_descriptor_obj:
+        _target_: gtsfm.frontend.global_descriptor.netvlad_global_descriptor.NetVLADGlobalDescriptor
+
+    # NOTE: May not converge with SequentialRetriever only.
+    retriever:
+      _target_: gtsfm.retriever.joint_netvlad_sequential_retriever.JointNetVLADSequentialRetriever
+      num_matched: 2
+      min_score: 0.2
+      max_frame_lookahead: 3
+
+  correspondence_generator:
+    _target_: gtsfm.frontend.correspondence_generator.synthetic_correspondence_generator.SyntheticCorrespondenceGenerator
+    dataset_root: /usr/local/gtsfm-data/Tanks_and_Temples_Barn_410
+    scene_name: Barn
+
+  two_view_estimator:
+    _target_: gtsfm.two_view_estimator.TwoViewEstimator
+    bundle_adjust_2view: False
+    eval_threshold_px: 4 # in px
+    ba_reproj_error_thresholds: [0.5]
+    bundle_adjust_2view_maxiters: 100
+
+    verifier:
+      _target_: gtsfm.frontend.verifier.loransac.LoRansac
+      use_intrinsics_in_verification: True
+      estimation_threshold_px: 0.5 # for H/E/F estimators
+
+    triangulation_options:
+      _target_: gtsfm.data_association.point3d_initializer.TriangulationOptions
+      mode:
+        _target_: gtsfm.data_association.point3d_initializer.TriangulationSamplingMode
+        value: NO_RANSAC
+
+    inlier_support_processor:
+      _target_: gtsfm.two_view_estimator.InlierSupportProcessor
+      min_num_inliers_est_model: 15
+      min_inlier_ratio_est_model: 0.1
+
+  multiview_optimizer:
+    _target_: gtsfm.multi_view_optimizer.MultiViewOptimizer
+
+    # comment out to not run
+    view_graph_estimator:
+      _target_: gtsfm.view_graph_estimator.cycle_consistent_rotation_estimator.CycleConsistentRotationViewGraphEstimator
+      edge_error_aggregation_criterion: MEDIAN_EDGE_ERROR
+
+    rot_avg_module:
+      _target_: gtsfm.averaging.rotation.shonan.ShonanRotationAveraging
+      # Use a very low value.
+      two_view_rotation_sigma: 0.1
+
+    trans_avg_module:
+      _target_: gtsfm.averaging.translation.averaging_1dsfm.TranslationAveraging1DSFM
+      robust_measurement_noise: True
+      projection_sampling_method: SAMPLE_INPUT_MEASUREMENTS
+
+    data_association_module:
+      _target_: gtsfm.data_association.data_assoc.DataAssociation
+      min_track_len: 2
+      triangulation_options:
+        _target_: gtsfm.data_association.point3d_initializer.TriangulationOptions
+        reproj_error_threshold: 10
+        mode:
+          _target_: gtsfm.data_association.point3d_initializer.TriangulationSamplingMode
+          value: RANSAC_SAMPLE_UNIFORM
+        max_num_hypotheses: 100
+      save_track_patches_viz: False
+
+    bundle_adjustment_module:
+      _target_: gtsfm.bundle.bundle_adjustment.BundleAdjustmentOptimizer
+      reproj_error_thresholds: [10, 5, 3] # for (multistage) post-optimization filtering
+      robust_measurement_noise: True
+      shared_calib: False
+      cam_pose3_prior_noise_sigma: 0.01
+      calibration_prior_noise_sigma: 1e-5
+      measurement_noise_sigma: 1.0
+
+  # # comment out to not run
+  # dense_multiview_optimizer:
+  #   _target_: gtsfm.densify.mvs_patchmatchnet.MVSPatchmatchNet