create_graph.py fails for hm3dsem dataset

[elias-gvin]

Also, when I am trying to run "create_graph.py" script, I am getting the following error:

Have you seen such a problem previously? What could be the reason?

BTW, the command in the README.md seems incorrect.
Path to the dataset should be:
main.dataset_path=data/hm3dsem_walks/val/00824-Dd4bFSTQ8gi/ instead of main.dataset_path=hm3dsem_walks/val/00824-Dd4bFSTQ8gi/ like in README.md

(hovsg) ➜  HOV-SG git:(main) ✗ python application/create_graph.py main.dataset=hm3dsem main.dataset_path=data/hm3dsem_walks/val/00824-Dd4bFSTQ8gi/ main.save_path=data/scene_graphs/00824-Dd4bFSTQ8gi
/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/scipy/__init__.py:155: UserWarning: A NumPy version >=1.18.5 and <1.26.0 is required for this version of SciPy (detected version 1.26.4
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
[2024-07-22 17:39:49,622][root][INFO] - Loaded ViT-H-14 model config.
[2024-07-22 17:39:57,253][root][INFO] - Loading pretrained ViT-H-14 weights (checkpoints/laion2b_s32b_b79k.bin).
Creating RGB-D point cloud:   0%|                                                                                                                                                   | 0/226 [00:00<?, ?it/s]
Error executing job with overrides: ['main.dataset=hm3dsem', 'main.dataset_path=data/hm3dsem_walks/val/00824-Dd4bFSTQ8gi/', 'main.save_path=data/scene_graphs/00824-Dd4bFSTQ8gi']
Traceback (most recent call last):
  File "/home/ncdev/repos/HOV-SG/application/create_graph.py", line 39, in <module>
    main()
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/main.py", line 94, in decorated_main
    _run_hydra(
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
    raise ex
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/home/ncdev/repos/HOV-SG/application/create_graph.py", line 19, in main
    hovsg.create_feature_map() # create feature map
  File "/home/ncdev/repos/HOV-SG/hovsg/graph/graph.py", line 154, in create_feature_map
    rgb_image, depth_image, pose, _, depth_intrinsics = self.dataset[i]
  File "/home/ncdev/repos/HOV-SG/hovsg/dataloader/hm3dsem.py", line 52, in __getitem__
    pose = self._load_pose(pose_path)
  File "/home/ncdev/repos/HOV-SG/hovsg/dataloader/hm3dsem.py", line 123, in _load_pose
    transformation_matrix = np.array(values).reshape((4, 4))
ValueError: cannot reshape array of size 1 into shape (4,4)

[abwerby]

I just pushed a fix, it should work now.

[elias-gvin]

@abwerby thank you for quick fix! I don't encounter such problems right now.

However, when I am trying to generate scene graph, I repeatedly face crashes:

(hovsg) ➜  HOV-SG git:(main) ✗ python application/create_graph.py main.dataset=hm3dsem main.dataset_path=data/hm3dsem_walks/val/00824-Dd4bFSTQ8gi/ main.save_path=data/scene_graphs/00824-Dd4bFSTQ8gi
/home/ncdev/miniforge3/envs/hovsg/lib/python3.9/site-packages/scipy/__init__.py:155: UserWarning: A NumPy version >=1.18.5 and <1.26.0 is required for this version of SciPy (detected version 1.26.4
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
[2024-07-23 12:37:18,487][root][INFO] - Loaded ViT-H-14 model config.
[2024-07-23 12:37:26,890][root][INFO] - Loading pretrained ViT-H-14 weights (checkpoints/laion2b_s32b_b79k.bin).
Creating RGB-D point cloud: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 226/226 [00:26<00:00,  8.58it/s]
Extracting features: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 226/226 [20:37<00:00,  5.47s/it]
Merging 3d masks sequentially
  6%|██████████                                                                                                                                                        | 14/225 [25:23<19:20:49, 330.09s/it][1]    100314 killed     python application/create_graph.py main.dataset=hm3dsem  

Traces show that some processes were killed on my machine, due to Out of memory problem. Despite the machine having 62 gigs of RAM.

What are the memory requirements for this script to run properly?

[Tom-Huang]

The problem is related to #16 . We will improve the memory efficiency of the seq_merge in two weeks.