Unable to load semantic annotation for Scene

[edwardjjj]

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: v0.3.0

Habitat-Sim: v0.3.0

🐛 Bug

I'm creating a sim of scene "00680-YmWinf3mhb5" but get the following warning message:

ResourceManager.cpp(369)::loadSemanticSceneDescriptor : SSD File Naming Issue! 
Neither SceneInstanceAttributes-provided name : `../data/scene_datasets/hm3d/train/00680-YmWinf3mhb5/YmWinf3mhb5.basis.scn` 
nor constructed filename : `../data/scene_datasets/hm3d/train/00680-YmWinf3mhb5/info_semantic.json` exist on disk.

Consequently, am getting empty list when calling the sim.semantic_annotations()

Steps to Reproduce

Steps to reproduce the behavior:

sim config

type: Sim-v0
forward_step_size: 0.25
turn_angle: 30
create_renderer: false
requires_textures: true
auto_sleep: false
step_physics: true
concur_render: false
needs_markers: true
update_articulated_agent: true
scene: ../data/scene_datasets/hm3d/train/00680-YmWinf3mhb5/YmWinf3mhb5.basis.glb
scene_dataset: [./data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json](https://file+.vscode-resource.vscode-cdn.net/home/edward/projects/habitat/src/data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json)
additional_object_paths: []
seed: ${habitat.seed}
default_agent_id: 0
debug_render: false
debug_render_articulated_agent: false
kinematic_mode: false
should_setup_semantic_ids: true
debug_render_goal: true
robot_joint_start_noise: 0.0
ctrl_freq: 120.0
ac_freq_ratio: 4
load_objs: true
hold_thresh: 0.15
grasp_impulse: 10000.0
agents:
  main_agent:
    height: 0.88
    radius: 0.18
    max_climb: 0.2
    max_slope: 45.0
    grasp_managers: 1
    sim_sensors:
      rgb_sensor:
        type: HabitatSimRGBSensor
        height: 480
        width: 640
        position:
        - 0.0
        - 0.88
        - 0.0
        orientation:
        - 0.0
        - 0.0
        - 0.0
        hfov: 79
        sensor_subtype: PINHOLE
        noise_model: None
        noise_model_kwargs: {}
      depth_sensor:
        type: HabitatSimDepthSensor
        height: 480
        width: 640
        position:
        - 0.0
        - 0.88
        - 0.0
        orientation:
        - 0.0
        - 0.0
        - 0.0
        hfov: 79
        sensor_subtype: PINHOLE
        noise_model: None
        noise_model_kwargs: {}
        min_depth: 0.5
        max_depth: 5.0
        normalize_depth: true
      semantic_sensor:
        type: HabitatSimSemanticSensor
        height: 480
        width: 640
        position:
        - 0.0
        - 0.88
        - 0.0
        orientation:
        - 0.0
        - 0.0
        - 0.0
        hfov: 79
        sensor_subtype: PINHOLE
        noise_model: None
        noise_model_kwargs: {}
    is_set_start_state: false
    start_position:
    - 0.0
    - 0.0
    - 0.0
    start_rotation:
    - 0.0
    - 0.0
    - 0.0
    - 1.0
    joint_start_noise: 0.1
    joint_start_override: null
    articulated_agent_urdf: null
    articulated_agent_type: null
    ik_arm_urdf: null
    motion_data_path: ''
agents_order:
- main_agent
default_agent_navmesh: true
navmesh_include_static_objects: true
habitat_sim_v0:
  gpu_device_id: 0
  gpu_gpu: false
  allow_sliding: false
  frustum_culling: true
  enable_physics: false
  enable_hbao: false
  physics_config_file: [./data/default.physics_config.json](https://file+.vscode-resource.vscode-cdn.net/home/edward/projects/habitat/src/data/default.physics_config.json)
  leave_context_with_background_renderer: false
  enable_gfx_replay_save: false
ep_info: null
object_ids_start: 100
renderer:
  enable_batch_renderer: false
  composite_files: null
  classic_replay_renderer: false

Build sim

from habitat.sims import make_sim
sim = make_sim(id_sim=config.type, config=config)

Expected behavior

I expect the simulation to properly load the semantic informations

[edwardjjj]

An Update to the cause.

It seems that the scene_dataset field in the simulator config will be overwrite by the
episode's scene_dataset_config field.
When I run debugger and step into the creation of env, I find the scene_dataset_config field has the wrong prefix. scene_dataset_config should have the same prefix as scene_id. Is there anyway to change it?

[aclegg3]

Yes, this is true. The episodes will define a dataset config based on the assets used to generate the episode. You could overwrite the dataset field within the episode configs (using python JSON library). Otherwise you're best off generating new episodes for the dataset you are using to be sure that the agent and object placements referenced within are valid for your assets.

[edwardjjj]

Yes, this is true. The episodes will define a dataset config based on the assets used to generate the episode. You could overwrite the dataset field within the episode configs (using python JSON library). Otherwise you're best off generating new episodes for the dataset you are using to be sure that the agent and object placements referenced within are valid for your assets.

Thank you for pointing this out. However, this is rather inconvenient. Is it possible to request a feature that can override this behavior using the config file?

For example, a field in the simulator config decides whether to use the path in the episode data or not.

[edwardjjj]

I have a follow up question about semantic annotations. In this tutorial I see that we can print hierarchical semantic information about the levels, regions and objects.

However in HM3D dataset I'm using, I only get object level information. when accessing semantic_scene.levels, I get [] back. It might be because this error message:

[Metadata] AttributesManagerBase.h(398)::buildAttrSrcPathsFromJSONAndLoad : <Scene Instance> : No Glob path result found for `../data/scene_datasets/hm3d/train/00680-YmWinf3mhb5/*.basis.scene_instance.json` so unable to load templates from that path.

So where can I find these basis.scene_instances.json files?

[edwardjjj]

I found the answer to the follow up question at Habitat-Sim#2346