This project was published in IEEE Robotics and Automation Letters RA-L
[ArXiv Paper]
For a quick overview please visit our website project or watch our video demo depicted below. For further questions, please contact us to [email protected]
-
29/12/2023: Add direct_floor_plan_estimation.v2 to wrapper vo-scale recover for different dataloaders versions e.g 360-MLC, MVL-Challenge.
-
28/12/2023: Add setup.py to the project and rename src folder to direct_floor_plan_estimation for better import-like format.
-
18/05/2023: Added
pcl.pylfiles to GoogleDrive. Fix issue #2 -
19/07/2022: Code and pilot scenes released. HERE
-
29/01/2023: Open access to MP3D-FPE dataset HERE.
This is the implementation of the 360-DFPE, sequential floor plan estimation using only monocular 360-images as input.
Using this REPO, you can:
- Excecute 360-DFPE for single and multiple room scenes in a sequential and non-sequential manner.
- Register multiple 360-layouts with estimated camera poses, assuming an unknown visual odometry scale and missed camera height. For convenience, this registration can be executed isolated from the floor plan estimation pipeline. see Computing metadata
- Evaluate corner and room metrics for floor plan estimation.
- python 3.7.7
- vispy 0.5.3
- numpy 1.18.5
- opencv-python 3.4.3.18
- pandas 1.0.5
git clone [email protected]:EnriqueSolarte/direct_360_FPE.git
cd direct_360_FPE
pip install -r requirements.txt
pip install .The dataset used in this project is our own collected MP3D-FPE dataset, which simulates a handled-camera walking through different rooms scenes rendered by MINOS simulator with MP3D real-world data. Upon this collected data, we annotate floor plan labels, estimated camera poses uisng OpenVSLAM[4], and estimated layout for each key-frame by HorizonNet[3]. Our dataset offers 360-rgb-images, depth maps, ground truth and estimated camera poses, floor plan labels, and estimated layouts.
MP3D-FPE is a collected dataset for the task of floor plan estimation. This dataset is stricly an extension of MP3D. Therefore, the data policies and copyrights of this dataset are under the the terms and condition described by Matterport.
For convenience, we have prepared a light-set of scenes which can be downloaded by running download_mp3d_fpe.py -h.
All hyperameter settings are stored in a yaml file at ./config/config.yaml. Addtitionally, for practical purposes, we define a data manager class DataManager, which handles the data in the MP3D-FPE dataset, i.e., ground truth information, estimated poses, rgb images, etc. The following is a typical initialization of our system:
from config import read_config
from data_manager import DataManager
from direct_floor_plan_estimation import DirectFloorPlanEstimation
config_file = "./config/config.yaml"
cfg = read_config(config_file=config_file)
dt = DataManager(cfg)
fpe = DirectFloorPlanEstimation(dt)
...
fpe.scale_recover.esimate_vo_scale()
...For executing 360-DFPE, we have created three main scripts, i.e., main_eval_scene.py, main_eval_list_scenes.py, main_eval_non_seq_approach.py.
python main_eval_scene.py --scene_name 1LXtFkjw3qL_0 --results ./test/python main_eval_scenes_list.py --scene_list ./data/scene_list_pilot.txt --results ./test/python main_eval_non_seq_approach.py --scene_list ./data/scene_list_pilot.txt --results ./test/Note that our formulation rely on monocular estimated camera poses, therefore the real scale of the odometry is missed. Additionally, since every layout geometry is estimated from a monocular inference of HorizonNet[3], the layout scale is also missed. For these reasons, we additionally implemented main_compute_metadata.py, which will compute these missed scales along with some additional information such as ground truth rooms, labels, and room-corners, and the missed scales need for 360-layout registration. For convenience, this script runs isolated from the floor plan estimation pipeline.
python main_compute_metadata.py --scene_list ./data/scene_list_pilot.txt --results ./test/
- Thanks to professor Wei-Chen Chiu for his unvaluable advises in this project.
- The credits of this repo are shared with Yueh-Cheng Liu.
Please cite our paper for any purpose of usage.
@ARTICLE{9772341,
author={Solarte, Bolivar and Liu, Yueh-Cheng and Wu, Chin-Hsuan and Tsai, Yi-Hsuan and Sun, Min},
journal={IEEE Robotics and Automation Letters},
title={360-DFPE: Leveraging Monocular 360-Layouts for Direct Floor Plan Estimation},
year={2022},
volume={7},
number={3},
pages={6503-6510},
doi={10.1109/LRA.2022.3173730}}