Visualizations of Citywide LiDAR Dataset Using Open3D

Table of Contents

1. Contributors
2. Project Overview
3. User Stories
4. Minimum Viable Products
   • User Interface and Workflow Design
   • Sample Results
   • Limitations
   • Useful Python Scripts
5. Getting Started
   • Prerequisites
   • Installation
6. Future Works

Contributors

Peinan Wang (LinkedIn): Backend Development; Point Cloud Visualization

Yuhao Hua (LinkedIn): Backend Development; Unit Testing

Luocheng Zhu (LinkedIn): Frontend Development

Project Overview

This is a team project for Northeastern University's CS 5500 - Foundations of Software Engineering.

Many professionals, such as urban planners, civil engineers, and environmental consultants, use geospatial information (e.g., building locations, terrain, vegetation, etc.) to make decisions. It is not always easy to obtain such information directly from online resources like Google Maps, especially when the site of interest is in a rural area or suburban. As a result, LiDAR (Light Detection and Ranging) technology has been widely used in the industry for geospatial data collection. In this project, we will build a web application to visualize and analyze LiDAR point cloud data. After uploading the point cloud data, users will be able to:

• See the 2D visualization of the point cloud data;
• See the 3D visualization of the area in an interactive mode;
• Check coverage and distribution of different features.

Please visit our Project Blog for the demo video and more details regarding the project's progress.

(back to top)

User Stories

As an urban planner, I want to have a 3D view of the city, including features like roads, parks, buildings, etc. So I can make better decisions on city planning. After uploading the Lidar point cloud, a 3D rendering will be generated in the user interface. It supports basic functionalities like click and drag, zoom-in and zoom-out, so I can use it to show my design idea.

If the point cloud data that I have are already classified, each class will be colored differently. My main focus is to tell:

• How much of the area is occupied by man-made structures/development (e.g. buildings, roads, bridges)
• Where are all the vegetation features located (e.g. grass, shrubs, trees)
• A clear display of the road systems
• A clear display of the powerline infrastructure (e.g. power tower, distribution line). This is an important factor to consider when I make urban development plans.

As an architect, I want to access various kinds of information when I crop a certain area of land. The information includes building coverage ratio, residential density, and afforested area. With the help of terrestrial equipment, I can also have a view of infrastructure and population, so that I can make a more elaborate analysis of site research and present more convincing data for my project.

As a civil engineer, I need to obtain site information such as ground type and topographic features, which can help me make decisions in road planning and location selection. In the map, I need to know the distribution of existing buildings, roads and powerlines. Also, the locations of farmlands, rivers and forests are important to me as well. It will be great if all the information is displayed in 3D so that I can see the terrain clearly and observe more details of the site.

(back to top)

Minimum Viable Products

We have developed the view.py file that supports all key features (i.e., 2D Visualization, 3D Visualization, feature analysis). This serves as the Proof of Concept for the proposed web application. Following the set-up instructions in Getting Started, we can run view.py locally and see the visualization results, subject to limitations.

More future works are needed for view.py to be a fully functional website.

(back to top)

User Interface and Workflow Design

1. Select a pre-classified LAS file

2. Click the "Submit" button to submit the file

3. Once a file is submitted successfully, a message will be displayed; Step 4) - 6) will be disabled until the file is uploaded and saved successfully.

4. Click the "Visualize" button to start the visualization process. Point cloud data information will be displayed in the table. 2D image of the site will be displayed.

5. Click the "View 3D Rendering" button to see the visualization rendering. Users will be directed to localhost:8888 to see the open3d webrtc window.

6. Click the "Stop Visualization" button to terminate the running visualization scripts.

Step 4) - 6) will be a "loop". We can click the "Visualize" button ONLY when there is NO visualization script running. Refreshing the web page will terminate the visualization script and clear the outputs.

(back to top)

Sample Results

We used the LiDAR dataset of a neighborhood in Surrey, BC (surveyed in 2013) as our sample data. The sample point cloud data can be found at Bulk Data - City of Surrey. We have uploaded this sample data on the website (hosted locally) and produced the following results. For more details, please check out the Demo Video in our Project Blog

(back to top)

Limitations

1. Currently this tool can only process LAS files, and the point cloud data need to be pre-classified. For the best user experience, the file size should be under 250 MB.
2. Web visualization function is not supported on ARM devices (e.g. M1/M2 Macbook). (Reference)
3. We faced some technical difficulties when we tried to deploy the website (the Dockerfile in this repo is still incomplete):
   • Open3D is not built properly in Dockerfile. Installing Open3D requires extra dependencies in Dockerfile. (Reference)
   • Currently functions in the LAS_Processor.py file do not work on a headless server. We followed the instruction in Open3d 0.16.0 documentation but we were not able to reproduce the headless rendering function. (Reference)
   • Web visualization (webrtc_server) may not work for external IPs. (It is supposed to work but we have not tested it yet). (Reference)

(back to top)

Useful Python Scripts

Since our web application is still a Proof of Concept pending deployment, one can only use our product on local devices with all dependencies and GPU graphic capacity. As a result, for research or demonstration purposes, it makes sense to just run some simple Python scripts. The data_processing folder includes Python programs to visualize a LAS file(visualize_classification.py) and to merge multiple LAS files (mergeLAS.py)

City of Vancouver provided the LiDAR dataset of the entire city at its Open Data Portal. We have written python scripts (in the LAS_downloading folder). Here are some locally-rendered results:

(back to top)

Getting Started

Prerequisite

We are using Open3D 0.16.0, which only supports Python 3.10 and above. This project is developed in a Python 3.10.8 virtual environment. Since Open3D is the only limiting factor for the minimum environment requirements, we recommend checking Open3D's official documentation for more information. You may need to install some additional dependencies for Open3D to work properly.

Installation

All Python modules required for this project are included in the requirements.txt. We recommend running the following command in a virtual environment.

  pip install -r requirements.txt

Future Works

The following future works need to be done. This project has been forked. Continuous contributions and improvements will be made in 2023.

Frontend Features

• Add a progress bar for the file-uploading process
• Add a progress bar for the visualization process
• Add a log-in function to support user registration

Website Deployment

• Solve the headless rendering issue of Open3D
• Write a Dockerfile that can set up the Open3D environment properly
• Deploy the website on a headless server
• Ensure the Web Visualizer works

Backend

• Improve the rendering speed (e.g. ignore some points)

Extended Functionalities

• Support rendering multiple files
• Support LAS file classification (Machine Learning model needed)
• Support object selection
• Add filters for selecting certain feature types

(back to top)