[Bug] Recent change in local_visualizer.py causes error on Windows

[MatthiasHinz]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: win32
Python: 3.9.16 | packaged by conda-forge | (main, Feb 1 2023, 21:28:38) [MSC v.1929 64 bit (AMD64)]
CUDA available: True
numpy_random_seed: 2147483648
GPU 0: NVIDIA GeForce RTX 3080 Ti
CUDA_HOME: C:\ProgramData\Anaconda3\envs\mmdetection
NVCC: Cuda compilation tools, release 11.6, V11.6.124
GCC: n/a
PyTorch: 1.12.0
PyTorch compiling details: PyTorch built with:

• C++ Version: 199711
• MSVC 192829337
• Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
• OpenMP 2019
• LAPACK is enabled (usually provided by MKL)
• CPU capability usage: AVX2
• CUDA Runtime 11.6
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.3.2 (built against CUDA 11.5)
• Magma 2.5.4
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.13.0
OpenCV: 4.7.0
MMEngine: 0.8.4
MMDetection: 3.0.0
MMDetection3D: 1.3.0+b84ba5a
spconv2.0: True

Reproduces the problem - code sample

Reproduces the problem - command or script

python mmdetection3d/tools/misc/browse_dataset.py .\configs_base_\datasets\kitti-3d-3class.py --task lidar_det --output-dir .\data\kitti\ --show-interval 30

Reproduces the problem - error message

Traceback (most recent call last):
File "E:\mmdetection3d\tools\misc\browse_dataset.py", line 152, in
main()
File "E:\mmdetection3d\tools\misc\browse_dataset.py", line 138, in main
visualizer.add_datasample(
File "C:\ProgramData\Anaconda3\envs\mmstone\lib\site-packages\mmengine\dist\utils.py", line 401, in wrapper
return func(*args, **kwargs)
File "E:\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 1082, in add_datasample
self.show(
File "C:\ProgramData\Anaconda3\envs\mmstone\lib\site-packages\mmengine\dist\utils.py", line 401, in wrapper
return func(*args, **kwargs)
File "E:\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 868, in show
self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501
AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

Additional information

Hello,

a recent change in the file local_visualizer.py requires that the environment variable DISPLAY exists in order to initialize the Open3d visualizer backend. Since Windows desktops do usually not set a DISPLAY variable, calls to the Det3DLocalVisualizer e.g. with the browse_dataset script, will terminate with the fatal exception shown above.

See: https://github.com/open-mmlab/mmdetection3d/blame/5c0613be29bd2e51771ec5e046d89ba3089887c7/mmdet3d/visualization/local_visualizer.py#L180

The browse_dataset will work normally again after either removing the if-condition or setting up a DISPLAY variable with any value. However, I think it would be good to rework the code section for system compatibility.

[giangdip2410]

I have the same bug in Window platform. Do you know how to fix it? Thank you,

[sunjiahao1999]

Thanks for your reminder, we'll re-test on windows and fix this!

[dtz6]

I got the same bug on Windows platform.

[dtz6]

Now I got the same bug on Linux:
Traceback (most recent call last):
File "projects/BEVFusion/demo/multi_modality_demo.py", line 78, in
main(args)
File "projects/BEVFusion/demo/multi_modality_demo.py", line 64, in main
visualizer.add_datasample(
File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dist/utils.py", line 419, in wrapper
return func(*args, **kwargs)
File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet3d/visualization/local_visualizer.py", line 1082, in add_datasample
self.show(
File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dist/utils.py", line 419, in wrapper
return func(*args, **kwargs)
File "/data/dengtianze/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet3d/visualization/local_visualizer.py", line 868, in show
self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501
AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

[lypan1208]

I've got the same bug, does anyone know how to fix this? Thanks!

[MatthiasHinz]

I've got the same bug, does anyone know how to fix this? Thanks!

Did you try out one of the workarounds that I suggested above?

For now, you could either try setting up an environment variable named DISPLAY or remove the if-clause in the code chunk that I linked. In my case, the error occurred because the following lines 181 and 182 were not executed if this variable is not present.

Be aware that setting up a DISPLAY environment variable might have undesired side effects with other programs that try to read it, so it's really just a workaround.

[lypan1208]

I've got the same bug, does anyone know how to fix this? Thanks!

Did you try out one of the workarounds that I suggested above?

For now, you could either try setting up an environment variable named DISPLAY or remove the if-clause in the code chunk that I linked. In my case, the error occurred because the following lines 181 and 182 were not executed if this variable is not present.

Be aware that setting up a DISPLAY environment variable might have undesired side effects with other programs that try to read it, so it's really just a workaround.

As you said, remove the if-clause is working! Appreciate for your help, Thank hou!

[zyu]

File "demo/pcd_demo.py", line 90, in
main()
File "demo/pcd_demo.py", line 80, in main
inferencer(**call_args)
File "e:\mm\mmdetection3d\mmdet3d\apis\inferencers\base_3d_inferencer.py", line 213, in call
visualization = self.visualize(ori_inputs, preds,
File "e:\mm\mmdetection3d\mmdet3d\apis\inferencers\lidar_det3d_inferencer.py", line 190, in visualize
self.visualizer.add_datasample(
File "C:\ProgramData\miniconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 418, in wrapper
return func(*args, **kwargs)
File "e:\mm\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 1082, in add_datasample
self.show(
File "C:\ProgramData\miniconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 418, in wrapper
return func(*args, **kwargs)
File "e:\mm\mmdetection3d\mmdet3d\visualization\local_visualizer.py", line 868, in show
self.view_control.convert_to_pinhole_camera_parameters() # noqa: E501
AttributeError: 'Det3DLocalVisualizer' object has no attribute 'view_control'

[z-mahmud22]

Thank you @MatthiasHinz for mentioning the issue and suggesting a workaround.

I followed your suggestion by commenting out line 180.
However, I was having this error:

TypeError: paint_uniform_color(): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.cuda.pybind.geometry.LineSet, color: numpy.ndarray[numpy.float64[3, 1]]) -> open3d.cuda.pybind.geometry.LineSet

Invoked with: LineSet with 12 lines., 0.0

Additionally, I had to make some changes to the local_visualizer.py file following this PR #2729 to get rid of that error. Now I can visualize the point cloud with predicted 3D bounding boxes. This is my testing environment:

OS: Windows 11
GPU: NVIDIA GeForce RTX 4090
CUDA: 11.8
Python: 3.8
Pytorch: 2.1.2
TorchVision: 0.16.2
OpenCV: 4.9.0.80
Open3D: 0.18.0
MMEngine: 0.10.2
MMDetection: 3.3.0
MMDetection3D: 1.4.0 (Built from source)

[GODBENGAY]

Thank you @MatthiasHinz for mentioning the issue and suggesting a workaround.

I followed your suggestion by commenting out line 180. However, I was having this error:

TypeError: paint_uniform_color(): incompatible function arguments. The following argument types are supported:
    1. (self: open3d.cuda.pybind.geometry.LineSet, color: numpy.ndarray[numpy.float64[3, 1]]) -> open3d.cuda.pybind.geometry.LineSet

Invoked with: LineSet with 12 lines., 0.0

Additionally, I had to make some changes to the file following this PR #2729 to get rid of that error. Now I can visualize the point cloud with predicted 3D bounding boxes. This is my testing environment:local_visualizer.py

OS: Windows 11
GPU: NVIDIA GeForce RTX 4090
CUDA: 11.8
Python: 3.8
Pytorch: 2.1.2
TorchVision: 0.16.2
OpenCV: 4.9.0.80
Open3D: 0.18.0
MMEngine: 0.10.2
MMDetection: 3.3.0
MMDetection3D: 1.4.0 (Built from source)

Can I See your local_visualizer.py？ I still can't run it

[hitbuyi]

I got the same bug on Windows platform.

[hitbuyi]

Thanks for your reminder, we'll re-test on windows and fix this!

this bug still exists in mmdet3D v1.4.0 which is released recently

[zkailinzhang]

same bug
mmcv 2.1.0
mmdet 3.3.0
mmdet3d 1.4.0
mmengine 0.10.4

[zkailinzhang]

import os
os.environ['DISPLAY'] = '1',,,可以解决