[Bug] Unable to run BEVFusion Demo #2802

Torreskai0722 · 2023-11-06T01:52:29Z

Prerequisite

I have searched Issues and Discussions but cannot get the expected help.
I have read the FAQ documentation but cannot get the expected help.
The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: linux
Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0]
CUDA available: True
numpy_random_seed: 2147483648
GPU 0: NVIDIA GeForce RTX 3080
CUDA_HOME: /usr/local/cuda-12
NVCC: Cuda compilation tools, release 12.1, V12.1.66
GCC: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
PyTorch: 2.1.0+cu121
PyTorch compiling details: PyTorch built with:

GCC 9.3
C++ Version: 201703
Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
OpenMP 201511 (a.k.a. OpenMP 4.5)
LAPACK is enabled (usually provided by MKL)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 12.1
NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
CuDNN 8.9.2
Magma 2.6.1
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-unused-private-field -Wno-aligned-allocation-unavailable -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.1.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.16.0+cu121
OpenCV: 4.8.1
MMEngine: 0.9.1
MMDetection: 3.2.0
MMDetection3D: 1.2.0+12b595c
spconv2.0: False

Reproduces the problem - code sample

(openmmlab) hydrapc@hydrapc:~/projects/DROI-BEV/mmdetection3d$ python projects/BEVFusion/demo/multi_modality_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin demo/data/nuscenes/ demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl projects/BEVFusion/configs/bevfusion_lidar-cam_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d.py bevfusion_converted.pth --cam-type all --score-thr 0.2 --show

Reproduces the problem - command or script

(openmmlab) hydrapc@hydrapc:~/projects/DROI-BEV/mmdetection3d$ python projects/BEVFusion/demo/multi_modality_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin demo/data/nuscenes/ demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl projects/BEVFusion/configs/bevfusion_lidar-cam_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d.py bevfusion_converted.pth --cam-type all --score-thr 0.2 --show

Reproduces the problem - error message

Traceback (most recent call last):
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/utils/misc.py", line 77, in import_modules_from_strings
    imported_tmp = import_module(imp)
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/__init__.py", line 1, in <module>
    from .bevfusion import BEVFusion
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/bevfusion.py", line 16, in <module>
    from .ops import Voxelization
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/__init__.py", line 1, in <module>
    from .bev_pool import bev_pool
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/__init__.py", line 1, in <module>
    from .bev_pool import bev_pool
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/bev_pool.py", line 3, in <module>
    from . import bev_pool_ext
ImportError: /home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/bev_pool_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 462, in fromfile
    import_modules_from_strings(**cfg_dict['custom_imports'])
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/utils/misc.py", line 84, in import_modules_from_strings
    raise ImportError(f'Failed to import {imp}')
ImportError: Failed to import projects.BEVFusion.bevfusion

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "projects/BEVFusion/demo/multi_modality_demo.py", line 78, in <module>
    main(args)
  File "projects/BEVFusion/demo/multi_modality_demo.py", line 42, in main
    model = init_model(args.config, args.checkpoint, device=args.device)
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/mmdet3d/apis/inference.py", line 59, in init_model
    config = Config.fromfile(config)
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 474, in fromfile
    raise ImportError(err_msg) from e
ImportError: Failed to import custom modules from {'imports': ['projects.BEVFusion.bevfusion'], 'allow_failed_imports': False}, the current sys.path is: 
    /home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/demo
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python38.zip
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/lib-dynload
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages
    /home/hydrapc/projects/DROI-BEV/mmdetection3d
    /tmp/tmp_f0g2gch
You should set `PYTHONPATH` to make `sys.path` include the directory which contains your custom module

Additional information

I follow the installation of mmdetection3d but cannot run the demo for BEVFusion.

The text was updated successfully, but these errors were encountered:

ranai-srivastav · 2023-11-08T01:01:17Z

I have the same issue as well.

Tried adding init.py to some folders and added the BEVFusion path to system path as well

Machine-NO-Learning · 2023-11-09T03:30:09Z

any solutions?

Machine-NO-Learning · 2023-11-09T05:43:50Z

there is one possible sol, config.py raise error is based on your import_module, but import_module raise error is not reason of wrong sys path, maybe you need install einops, so just pip install einops.

FraMarotta · 2023-11-22T13:54:55Z

I have the same problem, anyone has solved?

qiliang72 · 2023-11-23T09:20:05Z

Possible problem might be compatibility of gcc, g++, cuda version.
Maybe you can try to use GCC 9.3 which is your pytorch compiler version.
Also make sure to delete all your temporary files of previous builds. (This takes me too much time ><)

razgzy · 2023-11-28T11:39:15Z

I have the same problem

sss-stu-stupid · 2023-12-02T13:58:34Z

I have import error too, when from . import bev_pool_ext;
but it describes the problem as {ImportError: cannot import name 'bev_pool_ext' from partially initialized module 'projects.BEVFusion.bevfusion.ops.bev_pool' (most likely due to a circular import) (mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/init.py)}
This is the only difference in our logs.

BeMuCa · 2023-12-06T18:56:07Z

I have import error too, when from . import bev_pool_ext; but it describes the problem as {ImportError: cannot import name 'bev_pool_ext' from partially initialized module 'projects.BEVFusion.bevfusion.ops.bev_pool' (most likely due to a circular import) (mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/init.py)} This is the only difference in our logs.

This Solution from the OG BEVFusion Git solved it for me:

mit-han-lab/bevfusion#504

ranai-srivastav · 2023-12-10T09:18:31Z

@BeMuCa I don't have a mmdetection3d/mmdet3d/ops folder. This is my folder structure

~/software/mmdetection3d/mmdet3d$ ls
apis  configs  datasets  engine  evaluation  __init__.py  models  __pycache__  registry.py  structures  testing  utils  version.py  visualization

Torreskai0722 · 2023-12-12T06:42:31Z

My issue is resolved after I uninstall and reinstall numba:

pip uninstall numba
pip install numba

Besides, I also installed spconv which causes another cuda runtime error:

pip install spconv-cu120

seven-linglx · 2023-12-28T07:52:22Z

cd mmdetection3d
python projects/BEVFusion/setup.py develop

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] Unable to run BEVFusion Demo #2802

[Bug] Unable to run BEVFusion Demo #2802

Torreskai0722 commented Nov 6, 2023

ranai-srivastav commented Nov 8, 2023

Machine-NO-Learning commented Nov 9, 2023

Machine-NO-Learning commented Nov 9, 2023

FraMarotta commented Nov 22, 2023

qiliang72 commented Nov 23, 2023

razgzy commented Nov 28, 2023

sss-stu-stupid commented Dec 2, 2023 •

edited

Loading

BeMuCa commented Dec 6, 2023

ranai-srivastav commented Dec 10, 2023

Torreskai0722 commented Dec 12, 2023 •

edited

Loading

seven-linglx commented Dec 28, 2023

[Bug] Unable to run BEVFusion Demo #2802

[Bug] Unable to run BEVFusion Demo #2802

Comments

Torreskai0722 commented Nov 6, 2023

Prerequisite

Task

Branch

Environment

Reproduces the problem - code sample

Reproduces the problem - command or script

Reproduces the problem - error message

Additional information

ranai-srivastav commented Nov 8, 2023

Machine-NO-Learning commented Nov 9, 2023

Machine-NO-Learning commented Nov 9, 2023

FraMarotta commented Nov 22, 2023

qiliang72 commented Nov 23, 2023

razgzy commented Nov 28, 2023

sss-stu-stupid commented Dec 2, 2023 • edited Loading

BeMuCa commented Dec 6, 2023

ranai-srivastav commented Dec 10, 2023

Torreskai0722 commented Dec 12, 2023 • edited Loading

seven-linglx commented Dec 28, 2023

sss-stu-stupid commented Dec 2, 2023 •

edited

Loading

Torreskai0722 commented Dec 12, 2023 •

edited

Loading