Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

box3d_overlap may cause “RuntimeError: CUDA error: an illegal memory access was encountered” in some cases #1805

Open
mxh1999 opened this issue May 31, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@mxh1999
Copy link

mxh1999 commented May 31, 2024

🐛 Bugs / Unexpected behaviors

When I use box3d_overlap on GPU, it may cause "RuntimeError: CUDA error: an illegal memory access was encountered" under certain inputs.

Instructions To Reproduce the Issue:

from pytorch3d.ops import box3d_overlap
import torch
boxes1 = torch.tensor([[ 0.0622, -0.0626,  0.9963],
        [-0.0565,  0.1136,  0.7310],
        [-0.6875,  0.4739,  1.2525],
        [-0.5689,  0.2977,  1.5178],
        [ 0.2553,  0.1737,  1.0667],
        [ 0.1367,  0.3498,  0.8015],
        [-0.4944,  0.7101,  1.3230],
        [-0.3757,  0.5339,  1.5883]]).reshape(1,8,3).cuda()
boxes2 = torch.tensor([[-0.1583,  0.5106,  1.8723],
        [-0.1583,  0.5106,  0.2837],
        [-0.1583,  0.2942,  0.2837],
        [-0.1583,  0.2942,  1.8723],
        [-0.5130,  0.5106,  1.8723],
        [-0.5130,  0.5106,  0.2837],
        [-0.5130,  0.2942,  0.2837],
        [-0.5130,  0.2942,  1.8723]]).reshape(1,8,3).cuda()

_, iou3d = box3d_overlap(boxes1, boxes2)
print(iou3d)

This code will cause the problem. But if I remove .cuda(), it works fine.

I try to visualize these two boxes using open3d and I don't see anything unusual.
hMDmhG8hHo

My Environment

python                    3.8.19               h955ad1f_0
pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0
pytorch3d                 0.7.2                    pypi_0
scipy                     1.10.1                   pypi_0
torchaudio                0.11.0               py38_cu113 
torchvision               0.12.0               py38_cu113
numpy                     1.24.3           py38hf6e8229_1  
numpy-base                1.24.3           py38h060ed82_1
@bottler bottler added the bug Something isn't working label Jun 14, 2024
@AndreasLH
Copy link

AndreasLH commented Jun 17, 2024

Running into the exact same issue here. using

  • torch version 2.1.0
  • pytorch3d version 0.7.6
  • cuda_11.8.r11.8

@manurare
Copy link

The problem comes from MAX_TRIS when allocating a new array here. If MAX_TRIS is changed from 100 to 200 then it works. I wonder if it can be dinamically allocated instead... The intersection shape can have arbitrary number of triangles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants