run bugs

[JINAOLONG]

when i run
python FusionVisionV0.3.py --yolo_weight best.pt --fastsam_weight FastSAM-x.pt --confidence_threshold 0.7 --conf 0.4 --iou 0.5 --show_3dbbox
show that
Traceback (most recent call last):
File "D:\project-learning\FusionVision-main\FusionVisionV0.3.py", line 250, in
FusionVision(args)
File "D:\project-learning\FusionVision-main\FusionVisionV0.3.py", line 144, in FusionVision
[np.min(roi_points[:, 0]), np.min(roi_points[:, 1]), np.min(roi_points[:, 2])],
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\myproject\test_1\Lib\site-packages\numpy\core\fromnumeric.py", line 2953, in min
return _wrapreduction(a, np.minimum, 'min', axis, None, out,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\myproject\test_1\Lib\site-packages\numpy\core\fromnumeric.py", line 88, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: zero-size array to reduction operation minimum which has no identity

some time It can work.but the show window is blank，i dont konw why

[dagshub]

Join the discussion on DagsHub!

[safouaneelg]

Hello @JINAOLONG ,

Thank you for mentioning the problem.

The error you're encountering, ValueError: zero-size array to reduction operation minimum which has no identity occurs when applying a reduction operation (min or max) on an empty array. In the code, I think it's specifically the line 141 (that is responsible of 3D bounding box computation) that generates this error. This means that the array roi_points is empty, and thus operations like np.min() or np.max() on it will fail.

In order for me to help you, I need you to bring more context

1. are you using a pre-trained yolo model? if not what are you trying to detect?
2. Do you need to show the 3D bounding boxes? if it's not mandatory try running in the terminal the command without the argument --show_3dbbox.
3. If the 3D boxes are needed, try to train YOLO more epochs and on a higher number of images.
4. Can you also make sure that your objects are correctly detected within all the frames and show the YOLO and FastSAM detections in 2D views ? you can run the following command:

python FusionVisionV0.3.py --yolo_weight best.pt --fastsam_weight FastSAM-x.pt --show_yolo --show_mask --show_fastsam --confidence_threshold 0.7 --conf 0.4 --iou 0.5 --show_3dbbox

Best,
Safouane

[JINAOLONG]

1 I'm using a yolo model that I trained myself
2 I need 3D bounding boxes
3It works fine when running other code, as follows

[safouaneelg]

Thank you

I see that the objects thickness is rather small, it might be a limitation of the sensors

Could you add the following line code before the line 141 in FusionVisionV0.3.py and test again:

if roi_points.size == 0:
    print(f"Empty ROI points for bounding box: {box}")
    continue

you should have something like this:

Best,
Safouane

[xiexiao6]

Thank you for your reply.
The program works successfully, but there may be object recognition that is not as clear as in your readme file, after yolo recognizes the object correctly and fastsam splits it correctly, the final fusion does not look good，there is a problem of not recognizing all the objects and object recognition is incomplete as shown below
![XYFC_B0@917S$)}E%E1L$7K](https://github.com/safouaneelg/FusionVision/assets/113775558/ab8418d3-3461-4588-9494-66

1dea983c3a)

Looking forward to your reply
Best.

[xiexiao6]

ooops ，The last correctly recognized image was not successfully uploaded and is shown below

[safouaneelg]

Hello @xiexiao6, thank you for providing the screenshots.

I'm still a bit confused. So if I understand well, the program runs successfully up to the segmentation part. The moment you try the reconstruction you have that is where the error ValueError: zero-size array to reduction operation minimum which has no identity.

we have never encountered this error, however here are some debugging suggestions:

1- You can try to increase or decrease the depth_image max in the line isolated_depth = np.where((eroded_ann_mask > 0) & (depth_image < 1000), depth_image, np.nan)

2- you can print region of interest coordinates by adding the print line in the code as follow:

                    for box in bounding_boxes:
                        x_min, y_min, x_max, y_max = box
                        ############ Print the ROI
                        print(f"ROI: x_min={x_min}, y_min={y_min}, x_max={x_max}, y_max={y_max}")

3- you can also print the shape of the roi_points point as follow:

                        roi_points = np.asarray(denoised_pcd.points)
                        roi_points = roi_points[(roi_points[:, 0] >= x_min) & (roi_points[:, 0] <= x_max) &
                                                (roi_points[:, 1] >= y_min) & (roi_points[:, 1] <= y_max)]
                        # This line
                        print(f"ROI Points Shape: {roi_points.shape}")

4- you can also play on the eroded_ann_mask, this lines reduces a tiny bit the size of the mask to avoid reconstructing some background pixels. you can either modify the kernel size of the number of iterations or remove the eroding to see how the reconstruction is.

The goal would be to understand why roi_points is empty.

Hopping this might help.

Best,
Safouane