Thank you very much for using MC-Calib. Indeed, such a poor reprojection error should not occur. It seems that only some images are "outliers," causing problems during the calibration, and I would assume it is because their angle is quite tricky and that the intrinsic calibration for strong peripheral aberration is not very good.
I would like to suggest a few modifications to your current calibration pipeline:

• Image Quantity: You probably need more synchronized images to calibrate your system. More importantly, you may need more diversity in the motion used to capture the images. Some pictures should be closer, others further away, and you require a variety of angles and translations. Ensure that the keypoints cover the entire sensor. This is even more critical when dealing with fisheye cameras. Using at least 30 to 50 images is recommended, especially if you are calibrating both the intrinsic and extrinsic parameters from scratch.
• Synchronization and Rolling Shutter: I am not sure about the exact setup you are currently using, but if your cameras do not have a global shutter, then you need to acquire the images when the entire setup is perfectly still. The same goes for the synchronization between the two cameras.
• Marker Size: From the examples you shared with me, it seems that some markers are not detected. What I would recommend is to use a smaller number of markers to make them bigger and more easily detectable.
• Better intrinsic calibration: You do not always need to have the boards detected on both cameras; feel free to have some images specifically for one of the two sensors to make sure you cover as much as possible.
• Configuration File: Would it be possible to also share your configuration file? Maybe I can find something to say on them too

@rameau-fr
thanks.

configuration file
calib_param_scene_1.zip

Your calibration configuration looks great, but I would probably increase the min_per_pts to about 0.4 just to be sure to avoid critical scenarios. Also, I noticed that you generate your calibration boards from another website; I am quite unsure if these boards are compatible with MC-Calib. You should generate the calibration boards directly from the toolbox as described in the tutorial.

@zuixiaosanlang We've fixed a bug related to fisheye calibration. It's now merged to master branch. Please try the calibration again - it must result in the lower reprojection error.

@BAILOOL
thanks,
i notice this before. and i think i correct it, because i get the same result. i forget to point this out.
my code:

i also change to your new code.but get the same result.

Hello,
Have you regenerated the board using MC-Calib directly?
The Calib.io boards are most likely not compatible with the toolbox.

I am not sure if it's relevant to this issue, but the intrinsics initialization process in OpenCV fisheye is not stable which may affect the following calibration process, at least that's my case. The bug seems to be fixed in 4.10.x, check issue. If upgrade OpenCV is not an option, a workaround is to give correct initial values, and set flags to use initial guess.

Thank you so much for this valuable feedback @BobbleLaw. We are testing MC-Calib with OpenCV==4.10.0.