Alternate camera platforms

[jamesd256]

Hi,

Congratulations on a spectacular project, very inspiring.

What are your thoughts on supporting other camera platforms. There seem to be a few other solutions with similar spec to Basler for as little as one quarter the price.

I am very poor, but would like to be able to try out your amazing software. I am very willing to take on the work. just trying to get a sense of how much effort you think it would be.

I hope this is the right channel to reach you to discuss such things.

Thanks & regards,

James

[BZandi]

Thanks! Happy to hear that you find our project of value.

Generally, we use an abstract camera class camera.h, which is used in the singleCamera.cpp and stereoCamera.cpp file to retrieve the images. Additional camera platforms can be easly integrated by preparing the files in the device folder (see PupilEXT/src/devices).

Note that consumer webcams are not suited for the program structure in PupilEXT as we use an event-handler to grab the images from the camera (see singleCameraImageEventHandler.cpp). There are also additional features that can not be used with webcams, such as stereo camera recording because we applied a hardware trigger to synchronise the two cameras, which is impossible with webcams. Additionally, with webcams, there some numerous issues that need to be solved for accurate pupil measurements. Thus, it is recommended that the alternative camera brands offer machine vision cameras, i.e. industrial cameras with a C++ camera library.

If you want to run PupilEXT with very low investment, you have three options:

a) Take a look at the Basler dart series. They are affordable (~120 Euro) and should work with PupilEXT (not tested), as they use the Pylon library, which we use in PupilEXT to interface the camera(s). Link: https://www.baslerweb.com/en/products/cameras/area-scan-cameras/dart/

b) If you want to go even cheaper, you need to use a webcam (< 100 Euro). As mentioned, it is hard to integrate webcams in PupilEXT for real-time measurements, as we use an event handler to distribute the image inside the functions of PupilEXT. An alternative would be to record the images with an external tool and load the images into PupilEXT for offline analysis. We provide demo images that you can already use for offline analysis and testing the software. For this, download the demo dataset in section 2.4 (see readme file) and check out the respective video tutorial for offline analysis in which we load externally acquired images into PupilEXT. Link: https://vimeo.com/564778587

c) We are working on a Python version (library) of PupilEXT, which we publish later this year. With the Python library, you will be able to integrate any camera that you like.

I move this thread to the discussion forum as this might be an interesting topic for other users.

Best regards
Babak

[jamesd256]

That is very helpful, thank you Babak!

If I understand you correctly, because the lower cost Basler cameras are supported by Pylon, in theory it may be possible to achieve real-time measurement using these. By extension, if this proves the case, I assume you would not expect to have any issues achieving stereo imaging.

This sounds like the perfect route to try. If I do come up with any findings this way, I will share them here.

Forgive me if I've missed anything, but I would like to understand what performance benefit you might expect from a stereo setup over the single camera.

Good luck with the library version! Will you be licensing it as open source like the PupilExt project?

Thanks again & regards,

James

[BZandi]

Yes, the Python library of PupilEXT will also be open-source.

Although we did not test it, I am confident that it will work with the dart series, as all the USB3 cameras from Basler share the same commands for retrieving the images or adjusting the camera parameters through the Pylon library.

The advantage of stereo vision is that you can measure the pupil diameter directly in millimetres rather than pixel values. Stereo vision, however, comes with an increased effort as you need to purchase two cameras, and an additional microcontroller is necessary to synch the cameras.

Overall, you can go with a single camera, as we provide the option to measure a circular reference object of known size to calculate the pupil diameter in mm from the pixel values manually. Additionally, it is easier as you only need to connect the camera with your computer. Keep in mind that it is better to buy the monochrome version of the camera rather than the colour version.

Have a look at our paper in which we discuss the PupilEXT platform in detail:
https://www.frontiersin.org/articles/10.3389/fnins.2021.676220/full

[jamesd256]

Thank you once again. The explanation of the benefits of stereo makes perfect sense.

[BZandi]

A Python version of PupilEXT, called PyPupilEXT is now available. Note that the lib is currently in the experimental stage.

Link to the GitHub repo of PyPupilEXT:
https://github.com/openPupil/PyPupilEXT

[slee-101]

Hi,

This is incredible work and of high value to the research community!

Do you think there is any benefit in using frame grabbers? I don't see any from basler for USB 3.0 but they do (or did) for GigE PoE (https://www.graftek.biz/products/basler-adlink-gie74-pci-express-gige-vision-poe-4x-pcie-framegrabber-4-direct-camera-inputs). NI also has some framegrabbers for GigE.

[BZandi]

Thanks!

I would say that It depends on your motherboard type, the number of connected cameras, and how many additional devices (external hard drives, microcontroller) are connected to your PC via USB 3.0.

However, for our current stereo camera setup, we use a 4 Port USB 3.0 PCIe interface card, as our USB 3.0 Ports of the motherboard are used for other devices and we wanted to separate the camera connection.

Here is a link to the USB 3.0 PCIe card from Basler, which we currently use:
https://www.baslerweb.com/en/products/vision-components/pc-cards/usb-3-0-interface-card-pcie-fresco-fl1100-4hc-x4-4ports/

There are also two channel cards:
https://www.baslerweb.com/en/products/vision-components/pc-cards/

[slee-101]

Thank you for the information and thoughts! We will give this a try and share anything that might be useful to the community.

[berginator]

Hi,
I would also like to begin by thanking you for your work for the scientific community!
I am interested in trying out the Basler dart series. Do you absolutely recommend the 60fps mono version, or do you think one could have ok results with a lower frame rate?

[BZandi]

Thanks!

There is no general answer to this question, as it depends on the conditions of the pupil experiment. For example, if only the steady-state diameter is of interest, then 30 fps could be sufficient, as smoothing is done in the pre-processing of the pupil data anyway. However, if the short-term pupil light reflex or transient changes of the pupil in response to cognitive workloads are of interest, then 60 fps could make sense. However, the maximum frame rate can also be adjusted in PupilEXT to record, for example, at 30 fps with a 60 fps camera. Therefore, having the option to record at higher framerates is also nice if you see in your experiments that you need a higher acquiring speed.

In our setup, we use a 120 fps camera but we record at 30 fps most of the time, but I appreciate having the option to record at higher frame rates if I need to. Therefore, I would personally go for the 60 fps camera, but as I said, that depends on your experimental conditions. Attention should also be paid to the resolution of the camera, which has a great impact on the measurement accuracy.

Note that we never tested PupilEXT with the dart series, but I am from a theoretical point of view confident that it works, as we use Basler's Pylon library with the Pylon::CBaslerUniversalInstantCamera camera class, which should, according to the Pylon documentation, operate with all USB camera devices from Basler.

https://github.com/openPupil/Open-PupilEXT/blob/main/src/devices/singleCamera.h