FEATURE - support gpu acceleration for machine-learning

[martabal]

Immich v1.94.0 will introduce the ability to support GPU acceleration for machine-learning.

Nvidia - CUDA

Cuda acceleration shouldn't be too difficult to support, we should be able to reuse the custom scripts we created a few months ago.

Intel - OpenVINO

Openvino acceleration is not possible at the moment because it requires python 3.10. And with the current base image (ubuntu 23.10), python 3.10 is only available if you compile it. The best solution would be to wait until Ubuntu 24.04 LTS is released and use the Deadsnakes custom PPA to install python 3.10 or onnxruntime-openvino 1.17.

To do :

• Support cuda
• Support openvino

[martabal]

OpenVINO might support python 3.11 with the next onnxruntime-openvino release (1.17.0)

[martabal]

I tried openvino with python 3.10, but runned into issues with onnxruntime.

Edit: to be more precise : immich-app/immich#6869

[martabal]

Update:

• Immich v1.98.0 will no longer need the nightly version of onnxruntime
• I have a working version for Openvino but it must compile python 3.10 which is not a viable option for a docker mod (it is not realistic to wait 5 minutes for a docker to start)

[hydazz]

Update:

• Immich v1.98.0 will no longer need the nightly version of onnxruntime
• I have a working version for Openvino but it must compile python 3.10 which is not a viable option for a docker mod (it is not realistic to wait 5 minutes for a docker to start)

compile then package? or a deb?

[martabal]

compile then package? or a deb?

Not sure to correctly understand what you mean. Having a good python deb is hard, I would prefer to wait ubuntu 24.04 LTS.

[myanesp]

Hi there!

Any updates for this, for Intel iGPUs? Thanks :)

[martabal]

Hello!

Currently, OpenVINO is broken immich-app/immich#8226 (at least on my machines), so I can't really test it.

Once the dependency and the base image are updated, I'll implement it.

[martabal]

Done with #392