How to use GPU acceleration on Mac

[shiyuasuka]

I watched your demo video and it seems that only the 10-core chip of M1 Pro was used for processing. Is it possible to use the GPU of the M2 chip for acceleration?

[servin]

in theory if you are succeed doing the Core ML models you can have full advantage of any number of CPU, GPU and RAM allocated on your device because Core ML supports all the compute units available in your device: CPU, GPU and Apple's Neural Engine (NE). It's also possible for Core ML to run different portions of the model in different devices to maximize performance.

you can archive this by following this instructions

https://github.com/ggerganov/whisper.cpp#core-ml-support

my m1pro gets stuck in the process but you may have better luck

[SuperRegular]

Do you have any idea if the Core ML models can work on an older 2019 iMac with a 3.7 GHz 6-Core Intel Core i5? Or does it have to be M1 or M2?