Recommended HW for Frigate 0.16 onwards

[leccelecce]

Building on #14653 (reply in thread) and keen to get views of those using 0.15/looking to 0.16 onwards.

-Coral TPUs seem to be losing the relevance they did when Frigate was 0.10
-GPU acceleration is recommended and particularly as 8MP cameras keep developing becomes key
-GPU support for AI models looks to be becoming a requirement to use certain features in future, certainly if wanting to remain local

If we imagine an average Frigate user has 5x 4MP cameras, and wants to use Semantic search and future AI features like facial recognition - is it fair to say they should not be buying any hardware that can't support a modern GPU?

I have an Intel 10-700t iGPU and a Coral USB running in a Dell Optiplex Micro 5080. I have no option to put a GPU into this system, nor any of the rest of my lab (unless they're 75W cards, most of which are now the older generation from NVIDIA). Say I need a GTX 1650 minimum but preferably an RTX 30x0 for scaling and 8GB VRAM+. You're now outside most Intel NUCs, small/micro form factor machines like mine, and into either good sized rack machines or a desktop aren't you? Is that a reasonable future minimum to support most features?

Many of us also use HASS - which brings its own AI features. I wouldn't want to end up needing to build 2x GPU machines - is it feasible, if running on the same hardware, to share? This might be an unrealistic request e.g. assuming it is possible, it might require models to be swapped continually, harming latency on both applications.

A bit of a stream of consciousness for which I apologise - but would be good to understand maintainers views.

[NickM-27]

I have an Intel 10-700t iGPU and a Coral USB running in a Dell Optiplex Micro 5080

That would work fine for all features currently implemented in current or next release branches, except genai

[DamonLane]

@NickM-27 or others, could you answer the OP's more general question?

If we imagine an average Frigate user has 5x 4MP cameras, and wants to use Semantic search and future AI features like facial recognition - is it fair to say they should not be buying any hardware that can't support a modern GPU?

This is something I'm currently wondering about. So far I used my old laptop with an RK3399 processor to try out Frigate. I got a Coral and it works ok, but I don't have hardware acceleration working because I'm not using the -rk images, because I'm not using a -rockchip kernel, because of the other goals for the server and the fact that the operating systems that are recommended here for rockchip don't have images for this older chip.

I'm using Frigate mainly to detect birds. I moved from traditional heat-activated trail cameras to the optical activation that Frigate uses because birds have very little thermal mass to sense. Now I learn they are also hard to sense with optics. My laptop is plenty powerful for my other selfhosted tasks, but sending high res detect feed from one camera seems about as much as I'd want to stress the CPU. Once I get the server set up fully, I plan to join Frigate+ and I'm looking forward to yolonas for the better accuracy for small things. But if my rockchip had to do that without the Coral, I don't think it could manage, especially not if I wanted to use high res detect on the other two cameras too. It seems I'd need a newer rockchip and to use the -rk image. Or maybe I'll get an Intel or AMD mini computer and use openvino? This gets to the OP's question, is the Coral irrelevant in the future of Frigate? It seems to me Coral was the lever that allowed SBCs to perform well. Is that no longer going to be the case? Or are all systems including additional G/V/NPUs these days that allow them to handle this load?

[adamoutler]

As Hawkeye said, performance is slower on CPU. No reason for that if it can be done off-target via mqtt or API.

[hawkeye217]

At least for Semantic Search, adding network latency and running processes on another machine would likely increase the inference time significantly more than the current difference between a GPU and moderately powered CPU on a local machine.

[adamoutler]

Yes but it's just metadata that can be handled asynchronously right?

How long are we talking for semantic search on CPU? Is it noticable? I'm trying to avoid an inferior experience.

[hawkeye217]

My production instance of Frigate runs on an old i5 3rd gen, and semantic search inference with the small model only takes a few seconds when searching (with much of that time being database related). Embedding tracked objects in the background doesn't create any noticeable CPU increase for me, either.

[NickM-27]

Running on CPU with one of my test machines the inference time on CPU is 300ms