go2rtc high CPU usage after adding 5th camera

[tomaszsluszniak]

Hi all,
I’m experiencing an issue with my Frigate installation. I’m running Frigate in a Proxmox LXC with a Coral TPU. After adding a fifth camera, everything worked fine at first, but after a while (perhaps an hour or a day), it started using all the CPU resources.

The fifth camera is a HiLook PTZ Camera (PTZ-N4MP-P).

I’m currently running Frigate 0.15 beta 4, but the same issue occurred on the stable 0.14 version. After restarting Frigate, the CPU usage returns to normal, but the problem repeats after about an hour.

There are logs from Frigate and go2rtc. I can't see anything that might cause such a problem.

Frigate:
frigate-logs.txt

go2rtc:
go2rtc-logs.txt

I'm using Frigate only for HA integration. All streams are taken from my HIK NVR, where all cameras are connected directly.

My config:

database:
  path: /config/frigate.db
mqtt:
  host: ****
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: ****
  password: ****
birdseye:
  enabled: true
  mode: continuous
record:
  enabled: False
  retain:
    days: 2
    mode: motion
  alerts:
    retain:
      days: 2
      mode: motion
  detections:
    retain:
      days: 2
      mode: motion
review:
  alerts:
    labels:
      - car
      - face
      - person
      - dog
      - cat
objects:
  track:
    - car
    - face
    - person
    - dog
    - cat
  filters:
    person:
      threshold: 0.9
    car:
      threshold: 0.85
detectors:
  coral_pci:
    type: edgetpu
    device: pci
model:
  height: 300
  width: 300
  path: /models/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
environment_vars:
  LIBVA_DRIVER_NAME: i965
ffmpeg:
  output_args:
    record: preset-record-generic-audio-copy
  hwaccel_args: preset-vaapi
go2rtc:
  streams:
    brama_wjazdowa_sub:
      - rtsp://frigate:****@****:554/Streaming/channels/102
      - ffmpeg:brama_wjazdowa_sub#video=h265#hardware=vaapi
    brama_wjazdowa:
      - rtsp://frigate:****@****:554/Streaming/channels/101
      - ffmpeg:brama_wjazdowa#video=h265#hardware=vaapi
    drzwi_wejsciowe_sub:
      - rtsp://frigate:****@****:554/Streaming/channels/202
      - ffmpeg:drzwi_wejsciowe_sub#video=h265#hardware=vaapi
    drzwi_wejsciowe:
      - rtsp://frigate:****@****:554/Streaming/channels/201
      - ffmpeg:drzwi_wejsciowe#video=h265#hardware=vaapi
    garaz_sub:
      - rtsp://frigate:****@****:554/Streaming/channels/302
      - ffmpeg:garaz_sub#video=h265#hardware=vaapi
    garaz:
      - rtsp://frigate:****@****:554/Streaming/channels/301
      - ffmpeg:garaz#video=h265#hardware=vaapi
    piwnica_sub:
      - rtsp://frigate:****@****:554/Streaming/channels/402
      - ffmpeg:piwnica_sub#video=h265#hardware=vaapi
    piwnica:
      - rtsp://frigate:****@****:554/Streaming/channels/401
      - ffmpeg:piwnica#video=h265#hardware=vaapi
    taras_sub:
      - rtsp://frigate:****@****:554/Streaming/channels/502
      - ffmpeg:taras_sub#video=h265#audio=aac#hardware=vaapi
    taras:
      - rtsp://frigate:****@****:554/Streaming/channels/501
      - ffmpeg:taras#video=h265#audio=aac#hardware=vaapi
  log:
    level: trace
    api: trace
    exec: trace
    ngrok: trace
    rtsp: trace
    streams: trace
    webrtc: trace
  webrtc:
    mode: webrtc/tcp
    candidates:
      - ****:8555
      - stun:8555
cameras:
  drzwi_wejsciowe:
    ui:
      order: 1
    live:
      stream_name: drzwi_wejsciowe_sub
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/drzwi_wejsciowe_sub
          roles:
            - detect
          input_args: preset-rtsp-restream
        - path: rtsp://127.0.0.1:8554/drzwi_wejsciowe
          roles:
            - record
          input_args: preset-rtsp-restream
    detect:
      enabled: true
      height: 720
      width: 1280
      stationary:
        threshold: 50
        max_frames:
          default: 3000
    snapshots:
      enabled: true
    motion:
      mask: 0.014,0.068,0.264,0.069,0.266,0.026,0.014,0.024
    zones:
      pod_daszkiem:
        coordinates: 
          0.144,0.687,0.194,0.837,0.257,1,0.423,1,0.55,1,0.746,1,0.771,0.925,0.795,0.854,0.839,0.72,0.779,0.653,0.714,0.587,0.657,0.527,0.597,0.468,0.467,0.344,0.422,0.384,0.379,0.342,0.268,0.444,0.192,0.523,0.124,0.595
        inertia: 3
        loitering_time: 0
        objects:
          - cat
          - dog
          - person
          - face
      rabata:
        coordinates: 0.474,0.334,0.84,0.708,0.955,0.226,0.837,0.138,0.742,0.135,0.641,0.179
        inertia: 3
        loitering_time: 0
        objects:
          - person
          - dog
          - cat
          - face
    review:
      alerts:
        required_zones:
          - pod_daszkiem
          - rabata
  garaz:
    ui:
      order: 2
    live:
      stream_name: garaz_sub
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garaz_sub
          roles:
            - detect
          input_args: preset-rtsp-restream
        - path: rtsp://127.0.0.1:8554/garaz
          roles:
            - record
          input_args: preset-rtsp-restream
    detect:
      enabled: true
      height: 720
      width: 1280
      stationary:
        threshold: 50
        max_frames:
          default: 3000
    snapshots:
      enabled: true
    zones:
      audi:
        coordinates: 0.655,0.25,0.802,0.368,0.598,1,0.277,1,0.175,0.669
        inertia: 3
        loitering_time: 0
      golf:
        coordinates: 0.134,0.572,0.554,0.099,0.373,0.01,0.065,0.248
        inertia: 3
        loitering_time: 0
    motion:
      mask: 0.027,0.021,0.025,0.074,0.277,0.071,0.278,0.021
    review:
      alerts:
        required_zones:
          - audi
          - golf
  piwnica:
    ui:
      order: 3
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/piwnica_sub
          roles:
            - detect
          input_args: preset-rtsp-restream
        - path: rtsp://127.0.0.1:8554/piwnica
          roles:
            - record
          input_args: preset-rtsp-restream
    detect:
      enabled: true
      height: 720
      width: 1280
      stationary:
        threshold: 50
        max_frames:
          default: 3000
    snapshots:
      enabled: true
    motion:
      mask:
        - 0.264,0.065,0.264,0.029,0.015,0.029,0.016,0.066
        - 0.063,0.49,0.203,0.472,0.216,0.722,0.094,0.711
    zones:
      przy_drzwiach:
        coordinates: 0.121,0.686,0.074,0.846,0.216,0.973,0.276,0.752
        inertia: 3
        loitering_time: 0
      przy_ogrodzeniu:
        coordinates: 0.803,0,0.806,1,1,1,1,0
        loitering_time: 0
    review:
      alerts:
        required_zones:
          - przy_ogrodzeniu
          - przy_drzwiach
  brama_wjazdowa:
    ui:
      order: 0
    live:
      stream_name: brama_wjazdowa_sub
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/brama_wjazdowa_sub
          roles:
            - detect
          input_args: preset-rtsp-restream
        - path: rtsp://127.0.0.1:8554/brama_wjazdowa
          roles:
            - record
          input_args: preset-rtsp-restream
    detect:
      enabled: true
      height: 720
      width: 1280
      stationary:
        threshold: 50
        max_frames:
          default: 3000
    snapshots:
      enabled: true
    zones:
      kostka:
        coordinates: 
          0.307,0.467,0.321,0.48,0.338,0.491,0.362,0.519,0.59,0.474,0.561,0.454,0.54,0.44,0.521,0.424,0.514,0.416
        inertia: 3
        loitering_time: 0
      ulica:
        coordinates: 
          0,0.555,0.037,0.538,0.124,0.505,0.229,0.468,0.388,0.424,0.54,0.381,0.704,0.344,1,0.309,1,0.122,0.268,0.25,0,0.346
        inertia: 3
        loitering_time: 0
      zwirek:
        coordinates: 
          0.364,0.525,0.339,0.612,0.268,0.72,0.194,0.841,0.116,1,1,1,1,0.854,0.976,0.725,0.924,0.659,0.764,0.617,0.684,0.582,0.595,0.532,0.593,0.479
        inertia: 3
        loitering_time: 0
    motion:
      mask: 0,0.341,0.273,0.242,0.619,0.184,1,0.12,1,0,0.002,0.001
    review:
      alerts:
        required_zones:
          - zwirek
          - kostka
  taras:
    ui:
      order: 4
    live:
      stream_name: taras_sub
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/taras_sub
          roles:
            - detect
            - audio
          input_args: preset-rtsp-restream
        - path: rtsp://127.0.0.1:8554/taras
          roles:
            - record
          input_args: preset-rtsp-restream
    onvif:
      host: ****
      port: 80
      user: onvif
      password: ****
    audio:
      enabled: true
      listen:
        - bark
        - scream
        - speech
        - yell
        - crying
        - children_playing
    detect:
      enabled: true
      height: 720
      width: 1280
    snapshots:
      enabled: true
    motion:
      mask: 0.013,0.051,0.271,0.049,0.271,0.099,0.014,0.097
    zones:
      Magnolia:
        coordinates: 0.443,0.653,0,0.97,0,1,1,1,1,0.738,0.937,0.641
        loitering_time: 0
      Piwnica:
        coordinates: 0.229,0.567,0.313,0.576,0.312,0.675,0.205,0.75,0.174,0.707,0.23,0.671
        loitering_time: 0
      Taras:
        coordinates: 
          0.357,0.379,0.358,0.487,0.314,0.508,0.318,0.583,0.768,0.609,0.765,0.469,0.675,0.468,0.631,0.365
        loitering_time: 0
version: 0.15-0
camera_groups:
  Front:
    order: 1
    icon: LuHome
    cameras:
      - brama_wjazdowa
      - drzwi_wejsciowe
      - garaz
  Tył:
    order: 2
    icon: LuTrees
    cameras:
      - piwnica
      - taras

[NickM-27]

Why are you trying to transcode to h265

[tomaszsluszniak]

I left it after some testing. In my case it’s already h265 from the HIK NVR.

[NickM-27]

then you should remove that second line from the cameras

[tomaszsluszniak]

Okay, I changed it. But now I have such “artifacts” on all cameras.

This one is 4K.

[tomaszsluszniak]

And today, after leaving it for the night, there is the same story. Go2rtc eats all the CPU resources.

[Cold-Lemonade]

I may be mistaken, but I think the following code in your config file is misplaced:

environment:
  LIBVA_DRIVER_NAME: i965

This needs to go in your docker compose file.

[NickM-27]

environment_vars is a valid way to set that as well in the frigate config

[tomaszsluszniak]

I already had it in both places

[tomaszsluszniak]

I found a new entry in the log.

undefined error=buffer too small url=rtsp://frigate:****@****:554/Streaming/channels/501 caller=github.com/AlexxIT/go2rtc/internal/streams/producer.go:171

This is the only issue I found when the CPU is saturated.

How to deal with it? Can this cause this problem? Is this something I can "fix" by changing the configuration?