Help me understand framerate disrepency

[jamesonuk]

I have two Reolink RCL-820As setup in Frigate and I noticed a difference in the reported fps between the feed direct from the camera an the feed via go2RTC.

I have started doing some digging and when I grabbed the steam using ffmpeg

ffmpeg -i rtsp://user:pass@camera:554/ -use_wallclock_as_timestamps 1 -c copy /ff/camera.mkv

Then I get this for the direct camera feed (there are some issues in there but the metadata does show up as 15fps which is what I had changed it to for testing.

ffmpeg camera output

[rtsp @ 0x1d8c000] max delay reached. need to consume packet
[rtsp @ 0x1d8c000] RTP: missed 24 packets
[h264 @ 0x1d90e00] error while decoding MB 106 35, bytestream -5
[h264 @ 0x1d90e00] concealing 8743 DC, 8743 AC, 8743 MV errors in P frame
[rtsp @ 0x1d8c000] max delay reached. need to consume packet
[rtsp @ 0x1d8c000] RTP: missed 21 packets
[h264 @ 0x1d90e00] error while decoding MB 23 44, bytestream -7
[h264 @ 0x1d90e00] concealing 7386 DC, 7386 AC, 7386 MV errors in P frame
[rtsp @ 0x1d8c000] max delay reached. need to consume packet
[rtsp @ 0x1d8c000] RTP: missed 5 packets
[h264 @ 0x1d90e00] concealing 3885 DC, 3885 AC, 3885 MV errors in P frame
[rtsp @ 0x1d8c000] max delay reached. need to consume packet
[rtsp @ 0x1d8c000] RTP: missed 47 packets
Input #0, rtsp, from 'rtsp://user:password@camera:554/':
  Metadata:
    title           : Session streamed by "preview"
    comment         : reolink rtsp stream
  Duration: N/A, start: 0.000313, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 2560x1440, 15 fps, 15 tbr, 90k tbn, 30 tbc
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
Output #0, matroska, to '/ff/camera.mkv':
  Metadata:
    title           : Session streamed by "preview"
    comment         : reolink rtsp stream
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(progressive), 2560x1440, q=2-31, 15 fps, 15 tbr, 1k tbn, 90k tbc
    Stream #0:1: Audio: aac ([255][0][0][0] / 0x00FF), 16000 Hz, mono, fltp
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[matroska @ 0x1df3800] Non-monotonous DTS in output stream 0:0; previous: 386, current: 78; changing to 386. This may result in incorrect timestamps in the output file.
[matroska @ 0x1df3800] Non-monotonous DTS in output stream 0:0; previous: 386, current: 198; changing to 386. This may result in incorrect timestamps in the output file.
[matroska @ 0x1df3800] Non-monotonous DTS in output stream 0:0; previous: 386, current: 220; changing to 386. This may result in incorrect timestamps in the output file.
[matroska @ 0x1df3800] Non-monotonous DTS in output stream 0:0; previous: 386, current: 263; changing to 386. This may result in incorrect timestamps in the output file.
[matroska @ 0x1df3800] Non-monotonous DTS in output stream 0:0; previous: 386, current: 358; changing to 386. This may result in incorrect timestamps in the output file.
[rtsp @ 0x1d8c000] max delay reached. need to consume packetbitrate=7357.6kbits/s speed=1.59x
[rtsp @ 0x1d8c000] RTP: missed 2 packets
[rtsp @ 0x1d8c000] max delay reached. need to consume packetbitrate=6666.6kbits/s speed=1.26x
[rtsp @ 0x1d8c000] RTP: missed 4 packets
frame=  933 fps= 15 q=-1.0 Lsize=   64338kB time=00:01:02.96 bitrate=8370.7kbits/s speed=1.04x
video:63834kB audio:488kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.025104%

ffmpeg go2RTC output

Input #0, rtsp, from 'rtsp://192.168.40.201:8554/hall_camera_frigate':
  Metadata:
    title           : go2rtc/1.9.2
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 2560x1440, 90k tbr, 90k tbn, 180k tbc
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp
Output #0, matroska, to '/ff/go.mkv':
  Metadata:
    title           : go2rtc/1.9.2
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(progressive), 2560x1440, q=2-31, 90k tbr, 1k tbn, 90k tbc
    Stream #0:1: Audio: aac ([255][0][0][0] / 0x00FF), 16000 Hz, mono, fltp
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  954 fps= 15 q=-1.0 Lsize=   65705kB time=00:01:04.93 bitrate=8288.7kbits/s speed=1.05x

which does seem to suggest the feed is 15fps but the metadata appears to have bee lost (not sure what tbr, tbn, tbc mean)

If I open up the go2RTC stream in VLC it is reporting the frame rate as 29.97 (but if I play the mkv output from above there is no fps at all)

Now the go2RTC feed appears to play better than the direct camera feed and I am trying to work out what (if anything) go2RTC is doing to change the feed (or is this just an issue with what VLC is showing as the metadata ?)

[AlexxIT]

Some utilities can show FPS from stream metadata. Some utilities check real FPS from stream. Metadata can lie.

[jamesonuk]

Some utilities can show FPS from stream metadata. Some utilities check real FPS from stream. Metadata can lie.

I understand that but it is sourced from the same stream so it is go2rtc that is causing the difference. If go2rtc was just restreaming without any repackaging or adjustments they would be the same and it is that I am trying to understand.

when streaming from the camera is says

 Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(progressive), 2560x1440, q=2-31, 15 fps, 15 tbr, 1k tbn, 90k tbc

but when putting this stream through go2rtc it is says

Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(progressive), 2560x1440, q=2-31, 90k tbr, 1k tbn, 90k tbc

whilst writing this I came across https://superuser.com/questions/1362410/what-is-fps-tbr-tbn-tbc-in-ffmpeg which I guess explains the 90k values.

I guess my question is what go2RTC is doing to the RTSP stream? Is it just repackaging the stream into a transport stream container? (Several facets to this, I am seeing 25% CPU usage from go2RTC which would seem a bit high if it isn't doing anything, I have had some issues with a couple of camera feeds not working properly and I would just like to understand this at a lower level)

[AlexxIT]

If go2rtc accepts RTSP stream, it outputs RTSP stream without any changes. Same SDP info and same RTP packets. In case of other transport formats - it can convert the data.

[jamesonuk]

If go2rtc accepts RTSP stream, it outputs RTSP stream without any changes. Same SDP info and same RTP packets. In case of other transport formats - it can convert the data.

Which is what I was expecting. The go2RTC bit of my Frigate config is

go2rtc:
  streams:
    landing_camera_frigate:
      - rtsp://{FRIGATE_REOLINK_USER}:{FRIGATE_REOLINK_PWD}@landing-camera/Preview_01_main
    hall_camera_frigate:
      - rtsp://{FRIGATE_REOLINK_USER}:{FRIGATE_REOLINK_PWD}@hall-camera/Preview_01_main

but as you can see from the above, the output of viewing that RTSP stream directly and viewing it via go2RTC are giving different output (both in VLC and via ffmpeg directly). so something is different in the two streams

[AlexxIT]

I'll check with my camera