Help Needed: How to set up Frigate card w/ tapo camera 2-way audio?

[hohaha045]

Cross-posted in Reddit

SOLVED by igiannakas:
See

  streams:
    LivingRoom: 
      - ffmpeg:rtsp://rtspuname:[email protected]:554/stream1#audio=aac#video=copy
      - tapo://account_password_in_plain_text( the app/ cloud one) @172.24.4.80

*** Change your tapo cloud account password to something simpler.***
*** Change your tapo cloud account password to something simpler.***
*** Change your tapo cloud account password to something simpler.***

Original Issue Starts here:

Hello! Does anyone have experience setting up 2way audio with tapo camera?
I can listen to the camera audio but the microphone seems not working, but I couldn't figure it out.

My frigate local ip is 192.168.1.150:49182
and my tapo C210 ip is 192.168.1.10

I've added the following in the frigate config and I can see the live feed using Webrtc

cameras:
  Guardian: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://my_user:[email protected]:554/stream2 # <----- The stream you want to use for detection
          roles:
            - detect
            - audio
        - path: rtsp://my_user:[email protected]:554/stream1 # <----- The stream you want to use for record
          roles:
            - record
      #hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-aac

...
go2rtc:
    streams: 
        Guardian: # <- my tapo C210 camera name 
            - rtsp://my_user:[email protected]:554/stream1
            # - "ffmpeg:back#video=h264#audio=aac"
            - tapo://my_typo_cloud_user_name:[email protected]
        rtsp_cam: # <- for RTSP streams 
            - rtsp://my_user:[email protected]:554/stream1 # <- stream which supports video & aac audio 
            - "ffmpeg:rtsp_cam#audio=opus" # <- copy of the stream which transcodes audio to the missing codec (usually will be opus) 
        webrtc: 
            candidates: 
                - 192.168.1.150:49185 #<- this is my port since I am using DSM7 for docker 
                - stun:49185

I am not sure if the below necessary. Since it should be re-encoded the stream to H264 w/ aac audio? and I dont need to re-encode, the tapo is h.264, but I am not sure about the audio, but again, I can get the audio via Figate card live: webrtc so I am okay without it?

- "ffmpeg:back#video=h264#audio=aac"

and is the

- tapo://my_typo_cloud_user_name:[email protected]

necessary also? in the frigate card doc
"Camera must be supported by go2rtcfor 2-way audio (see supported cameras)."
and inside the go2rtc

and for port 49185 that should be correct, this is my port mapping,

I also added Port Forwarding in the router

And my Frigate card config

type: custom:frigate-card
cameras:
  - camera_entity: camera.guardian
    live_provider: go2rtc
    go2rtc:
      modes:
        - webrtc
...
menu:
  buttons:
    microphone:
      enabled: true
...
live:
  microphone:
    always_connected: false
    disconnect_seconds: 60

Any help appreciated
PS. I've blocked internet access for the camera, could that be the issue?

[PatSloan]

no clue if this is even a problem, but you are missing the '1' at the end of the stream URL:

"path: rtsp://my_user:[email protected]:554/stream # <----- The stream you want to use for record"

should be:

"path: rtsp://my_user:[email protected]:554/stream1 # <----- The stream you want to use for record"

[hohaha045]

thanks! I but turns out its just some copy and paste mistake when I masking my password

[hoangmnsd]

@hohaha045 Did you make it work?

[hohaha045]

Nope, but will re-visit this issue when I am a bit more free

[groyee]

Was anyone able to make it work?

I am still unable to make it work:

1. Some examples are with admin username, some without.
2. Some examples are with MD5, some sha256.
3. What is cloud here? Is it tapo care? https://www.tp-link.com/th/tapocare/

Would be great if anyone that made it work can share.

I have Tapo C110 camera

Thanks!

[groyee]

OK. Figured this out

Just in case anyone has the same problem. Some people say you need to encrypt the password with MD5, some say SHA256... none of it worked for me. Putting plain password (password that you use to login to your mobile app Tapo application) works. Here is my config:

go2rtc:
  streams:
    tapo_camera:
      - rtsp://admin:[email protected]:554/stream1
      - tapo://[email protected]

Obviously this works only on LAN. No idea how to make it work from remote...

[dimatx]

@groyee Can you please share the full config for that camera? Which ffmpeg profile/parameters did you have to use, if any?

[groyee]

Sure. If you use WebRTC, I am not even sure if you need ffmpeg transcoding here since the audio in this camera is PCM but to be on the safe side I did add it.

go2rtc:
  streams:
    tapo1:
      - rtsp://admin:[email protected]:554/stream2 #You can use stream1 instead for high resolution
      - "ffmpeg:tapo1#video=copy#audio=opus" #works without it also
      - tapo://[email protected]
cameras:
  tapo1:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/tapo1
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
    live:
      stream_name: tapo1
    snapshots: 
      enabled: True
    record: # 
      enabled: True

That's pretty much it.... Then I have my frigate card:

type: custom:frigate-card
cameras:
  - camera_entity: camera.tapo1
    go2rtc:
      modes: []
    webrtc_card: {}
    live_provider: go2rtc
view:
  update_force: true
  update_seconds: 0
menu:
  buttons:
    mute:
      enabled: true
    play:
      enabled: true
    microphone:
      enabled: true
      type: momentary
      alignment: matching
  style: overlay
live:
  preload: true
  controls:
    thumbnails:
      media: clips
  auto_play: all
  auto_mute: all
  microphone:
    always_connected: true
media_viewer:
  auto_mute: never
  controls:
    next_previous: {}
image:
  mode: camera

As long as you use https (not http) or you disabled this restriction in Chrome and as long as you are on the same network (or at least not behind symmetric nat) it should work

[hohaha045]

Are you using the TP-link Cloud password ? or the one on VLC "rtsp:username:[email protected](tapo_camera_ip):554/stream1" ?

- tapo://[email protected]

[dimatx]

Thanks @groyee. And what benefit does having both the rtsp:// and the tapo:// URI for your tapo1?

[groyee]

Honestly, I am not sure. I believe it will work without it as well

[groyee]

The one I use in the Tapo mobile app.

Open the Tapo app on your mobile-> View account (the password you used there, that's the password)

[hohaha045]

Ahh still not work for me, maybe only work on LAN as you said, but what's the point of 2-way if only work on LAN lol

[groyee]

Well... It works on WAN as well if you are not behind symmetric NAT. https://tomchen.github.io/symmetric-nat-test/

If you try to access it from a mobile (3G, 4G, 5G) then most likely you are behind symmetric NAT and it won't work. But many times you are on a WIFI at work or other places and it can be a normal NAT and then it should work

[hohaha045]

Huh, interesting, I am using the same network via Wifi, but my HA is under a cloudflare domain name

[derekcentrico]

@groyee have you upgraded to 0.14 yet? I tried your solution for my Tapo C120s. No two-way communications come up even though go2rtc probe sees the camera as capable.

My dahua cameras have the mic capability within Frigate's live view. The Tapo cameras do not.

[sam7567]

I figured this out recently and still looking for reliable solution. Just install and start go2rtc service in Home Assistant for basic troubleshooting. That is quicker to troubleshoot. Basically Tapo newer firmware is blocking things. You may follow JurajNyiri/HomeAssistant-Tapo-Control#625 for more info. I downgraded one of my C220 firmware and it started working. but only problem I noticed after that, is you can not go back to latest firmware anymore. (May be I don't know how to restore to Latest firmware) and I have still not figured out why Frigate video recorded without audio. I tried aac/pc/opus one at a time but no success however, 2-Way Audio worked perfectly.

In go2rtc.yaml populate your config as below:

streams:
  - tapo://<<YOUR CLOUD PASSWORD>>@192.168.X.X

Next: Click on probe on that Camera under go2rtc stream tab

Before Firmware flash:
streams: dial tcp 192.168.7.26:8800: i/o timeout

After Firmware Flash:

{
  "producers": [
    {
      "id": 2149370720,
      "format_name": "tapo",
      "protocol": "http",
      "remote_addr": "192.168.7.25:8800",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, PCMA/8000",
        "audio, sendonly, PCMA/8000"
      ],
      "receivers": [
        {
          "id": 25,
          "codec": {
            "codec_name": "pcm_alaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "childs": [
            26
          ]
        },
        {
          "id": 27,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "childs": [
            28
          ]
        }
      ],
      "senders": [
        {
          "id": 30,
          "codec": {
            "codec_name": "ANY",
            "codec_type": ""
          },
          "parent": 29
        }
      ]
    }
  ],
  "consumers": [
    {
      "id": 24,
      "format_name": "probe",
      "protocol": "http",
      "remote_addr": "172.30.32.2:59444 forwarded 192.168.4.53, 172.30.32.1",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
      "medias": [
        "audio, sendonly, ALL",
        "video, sendonly, ALL",
        "audio, recvonly, ANY"
      ],
      "receivers": [
        {
          "id": 29,
          "codec": {
            "codec_name": "ANY",
            "codec_type": ""
          },
          "childs": [
            30
          ]
        }
      ],
      "senders": [
        {
          "id": 26,
          "codec": {
            "codec_name": "pcm_alaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "parent": 25
        },
        {
          "id": 28,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 27
        }
      ]
    }
  ]
}

[Andurilll]

Anybody has done this 2-Way-Audio with frigate 0.14 & Tapo C210?

[hohaha045]

Try follow groyee reply above, and you should read this if you haven't already. But mine is not working lol But will re-try everything tomorrow

[hohaha045]

Same old news, didnt work

[igiannakas]

I’m using a C110 with the latest firmware and it worked. Settings as below:

go2rtc:
  streams:
    LivingRoom: 
      - ffmpeg:rtsp://rtspuname:[email protected]:554/stream1#audio=aac#video=copy
      - tapo://[email protected]

[hohaha045]

I’m using a C110 with the latest firmware and it worked. Settings as below:

go2rtc:
  streams:
    LivingRoom: 
      - ffmpeg:rtsp://rtspuname:[email protected]:554/stream1#audio=aac#video=copy
      - tapo://[email protected]

Do you mind show your full config? and the frigate card setting? I am trying the config you share, and something is happening, my c210 speaker/mic volume becomes unavailable.

[hohaha045]

I’m using a C110 with the latest firmware and it worked. Settings as below:

go2rtc:
  streams:
    LivingRoom: 
      - ffmpeg:rtsp://rtspuname:[email protected]:554/stream1#audio=aac#video=copy
      - tapo://[email protected]

Oh WOW, never mind. ITS WORKING!! I just change my tapo cloud account to something simpler- no symbol. And it freaking worked. Thank you so much!

[igiannakas]

Awesome! Glad it worked out:) even if it was 9 months later 😀

[igiannakas]

It also works without the ffmpeg statement - however I noticed lower quality playback when I removed it so I left it in. It may be a bug with my system though so give it a try if you want

[igiannakas]

Following up on this topic - In v0.15 I now need to set input_args: preset-rtsp-generic in the FFMPEG settings for it to work. For some reason it's not functioning with va-api. Also it appears like the video freezes when using two way audio.

[MarcoGeraldi]

Stuck in the same place: everything works except the two way audio and the native frigate PTZ controls in the lovelace UI (I had to use the ONVIF service but it is a bit buggy) .

Did you manage to find a fix or the issue is still present ?