Don't found how to optimise my data consumption

[Mlebailly22]

Hi,
I'm asking for your help because I've got a little issue.
I'm currently working on an ESP32 project that have to save in Firebase a tension every 30sec.
But my esp will work on 4G, so every Bytes is important .

I try to configure my connection with this parameters :

auth.user.email = MY_EMAIL;
auth.user.password = MY_PASSWORD;

config.database_url = MY_DATABASE_URL;
config.api_key = MY_API_KEY;
config.token_status_callback = tokenStatusCallback;

Firebase.reconnectNetwork(true);

fbdo.setBSSLBufferSize(2048, 1024);
stream.setBSSLBufferSize(2048, 1024);

Firebase.begin(&config, &auth);
stream.keepAlive(10, 10, 2);
fbdo.keepAlive(10, 10, 2);

config.timeout.networkReconnect = 10000;
config.tcp_data_sending_retry = 3;

And in my loop I run every 30ms

if (Firebase.ready())
    {
      // Check if firebase is connected
    }
    if (!stream.httpConnected())
    {
      // Server was disconnected!

    }

Actually, I don't really know why but I think it's to keep the library awake

And finally i have my main code that set a JSON to Firebase, sometimes every 30sec or sometimes every 5 or 10 minute.

if(Firebase.ready())
        {
            Serial.println("Data changed");
            tensionToSend = ANALOG_Get(TENSION);
            Serial.printf("New Tension : %d\n" , tensionToSend);
        
            FirebaseJson json;
            json.set("tension", tensionToSend);
            
            if(Firebase.setJSON(fbdo, "/Data", json))
            {
                Serial.println("Data sent");
                toSend = false;
                nbError = 0;
            }
            else
            {
                Serial.printf("Data not sent : %s", fbdo.errorReason().c_str());
                nbError++;
            }
            if(nbError > 5)
            {
                Serial.println("Too many errors, reseting");
                ESP.restart();
            }
        }

But my issue is that when I look on Wireshark what my ESP does, I see that sometimes it just send the data,

without any problem, and sometimes he need a lot of exchange (Hello Server) before sending my data

Why this red part is coming like if my connection was broken, but I have a "Keep-Alive" message every 10sec. I was thinking that this message was enough to keep my connection alive .

And the "restart" of my connection is unpredictable, sometimes it append 30 secondes after an other restart, and sometimes I can send 5 ou 6 messages without any problem.

Have you any idea how can I keep my connection awake to avoid 4 or 5ko of data lost every times I get this "Hello Server".

Thank you very much for all your help. ❤️
And I'm sorry for my english 😉

[mobizt]

Firebase server will close the session in 50 seconds for the session reusage by keeping session opened since connected.

Library will close the session on its end after 3 minutes.

Any request after 50 seconds and within 3 minutes since connected will fail because of response time out.

User needs to close the session when sending request during that period (50 sec. - 3 min.).

Note that the above statement except for SSE (stream).

[Mlebailly22]

okay, so if I understood correctly, as long as there are less than 50 seconds between each message my connection remains open? And I wouldn't have the opening message?
Is this rule true even without TCP Keep-Alive?
How can I close the session if I know I want to send between 50s and 3min?

Thank you so much for your help and reactivity ❤

[mobizt]

Actually, the session was closed in 3 min with no TCP keepalive.

With TCP keepalive (implemented internally by lwIP), the session will be closed when sending the new request and the lwIP socket indicates that the server connection was terminated.

I don't go in deep how lwIP implements this unless we only check the server connection status via client.connected() which the client in this normal use case is WiFiClient (used modified version to fix some issue) that implemented internally.

TCP client instance that used in FirebaseData object can be accessed by user as it is public object.

You can check the sever connection status using fbdo.tcpClient.connected().

Then you don't need to close the session during 50 sec to 3 min with TCP keepalive.

In case connection without TCP keepalive, you have to manage the time to close the session yourself because fbdo.tcpClient.connected() will always return true during 3 min.

[Mlebailly22]

Thank you very much for all your help !

[mobizt]

You have to keep in mind that, the data transfer via HTTPS has larger overhead for SSL data than non-secure HTTP connection.

[Mlebailly22]

and if my data is not sensitive at all, do I have a way to use a non-secure connection?

If I just change my url from "https://www ..." to "http://www" will that be enough ?

Thanks

[mobizt]

Firebase accepts only secure connection via port 443.

[mobizt]

As you know, TCP Keepalive causes more data usage as it probes the server connection periodically.

If data usage is your concern, do not use TCP keepalive.

[Mlebailly22]

Yes, I see that, I'll stop to use that, and keep it only for my stream.