No polling but SSE ( Server Side Events ) support

[tjunussov]

DESCRIPTION

Today Tasmota uses REST API on WEB UI, ui polls every time sensors data or switch states,
The modern way to not use polling, but use websockets or the easiest is to switch to Server Side Events, it is supported almost on all browsers, js code easily can be modified to EventResource, it is kinda COMET, this guy already implemented it on ESP8266,
https://github.com/IU5HKU/ESP8266-ServerSentEvents

ADDITIONAL CONTEXT

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>

//declare the webserver
ESP8266WebServer server(80);

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect
  }
  
  if (WiFi.status() != WL_CONNECTED)
  {
    Serial.print(F("Connecting"));
        
    WiFi.persistent(false);       // WiFi config isn't saved in flash
    WiFi.mode(WIFI_STA);          // use WIFI_AP_STA if you want an AP
    WiFi.hostname("ESP8266");    // must be called before wifi.begin()
    WiFi.begin("YourSSID", "YourPassword");
   
    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(F("."));
    }
  }
     
  Serial.println();
  Serial.print(F("IP address: "));
  Serial.println(WiFi.localIP());  
  
  //start the webserver
  server.begin();
  //Server Sent Events will be handled from this URI
  server.on("/ssedata", handleSSEdata);
}

void loop() {
  // listen for incoming clients
  server.handleClient();
}

void handleSSEdata(){
  WiFiClient client = server.client();
  
  if (client) {
    Serial.println("new client");
    serverSentEventHeader(client);
    while (client.connected()) {
      serverSentEvent(client);
      delay(16); // round about 60 messages per second
    }

    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
    Serial.println("client disconnected");
  }
}

void serverSentEventHeader(WiFiClient client) {
  client.println("HTTP/1.1 200 OK");
  client.println("Content-Type: text/event-stream;charset=UTF-8");
  client.println("Connection: close");  // the connection will be closed after completion of the response
  client.println("Access-Control-Allow-Origin: *");  // allow any connection. We don't want Arduino to host all of the website ;-)
  client.println("Cache-Control: no-cache");  // refresh the page automatically every 5 sec
  client.println();
  client.flush();
}

void serverSentEvent(WiFiClient client) {
  client.println("event: esp8266"); // this name could be anything, really.
  client.print("data: {");
  client.print("\"A0\": ");
  client.print(1.0 * analogRead(0) / 1024.0);
  client.print(", \"in5\": ");
  client.print(digitalRead(5));
  client.print(", \"in6\": ");
  client.print(digitalRead(6));
  client.print(", \"text\": ESP8266");    // added just to show how you can add your own parameters
  client.println("}");
  client.println();
  client.flush();
}

[tjunussov]

Polling is wasting network data/energy, if data not changed, why we need poll it every time,
I will provide sample code to front-end and test it

[tjunussov]

this html code causing polling every 2345ms

function la(p) {
                var a = '';
                if (la.arguments.length == 1) {
                    a = p;
                    clearTimeout(lt);
                }
                if (x != null) {
                    x.abort();
                }
                x = new XMLHttpRequest();
                x.onreadystatechange = function() {
                    if (x.readyState == 4 && x.status == 200) {
                        var s = x.responseText.replace(/{t}/g, "<table style='width:100%'>").replace(/{s}/g, "<tr><th>").replace(/{m}/g, "</th><td style='width:20px;white-space:nowrap'>").replace(/{e}/g, "</td></tr>").replace(/{c}/g, "%'><div style='text-align:center;font-weight:");
                        eb('l1').innerHTML = s;
                    }
                }
                ;
                x.open('GET', '.?m=1' + a, true);
                x.send();
                lt = setTimeout(la, 2345);
            }

we need to switch to something like

if(typeof(EventSource)!=="undefined") {
    var e = new EventSource('?m=1');
    e.onmessage = function(event) {
       var s = event.data.replace(/{t}/g, "<table style='width:100%'>").replace(/{s}/g, "<tr><th>").replace(/{m}/g, "</th><td style='width:20px;white-space:nowrap'>").replace(/{e}/g, "</td></tr>").replace(/{c}/g, "%'><div style='text-align:center;font-weight:");
        eb('l1').innerHTML = s;
    }
} else {
// go with old school XMLHttpRequest
}
 

[ascillato2]

Hi,

Can you provide a pull request?

[barbudor]

What change is required on the Tasmota web server side to send the events ?
If you keep both SSE and polling-mode fallback, this is leading to increase the overall flash usage on the ESP (bigger HTML/js page + SSE code on the ESP side)

[tjunussov]

i see,
size is concern, but i think I could do it with 50-100bytes
Could you please point me where "?m1" is handled in what ino file ? I'm quite new to Arduino Tasmota, i came from Frontend world

[ascillato]

@tjunussov

What do you mean by ?m1 ?

If you meant http://IP/m1?, all sub menus are handled in the following file (except for the ones of each driver that are handled in the drivers' file). Anyway there isn't any m1.

Tasmota/tasmota/xdrv_01_webserver.ino

Lines 407 to 434 in a4e8b9d

	typedef struct WebServerDispatch_t {
	char uri[3]; // the prefix "/" is added automatically
	uint8_t method;
	void (*handler)(void);
	} WebServerDispatch_t;
	const WebServerDispatch_t WebServerDispatch[] PROGMEM = {
	{ "", HTTP_ANY, HandleRoot },
	{ "up", HTTP_ANY, HandleUpgradeFirmware },
	{ "u1", HTTP_ANY, HandleUpgradeFirmwareStart }, // OTA
	{ "u2", HTTP_OPTIONS, HandlePreflightRequest },
	{ "u3", HTTP_ANY, HandleUploadDone },
	{ "cs", HTTP_GET, HandleConsole },
	{ "cs", HTTP_OPTIONS, HandlePreflightRequest },
	{ "cm", HTTP_ANY, HandleHttpCommand },
	#ifndef FIRMWARE_MINIMAL
	{ "cn", HTTP_ANY, HandleConfiguration },
	{ "md", HTTP_ANY, HandleModuleConfiguration },
	{ "wi", HTTP_ANY, HandleWifiConfiguration },
	{ "lg", HTTP_ANY, HandleLoggingConfiguration },
	{ "tp", HTTP_ANY, HandleTemplateConfiguration },
	{ "co", HTTP_ANY, HandleOtherConfiguration },
	{ "dl", HTTP_ANY, HandleBackupConfiguration },
	{ "rs", HTTP_ANY, HandleRestoreConfiguration },
	{ "rt", HTTP_ANY, HandleResetConfiguration },
	{ "in", HTTP_ANY, HandleInformation },
	#endif // Not FIRMWARE_MINIMAL
	};

[Jason2866]

Webfrontend and http is trimed for minimal code size and is NOT intended to be used for communicating with the device (it is a fallback option). Tasmota is designed to be used with mqtt.
If your suggestion is reducing code size and RAM or has the same foot as the current implementation
a PR would be nice.
@barbudor why both? If SSE is smaller and more powerfull?

[barbudor]

Because his code includes

else {
// go with old school XMLHttpRequest
}

I understand he wants to keep both. May be in case a browser doesn't support it ?
Which would definitively increase the code size

[tjunussov]

Code could be rewritten to

function t(p) { 
  eb('l1').innerHTML = (p.responseText||p.data).replace(/{t}/g, "<table style='width:100%'>").replace(/{s}/g, "<tr><th>").replace(/{m}/g, "</th><td style='width:20px;white-space:nowrap'>").replace(/{e}/g, "</td></tr>").replace(/{c}/g, "%'><div style='text-align:center;font-weight:");
      }
  }

function la(p) {
                if(typeof(EventSource)!=="undefined") {
                  var e = new EventSource('?m=1');
                        e.onmessage = t;
                  return;
                }
                var a = '';
                if (la.arguments.length == 1) {
                    a = p;
                    clearTimeout(lt);
                }
                if (x != null) {
                    x.abort();
                }
                x = new XMLHttpRequest();
                x.onreadystatechange = function() {
                    if (x.readyState == 4 && x.status == 200) {
                        t(x);
                    }
                }
                x.open('GET', '.?m=1' + a, true);
                x.send();
                lt = setTimeout(la, 2345);
            }

and couting characters, around 110-120 characters will be added to codebase

 if(typeof(EventSource)!=="undefined") {var e = new EventSource('?m=1'); e.onmessage = t;return;}t(x);
function t(p) {} 

[arendst]

I have it working but I doubt if it lowers network load as it pushes data every 100ms.....

It probably needs a better server side handler which only pushes data if anything is changed. That's a major effort as the data is volatile as being provided by many sensors and drivers. Checking for changes is almost undoable as the data is chunked too.

I'm afraid we have to stick to the old polling feature.

[tjunussov]

1. To offload network, we need to implement change detection in data, ( thus increase code base )
2. Main concern i see, if multiple clients will be connected and wait for data, they will allocate socket connection per each client ( increase resource allocation )

[barbudor]

Tasmota is not meant to be used from the web GUI as the main interface and certainly not for multiple clients at the same time.
So the constraint to reduce such a network load which would occur only in limited cases is far behind the RAM and CPU usage that needs to be minimized.
CPU usage as a direct impact on power consumption which is a problem on some commercial device where the 3v3 power is not designed to sustain full time CPU operations.

To reduce network load, close webgui and use MQTT. Backend solutions provide a better place to deal with multiple viewers and optimized display.