write(file) and write(callback) functions for efficient sending

and WiFiUdpSender fix for AT 1.7.1

README.md

@@ -139,6 +139,8 @@ This library implements Arduino WiFi networking API. The last version of this AP
 ### the WiFiClient class differences
 
 * `connectSSL` is not implemented. It will be implemented as soon the AT firmware supports passive mode for SSL connection.
+* `write(file)` variant of write function for efficient sending of SD card file. see SDWebServer.ino example 
+* `write(callback)` variant of write function for efficient sending with a callback function. see SDWebServer.ino example 
 * `abort` closes the TCP connection without waiting for the remote side 
 * `getLinkId` returns the coresponding linkId of AT firmware for advanced use. It returns WIFIESPAT_NO_LINK if the client is unconnected. The valid range is from 0 to WIFIESPAT_LINKS_COUNT. 
 
@@ -159,6 +161,7 @@ The Arduino UDP API requires to start a listening port to send an UDP message. T
 You can use WiFiUdpSender class if you only send messages. See the UdpSender.ino example.
 
 * `beginMulticast` is not implemented
+* `write(callback)` variant of write function for efficient sending with a callback function 
 * `getLinkId` returns the corresponding linkId of AT firmware for advanced use. It returns WIFIESPAT_NO_LINK if the UDP is not initialized. The valid range is from 0 to WIFIESPAT_LINKS_COUNT.
 
 ## Logging
@@ -189,13 +192,19 @@ It is recommended to use WiFiClient.flush() after completing the output. WiFiCli
 
 The buffers size can be changed in WiFiEspAtConfig.h or set on build command line. The TCP TX buffer can be set to 0 and the RX buffer must be at least 1 (for peek()), but then please use buffers in sketch for example with [StreamLib's](https://github.com/jandrassy/StreamLib) wrapper class BufferedPrint. 
 
-The size of the UDP TX buffer can be set to zero in WiFiEspAtConfig.h if the complete message is sent with one print(msg) or one write(msg, length). Otherwise the size of the UDP buffers limits the size of the message. If the composed message is larger then the buffer it will be send as partial UDP messages. If the size of received message is larger then the UDP TX buffer, the message will be dropped (with WiFi.getLastDriverError() set to EspAtDrvError::UDP_LARGE).
+The size of the UDP TX buffer can be set to zero in WiFiEspAtConfig.h if the complete message is sent with one print(msg), one write(msg, length) or with write(callback). Otherwise the size of the UDP buffers limits the size of the message. If the composed message is larger then the buffer it will be send as partial UDP messages. If the size of received message is larger then the UDP TX buffer, the message will be dropped (with WiFi.getLastDriverError() set to EspAtDrvError::UDP_LARGE).
 
 To set different custom sizes of buffers for different boards, you can create a file boards.local.txt next to boards.txt file in hardware package. Set build.extra_flags for individual boards. For example for Mega you can add to boards.local.txt a line with -D options to define the macros.
 
 mega.build.extra_flags=-DWIFIESPAT_TCP_RX_BUFFER_SIZE=128 -DWIFIESPAT_TCP_TX_BUFFER_SIZE=128
 
 
+### the `write(callback)` function
+
+While the internal buffering of the library and the use of Nagle's algorithm by AT firmware prevents sending client.prints in many very very small TCP packets, with the write(callback) function all prints executed in the callback function are send to AT firmware with one AT+CIPSENDEX command resulting in efficient TCP or UDP packet size. AT+CIPSENDEX is limited to 2 kbytes and `\\0` terminates the command so it can't occur in data. 
+
+The SDWebServer example shows the use of the `write(callback)` function with C++ anonymous lambda functions as callbacks.
+
 ### EspAtDrv Errors
 
 The library functions with bool as return type return false in case of fail. The functions which return a value return 0 or - 1 in case of error, depending on the semantic of the function. To get the reason of the error the sketch can test the WiFi.getLastDriverError(). The error codes are enumerated in util/EspAtDrvTypes.h.

examples/Advanced/SDWebServer/SDWebServer.ino

@@ -0,0 +1,158 @@
+/*
+  SD Web Server example sketch for WiFiEspAT
+
+  the example demonstrates the write(file) and write() with callback functions
+
+  this example doesn't fit into 2 kB SRAM, so it can't be run on Uno, Nano, Mini
+
+  created in December 2019 for WiFiEspAT library
+  by Juraj Andrassy https://github.com/jandrassy
+
+*/
+
+#include <WiFiEspAT.h>
+#include <SD.h>
+#include <StreamLib.h> // install in Library Manager. Used to generate HTML of directory listing
+
+// Emulate Serial1 on pins 6/7 if not present
+#if defined(ARDUINO_ARCH_AVR) && !defined(HAVE_HWSERIAL1)
+#include <SoftwareSerial.h>
+SoftwareSerial Serial1(6, 7); // RX, TX
+#define AT_BAUD_RATE 9600
+#else
+#define AT_BAUD_RATE 115200
+#endif
+
+const int SDCARD_CS = 4;
+
+WiFiServer server(80);
+
+void setup() {
+
+  Serial.begin(115200);
+  while (!Serial);
+
+  if (!SD.begin(SDCARD_CS)) {
+    Serial.println(F("SD card initialization failed!"));
+    // don't continue
+    while (true);
+  }
+
+  Serial1.begin(AT_BAUD_RATE);
+  WiFi.init(Serial1);
+
+
+  if (WiFi.status() == WL_NO_MODULE) {
+    Serial.println(F("Communication with WiFi module failed!"));
+    // don't continue
+    while (true);
+  }
+
+  // waiting for connection to Wifi network set with the SetupWiFiConnection sketch
+  Serial.println(F("Waiting for connection to WiFi"));
+  while (WiFi.status() != WL_CONNECTED) {
+    delay(1000);
+    Serial.print('.');
+  }
+  Serial.println();
+
+  server.begin(3);
+
+  IPAddress ip = WiFi.localIP();
+  Serial.println();
+  Serial.println(F("Connected to WiFi network."));
+  Serial.print(F("To access the server, enter \"http://"));
+  Serial.print(ip);
+  Serial.println(F("/\" in web browser."));
+}
+
+void loop() {
+
+  static File file; // must by static to be accessible by the lambda functions
+  static char fn[32];
+
+  WiFiClient client = server.available();
+
+  if (client && client.connected()) {
+    if (client.find(' ')) { // GET /fn HTTP/1.1
+      int l = client.readBytesUntil(' ', fn, sizeof(fn) - 1); // read the filename from URL
+      fn[l] = 0;
+      while (client.read() != -1); // discard the rest of the request
+      file = SD.open(fn);
+      if (!file) { // file was not found
+        client.write([](Print& p) { // anonymous lambda function to be called by the library
+          p.println(F("HTTP/1.1 404 Not Found"));
+          p.println(F("Connection: close"));
+          p.print(F("Content-Length: "));
+          p.println(strlen(" not found") + strlen(fn));
+          p.println();
+          p.print(fn);
+          p.print(F(" not found"));
+        });
+      } else if (file.isDirectory()) {
+        client.write([](Print& p) { // anonymous lambda function to be called by the library
+          p.println(F("HTTP/1.1 200 OK"));
+          p.println(F("Connection: close"));
+          p.println(F("Content-Type: text/html"));
+          p.println(F("Transfer-Encoding: chunked"));
+          p.println();
+          char buff[64]; // buffer for chunks
+          ChunkedPrint chunked(p, buff, sizeof(buff));
+          chunked.begin();
+          chunked.printf(F("<!DOCTYPE html>\r\n<html>\r\n<body>\r\n<h3>Folder '%s'</h3>\r\n"), fn);
+          while (true) {
+            File entry = file.openNextFile();
+            if (!entry)
+              break;
+            if (strcmp(fn, "/") == 0) {
+              chunked.printf(F("<a href='%s'>"), entry.name());
+            } else {
+              chunked.printf(F("<a href='%s/%s'>"), fn, entry.name());
+            }
+            chunked.print(entry.name());
+            if (entry.isDirectory()) {
+              chunked.println(F("/</a><br>"));
+            } else  {
+              chunked.printf(F("</a> (%ld b)<br>\r\n"), entry.size());
+            }
+            entry.close();
+          }
+          chunked.println(F("</body>\r\n</html>"));
+          chunked.end();
+        });
+      } else {
+        client.write([](Print& p) { // anonymous lambda function to be called by the library
+          p.println(F("HTTP/1.1 200 OK"));
+          p.println(F("Connection: close"));
+          p.print(F("Content-Length: "));
+          p.println(file.size());
+          p.print(F("Content-Type: "));
+          const char* ext = strchr(file.name(), '.');
+          p.println(getContentType(ext));
+          p.println();
+        });
+        client.write(file); // send the file as body of the response
+        file.close();
+      }
+    }
+    client.stop();
+  }
+}
+
+const char* getContentType(const char* ext){
+  if (!strcmp(ext, ".HTM"))
+    return "text/html";
+  if (!strcmp(ext, ".CSS"))
+    return "text/css";
+  if (!strcmp(ext, ".JS"))
+    return "application/javascript";
+  if (!strcmp(ext, ".PNG"))
+    return "image/png";
+  if (!strcmp(ext, ".GIF"))
+    return "image/gif";
+  if (!strcmp(ext, ".JPG"))
+    return "image/jpeg";
+  if (!strcmp(ext, ".XML"))
+    return "text/xml";
+  return "text/plain";
+}

library.properties

@@ -1,5 +1,5 @@
 name=WiFiEspAT
-version=1.0.2
+version=1.1.0
 author=Juraj Andrassy
 maintainer=Juraj Andrassy <[email protected]>
 sentence=Enables network connection with esp8266 as network adapter.

src/WiFiClient.cpp

@@ -74,6 +74,14 @@ void WiFiClient::flush() {
   stream.flush();
 }
 
+size_t WiFiClient::write(Stream& file) {
+  return stream.write(file);
+}
+
+size_t WiFiClient::write(SendCallbackFnc callback) {
+  return stream.write(callback);
+}
+
 int WiFiClient::available() {
   return stream.available();
 }

src/WiFiClient.h

@@ -49,6 +49,9 @@ class WiFiClient : public Client {
   virtual size_t write(const uint8_t *buf, size_t size);
   virtual void flush();
 
+  size_t write(Stream& file);
+  size_t write(SendCallbackFnc callback);
+
   virtual int available();
   virtual int read();
   virtual int read(uint8_t *buf, size_t size);

src/WiFiEspAtBuffStream.cpp

@@ -86,6 +86,16 @@ int WiFiEspAtBuffStream::availableForWrite() {
   return txBufferSize - txBufferLength;
 }
 
+size_t WiFiEspAtBuffStream::write(Stream& file) {
+  flush();
+  return EspAtDrv.sendData(linkId, file, udpHost, udpPort);
+}
+
+size_t WiFiEspAtBuffStream::write(SendCallbackFnc callback) {
+  flush();
+  return EspAtDrv.sendData(linkId, callback, udpHost, udpPort);
+}
+
 int WiFiEspAtBuffStream::available() {
   if (linkId == NO_LINK)
     return 0;

src/WiFiEspAtBuffStream.h

@@ -22,6 +22,7 @@
 
 #include <Stream.h>
 #include <IPAddress.h>
+#include "utility/EspAtDrvTypes.h"
 
 class WiFiEspAtBuffStream {
 
@@ -39,6 +40,9 @@ class WiFiEspAtBuffStream {
   void flush();
   int availableForWrite();
 
+  size_t write(Stream& file);
+  size_t write(SendCallbackFnc callback);
+
   int8_t getWriteError() {return writeError;}
 
   int available();

src/WiFiEspAtConfig.h

@@ -20,8 +20,6 @@
 #ifndef _WIFIESPAT_CONFIG_H_
 #define _WIFIESPAT_CONFIG_H_
 
-#define AT_LOBO
-
 #ifndef WIFIESPAT_INTERNAL_AP_LIST_SIZE
 #if defined(__AVR__) && RAMEND <= 0x8FF
 #define WIFIESPAT_INTERNAL_AP_LIST_SIZE 3

src/WiFiUdp.cpp

@@ -35,6 +35,7 @@ int WiFiUdpSender::beginPacket(const char *host, uint16_t port) {
   }
   linkId = EspAtDrv.connect("UDP", host, port);
   stream.setLinkId(linkId);
+  stream.setUdpPort(host, port);
   return (linkId != NO_LINK);
 }
 
@@ -64,6 +65,10 @@ int WiFiUdpSender::availableForWrite() {
   return stream.availableForWrite();
 }
 
+size_t WiFiUdpSender::write(SendCallbackFnc callback) {
+  return stream.write(callback);
+}
+
 IPAddress WiFiUdpSender::remoteIP() {
   IPAddress ip;
   uint16_t port = 0;

src/WiFiUdp.h

@@ -40,6 +40,8 @@ class WiFiUdpSender : public UDP {
   virtual void flush();
   virtual int availableForWrite();
 
+  size_t write(SendCallbackFnc callback);
+
   using Print::write;
 
   virtual IPAddress remoteIP();