Merge pull request #142 from MathewHDYT/master

Implement non blocking OTA update over MQTT

.github/workflows/arduino-compile.yml

@@ -33,6 +33,7 @@ jobs:
         - name: ArduinoJson
         - name: WiFiEsp
         - name: TinyGSM
+        - name: StreamUtils
 
     steps:
       - uses: actions/checkout@v3

.github/workflows/esp32-compile.yml

@@ -31,6 +31,7 @@ jobs:
         - name: TBPubSubClient
         - name: ArduinoHttpClient
         - name: ArduinoJson
+        - name: StreamUtils
 
     strategy:
       matrix:

.github/workflows/esp8266-compile.yml

@@ -32,6 +32,7 @@ jobs:
         - name: ArduinoHttpClient
         - name: ArduinoJson
         - name: Seeed_Arduino_mbedtls
+        - name: StreamUtils
 
     strategy:
       matrix:

README.md

@@ -59,7 +59,7 @@ Example implementations for all base features, mentioned above, can be found in
 
 ### No PROGMEM support causing crashes
 
-All constant variables are per default in flash memory to decrease the memory footprint of the library, if the libraries used or the board itself don't support `PROGMEM`. This can cause crashes to mitigate that simply add a `#define THINGSBOARD_ENABLE_PROGMEM 0` before including the ThingsBoard header file.
+If the device is crashing with an `Exception` especially `Exception (3)`, more specifically `LoadStoreError` or `LoadStoreErrorCause` this might be because, all constant variables are per default in flash memory to decrease the memory footprint of the library, if the libraries used or the board itself don't support `PROGMEM`. This can cause crashes to mitigate that simply add a `#define THINGSBOARD_ENABLE_PROGMEM 0` before including the ThingsBoard header file.
 
 ```c++
 // If not set otherwise the value is 1 per default if the pgmspace include exists,
@@ -86,7 +86,7 @@ To remove the need for the `MaxFieldsAmt` template argument in the constructor a
 The buffer size for the serialized JSON is fixed to 64 bytes. The SDK will not send data, if the size of it is bigger than the size originally passed in the constructor as a template argument (`PayLoadSize`). Respective logs in the `"Serial Monitor"` window will indicate the condition:
 
 ```
-[TB] Buffer size (64) to small for the given payloads size (83), increase with setBufferSize accordingly
+[TB] Buffer size (64) to small for the given payloads size (83), increase with setBufferSize accordingly or set THINGSBOARD_ENABLE_STREAM_UTILS to 1 before including ThingsBoard
 ```
 
 If that's a case, the buffer size for serialization should be increased. To do so, `setBufferSize()` method can be used or alternatively the `bufferSize` passed to the constructor can be increased as illustrated below:
@@ -107,6 +107,19 @@ void setup() {
 }
 ```
 
+Alternatively it is possible to enable the mentioned `THINGSBOARD_ENABLE_STREAM_UTILS` option, which sends messages that are bigger than the given buffer size with a method that skips the internal buffer, be aware tough this only works for sent messages. The internal buffer size still has to be big enough to receive the biggest possible message received by the client that is sent by the server.
+
+```cpp
+// Enable skipping usage of the buffer for sends that are bigger than the internal buffer size
+#define THINGSBOARD_ENABLE_STREAM_UTILS 1
+
+// For the sake of example
+WiFiEspClient espClient;
+
+// The SDK setup with 64 bytes for JSON buffer
+ThingsBoard tb(espClient);
+```
+
 ### Too much data fields must be serialized
 
 A buffer allocated internally by `ArduinoJson` library is fixed and is capable for processing not more than 8 fields. If you are trying to send more than that, you will get a respective log showing an error in the `"Serial Monitor"` window:

examples/0000-arduino_send_telemetry/0000-arduino_send_telemetry.ino

@@ -45,9 +45,9 @@ constexpr uint16_t THINGSBOARD_PORT = 1883U;
 // Maximum size packets will ever be sent or received by the underlying MQTT client,
 // if the size is to small messages might not be sent or received messages will be discarded
 #if THINGSBOARD_ENABLE_PROGMEM
-constexpr uint32_t MAX_MESSAGE_SIZE PROGMEM = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE PROGMEM = 128U;
 #else
-constexpr uint32_t MAX_MESSAGE_SIZE = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE = 128U;
 #endif
 
 // Baud rate for the debugging serial connection
@@ -90,7 +90,7 @@ void InitWiFi() {
   // Attempting to establish a connection to the given WiFi network
   WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
   while (WiFi.status() != WL_CONNECTED) {
-    // Delay 500ms until a connection has been succesfully established
+    // Delay 500ms until a connection has been successfully established
     delay(500);
 #if THINGSBOARD_ENABLE_PROGMEM
     Serial.print(F("."));
@@ -159,7 +159,7 @@ void loop() {
   if (!tb.connected()) {
     // Reconnect to the ThingsBoard server,
     // if a connection was disrupted or has not yet been established
-    char message[ThingsBoard::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
+    char message[Helper::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
     snprintf_P(message, sizeof(message), CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN);
     Serial.println(message);
     if (!tb.connect(THINGSBOARD_SERVER, TOKEN, THINGSBOARD_PORT)) {
@@ -180,8 +180,8 @@ void loop() {
   // Uploads new telemetry to ThingsBoard using MQTT.
   // See https://thingsboard.io/docs/reference/mqtt-api/#telemetry-upload-api
   // for more details
-  tb.sendTelemetryInt(TEMPERATURE_KEY, random(10, 31));
-  tb.sendTelemetryInt(HUMIDITY_KEY, random(40, 90));
+  tb.sendTelemetryData(TEMPERATURE_KEY, random(10, 31));
+  tb.sendTelemetryData(HUMIDITY_KEY, random(40, 90));
 
   tb.loop();
 }

examples/0000-arduino_send_telemetry/README.md

@@ -1,7 +1,7 @@
 # Sending telemetry data
 
 ## Devices
-| Supported Devcies |
+| Supported Devices |
 |-------------------|
 |  Arduino Uno      |
 |  ESP8266          |
@@ -10,6 +10,6 @@
 [Telemetry](https://thingsboard.io/docs/user-guide/telemetry/)
 
 ## Feature
-Allows to upload telemetry values to the cloud, compared to attributes
+Allows uploading telemetry values to the cloud, compared to attributes
 these values keep track of their previous values meaning we can draw graphs with them.
-Meant for values wich change over time and where a history might be useful (temperature, humidity, ...)
+Meant for values which change over time and where a history might be useful (temperature, humidity, ...)

examples/0001-arduino_send_batch/0001-arduino_send_batch.ino

@@ -45,9 +45,9 @@ constexpr uint16_t THINGSBOARD_PORT = 1883U;
 // Maximum size packets will ever be sent or received by the underlying MQTT client,
 // if the size is to small messages might not be sent or received messages will be discarded
 #if THINGSBOARD_ENABLE_PROGMEM
-constexpr uint32_t MAX_MESSAGE_SIZE PROGMEM = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE PROGMEM = 128U;
 #else
-constexpr uint32_t MAX_MESSAGE_SIZE = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE = 128U;
 #endif
 
 // Baud rate for the debugging serial connection
@@ -96,7 +96,7 @@ void InitWiFi() {
   // Attempting to establish a connection to the given WiFi network
   WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
   while (WiFi.status() != WL_CONNECTED) {
-    // Delay 500ms until a connection has been succesfully established
+    // Delay 500ms until a connection has been successfully established
     delay(500);
 #if THINGSBOARD_ENABLE_PROGMEM
     Serial.print(F("."));
@@ -160,7 +160,7 @@ void loop() {
   if (!tb.connected()) {
     // Reconnect to the ThingsBoard server,
     // if a connection was disrupted or has not yet been established
-    char message[ThingsBoard::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
+    char message[Helper::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
     snprintf_P(message, sizeof(message), CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN);
     Serial.println(message);
     if (!tb.connect(THINGSBOARD_SERVER, TOKEN, THINGSBOARD_PORT)) {

examples/0001-arduino_send_batch/README.md

@@ -1,7 +1,7 @@
 # Sending telemetry / attribute data
 
 ## Devices
-| Supported Devcies |
+| Supported Devices |
 |-------------------|
 |  Arduino Uno      |
 |  ESP8266          |
@@ -11,7 +11,7 @@
 [Attributes](https://thingsboard.io/docs/user-guide/attributes/)
 
 ## Feature
-Allows to upload telemetry values to the cloud, as well as attributes.
+Allows uploading telemetry values to the cloud, as well as attributes.
 Telemetry values keep track of their previous values meaning we can draw graphs with them.
-Meant for values wich change over time and where a history might be useful (temperature, humidity, ...)
-Where as attributes are meant for data, which does not require a history and is more seldomly updated (version, settings, ...)
+Meant for values which change over time and where a history might be useful (temperature, humidity, ...)
+Whereas attributes are meant for data, which does not require a history and is more seldom updated (version, settings, ...)

examples/0002-arduino_rpc/0002-arduino_rpc.ino

@@ -45,9 +45,9 @@ constexpr uint16_t THINGSBOARD_PORT = 1883U;
 // Maximum size packets will ever be sent or received by the underlying MQTT client,
 // if the size is to small messages might not be sent or received messages will be discarded
 #if THINGSBOARD_ENABLE_PROGMEM
-constexpr uint32_t MAX_MESSAGE_SIZE PROGMEM = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE PROGMEM = 128U;
 #else
-constexpr uint32_t MAX_MESSAGE_SIZE = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE = 128U;
 #endif
 
 // Baud rate for the debugging serial connection
@@ -99,7 +99,7 @@ void InitWiFi() {
   // Attempting to establish a connection to the given WiFi network
   WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
   while (WiFi.status() != WL_CONNECTED) {
-    // Delay 500ms until a connection has been succesfully established
+    // Delay 500ms until a connection has been successfully established
     delay(500);
 #if THINGSBOARD_ENABLE_PROGMEM
     Serial.print(F("."));
@@ -221,7 +221,7 @@ void loop() {
   if (!tb.connected()) {
     // Reconnect to the ThingsBoard server,
     // if a connection was disrupted or has not yet been established
-    char message[ThingsBoard::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
+    char message[Helper::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
     snprintf_P(message, sizeof(message), CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN);
     Serial.println(message);
     if (!tb.connect(THINGSBOARD_SERVER, TOKEN, THINGSBOARD_PORT)) {

examples/0002-arduino_rpc/README.md

@@ -1,7 +1,7 @@
 # Calling methods from the cloud on this device (Server-side RPC)
 
 ## Devices
-| Supported Devcies |
+| Supported Devices |
 |-------------------|
 |  Arduino Uno      |
 |  ESP8266          |

examples/0003-esp8266_esp32_send_data/0003-esp8266_esp32_send_data.ino

@@ -21,6 +21,11 @@
 // which might not be avaialable on lower end devices.
 #define ENCRYPTED false
 
+// Enables sending messages that are bigger than the predefined message size,
+// where the message will be sent byte by byte as a fallback instead.
+// Requires an additional library, see https://github.com/bblanchon/ArduinoStreamUtils for more information.
+#define THINGSBOARD_ENABLE_STREAM_UTILS 1
+
 
 #if USING_HTTPS
 #include <ThingsBoardHttp.h>
@@ -91,9 +96,9 @@ constexpr uint16_t THINGSBOARD_PORT = 1883U;
 // Maximum size packets will ever be sent or received by the underlying MQTT client,
 // if the size is to small messages might not be sent or received messages will be discarded
 #if THINGSBOARD_ENABLE_PROGMEM
-constexpr uint32_t MAX_MESSAGE_SIZE PROGMEM = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE PROGMEM = 128U;
 #else
-constexpr uint32_t MAX_MESSAGE_SIZE = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE = 128U;
 #endif
 
 // Baud rate for the debugging serial connection
@@ -211,7 +216,7 @@ void InitWiFi() {
   // Attempting to establish a connection to the given WiFi network
   WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
   while (WiFi.status() != WL_CONNECTED) {
-    // Delay 500ms until a connection has been succesfully established
+    // Delay 500ms until a connection has been successfully established
     delay(500);
 #if THINGSBOARD_ENABLE_PROGMEM
     Serial.print(F("."));
@@ -286,14 +291,14 @@ void loop() {
 #else
   Serial.println("Sending temperature data...");
 #endif
-  tb.sendTelemetryInt(TEMPERATURE_KEY, random(10, 31));
+  tb.sendTelemetryData(TEMPERATURE_KEY, random(10, 31));
 
 #if THINGSBOARD_ENABLE_PROGMEM
   Serial.println(F("Sending humidity data..."));
 #else
   Serial.println("Sending humidity data...");
 #endif
-  tb.sendTelemetryFloat(HUMIDITY_KEY, random(40, 90));
+  tb.sendTelemetryData(HUMIDITY_KEY, random(40, 90));
 
 #if !USING_HTTPS
   tb.loop();

examples/0003-esp8266_esp32_send_data/README.md

@@ -1,7 +1,7 @@
 # Sending telemetry / attribute data
 
 ## Devices
-| Supported Devcies |
+| Supported Devices |
 |-------------------|
 |  ESP32            |
 |  ESP8266          |
@@ -11,7 +11,7 @@
 [Attributes](https://thingsboard.io/docs/user-guide/attributes/)
 
 ## Feature
-Allows to upload telemetry values to the cloud, as well as attributes.
+Allows uploading telemetry values to the cloud, as well as attributes.
 Telemetry values keep track of their previous values meaning we can draw graphs with them.
-Meant for values wich change over time and where a history might be useful (temperature, humidity, ...)
-Where as attributes are meant for data, which does not require a history and is more seldomly updated (version, settings, ...)
+Meant for values which change over time and where a history might be useful (temperature, humidity, ...)
+Whereas attributes are meant for data, which does not require a history and is more seldom updated (version, settings, ...)

examples/0004-arduino-sim900_send_telemetry/0004-arduino-sim900_send_telemetry.ino

@@ -59,9 +59,9 @@ constexpr uint16_t THINGSBOARD_PORT = 1883U;
 // Maximum size packets will ever be sent or received by the underlying MQTT client,
 // if the size is to small messages might not be sent or received messages will be discarded
 #if THINGSBOARD_ENABLE_PROGMEM
-constexpr uint32_t MAX_MESSAGE_SIZE PROGMEM = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE PROGMEM = 128U;
 #else
-constexpr uint32_t MAX_MESSAGE_SIZE = 128U;
+constexpr uint16_t MAX_MESSAGE_SIZE = 128U;
 #endif
 
 // Baud rate for the debugging serial connection
@@ -192,7 +192,7 @@ void loop() {
   if (!tb.connected()) {
     // Reconnect to the ThingsBoard server,
     // if a connection was disrupted or has not yet been established
-    char message[ThingsBoard::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
+    char message[Helper::detectSize(CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN)];
     snprintf_P(message, sizeof(message), CONNECTING_MSG, THINGSBOARD_SERVER, TOKEN);
     Serial.println(message);
     if (!tb.connect(THINGSBOARD_SERVER, TOKEN, THINGSBOARD_PORT)) {
@@ -213,14 +213,14 @@ void loop() {
 #else
   Serial.println("Sending temperature data...");
 #endif
-  tb.sendTelemetryInt(TEMPERATURE_KEY, random(10, 31));
+  tb.sendTelemetryData(TEMPERATURE_KEY, random(10, 31));
 
 #if THINGSBOARD_ENABLE_PROGMEM
   Serial.println(F("Sending humidity data..."));
 #else
   Serial.println("Sending humidity data...");
 #endif
-  tb.sendTelemetryFloat(HUMIDITY_KEY, random(40, 90));
+  tb.sendTelemetryData(HUMIDITY_KEY, random(40, 90));
 
   tb.loop();
 }

examples/0004-arduino-sim900_send_telemetry/README.md

@@ -1,7 +1,7 @@
 # Sending telemetry data
 
 ## Devices
-| Supported Devcies    |
+| Supported Devices    |
 |----------------------|
 |  Arduino Uno         |
 |  GSM Modem (SIM9000) |
@@ -10,6 +10,6 @@
 [Telemetry](https://thingsboard.io/docs/user-guide/telemetry/)
 
 ## Feature
-Allows to upload telemetry values to the cloud, compared to attributes
+Allows uploading telemetry values to the cloud, compared to attributes
 these values keep track of their previous values meaning we can draw graphs with them.
-Meant for values wich change over time and where a history might be useful (temperature, humidity, ...)
+Meant for values which change over time and where a history might be useful (temperature, humidity, ...)

examples/0005-arduino-sim900_send_telemetry_http/0005-arduino-sim900_send_telemetry_http.ino

@@ -192,12 +192,12 @@ void loop() {
 #else
   Serial.println("Sending temperature data...");
 #endif
-  tb.sendTelemetryInt(TEMPERATURE_KEY, random(10, 31));
+  tb.sendTelemetryData(TEMPERATURE_KEY, random(10, 31));
 
 #if THINGSBOARD_ENABLE_PROGMEM
   Serial.println(F("Sending humidity data..."));
 #else
   Serial.println("Sending humidity data...");
 #endif
-  tb.sendTelemetryFloat(HUMIDITY_KEY, random(40, 90));
+  tb.sendTelemetryData(HUMIDITY_KEY, random(40, 90));
 }

examples/0005-arduino-sim900_send_telemetry_http/README.md

@@ -1,7 +1,7 @@
 # Sending telemetry data
 
 ## Devices
-| Supported Devcies    |
+| Supported Devices    |
 |----------------------|
 |  Arduino Uno         |
 |  GSM Modem (SIM9000) |
@@ -10,6 +10,6 @@
 [Telemetry](https://thingsboard.io/docs/user-guide/telemetry/)
 
 ## Feature
-Allows to upload telemetry values to the cloud using HTTP (normally MQTT is used), compared to attributes
+Allows uploading telemetry values to the cloud using HTTP (normally MQTT is used), compared to attributes
 these values keep track of their previous values meaning we can draw graphs with them.
-Meant for values wich change over time and where a history might be useful (temperature, humidity, ...)
+Meant for values which change over time and where a history might be useful (temperature, humidity, ...)