Merge pull request #17 from nutechsoftware/sm-dev

Next release 1.0.6 P6 improve and fix Twilio.

Author: f34rdotcom

CHANGELOG.md

@@ -3,11 +3,8 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
-
-## [1.0.6 P5] - 2021-01-10
-- [X] CORE: Improve names of components in menu config build.
-- [X] STSTK: DSC panels have no special command to disarm just the code is used. Sending the code while armed disarms and while disarmed will arm. To deal with this quirk on DSC if the panel is already DISARMED and the ST Disarm button is pushed the code will not send anything to the panel.
-## [Unreleased] WIP
+## [Unreleased] Open issues
+- [ ] CORE: Certificate store or some way to avoid having to store fixed static public keys for twilio, pushover and other REST api plugins.
 - [ ] CORE: FIXME: Setting HOST NAME when using static IP over ethernet not working.
 - [ ] CORE: FIXME: reboot of esp32 causes connected ser2sock clients to hang. So far various attempts to fix have been unsuccessful. Will need to do some network captures to determine the problem.
 - [ ] CORE: HUP/RESTART needs to be centralized so cleanup ex(_fifo_destroy) can happen first. How to connect with STSDK having its own restart calls.
@@ -23,16 +20,26 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - [ ] CORE: TODO: better hardware abstraction. Need to remove _esp_ specific code to make it easier to port to other hardware. Trying to keep the code as POSIX as possible with the limited resources I have.
 - [ ] CORE: TODO: ```'ping'``` command could come in handy. Again today needed this with ST MQTT servers seeming to be down.
 - [ ] STSDK: TODO: Add SmartThings Zone devices.
-## [1.0.6 P4] - 2021-01-08
+## [Unreleased] WIP Sean Mathews @f34rdotcom
+- [X] CORE: Move log mode init.
+- [X] API: Event format string for no match case to show event ID.
+- [X] CORE: Modify changelog(this file) to include info for blame.
+- [X] TWILIO: Fix syntax use newer format used in pushover module. Add simple HTTP response testing and and reporting to aid in setup.
+- [X] PUSHOVER & TWILIO: Fix bug in NV storage. Someday cleanup and make the prefix an arg to the NV routines. Would require less use if 'key' var but would it be easier to read?
+- [X] TWILIO: Update api.twilio.com public key. Uggg! This already changed :(. I need allow update of this key via CLI and I need to find a way to get the key to lookup the root certificate...
+## [1.0.6 P5] - 2021-01-10 Sean Mathews @f34rdotcom
+- [X] CORE: Improve names of components in menu config build.
+- [X] STSTK: DSC panels have no special command to disarm just the code is used. Sending the code while armed disarms and while disarmed will arm. To deal with this quirk on DSC if the panel is already DISARMED and the ST Disarm button is pushed the code will not send anything to the panel.
+## [1.0.6 P4] - 2021-01-08 Sean Mathews @f34rdotcom
 - [ ] CORE: ad2term arg to reset AD2 device and then start the terminal. Any argument string will send hold the reset line on the AD2pHat board low for 1 second as the terminal is started causing the AD2* to hard reset. Useful when firmware on the AD2* gets corrupted and its hard to get into the boot loader.
-## [1.0.6 P3] - 2021-01-08
+## [1.0.6 P3] - 2021-01-08 Sean Mathews @f34rdotcom
 - [X] CORE: Consolidate some common functions from Twilio and Pushover clients into ad2_utils functions.
 - [X] PUSHOVER: New notification component Pushover.net. A simple HTTPS request much like SendGrid or Twilio but I prefer this command format and will be changing other commands to match soon.
-## [1.0.6 P2] - 2020-12-29
+## [1.0.6 P2] - 2020-12-29 Sean Mathews @f34rdotcom
 - [X] CORE: refactor where serial starts to allow to enter ad2term mode early before any other  code starts. This allows to update firmware over the USB cable in ad2term mode using the GUI app or other AD2* firmware update tool.
-## [1.0.6 P1] - 2020-12-21
+## [1.0.6 P1] - 2020-12-21 Sean Mathews @f34rdotcom
 - [X] OTA: Some change in mbedtls happend at some point causing OTA to now fail. Releaseing P1 to fix.
-## [1.0.6] - 2020-12-19
+## [1.0.6] - 2020-12-19 Sean Mathews @f34rdotcom
 - [X] SER2SOCKD: Add new component ser2sock daemon.
   - [X] README
   - [X] CMD
@@ -41,7 +48,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
   - [X] QA/TESTING
 - [X] API: Refactor api. Simplify subscriber storage class. Add new subscriber callback type. Added ON_RAW_RX_DATA so subscribers can get a raw AD2* stream needed by ser2sockd to avoid writing in ser2sock client and uart client routines.
 - [X] CORE: Extract ser2sock fragments and move into ser2sock component.
-## [1.0.5] - 2020-12-15
+## [1.0.5] - 2020-12-15 Sean Mathews @f34rdotcom
 - [X] CORE: Tidy: Improve Kconfig menuconfig.
 - [X] API: System specific nibble was a bit. Fixed.
 - [X] API: ON_LOW_BATTERY() toggle ON/OFF with no battery on on test panel. This one keeps poping up...
@@ -58,7 +65,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - [X] TWILIO: Missed log refactor after serialize refactor. No logic changes.
 - [X] TWILIO: Fix ~250 byte memory leak in Twilio json code.. No logic changes.
 - [X] CORE: tidy.
-## [1.0.4] - 2020-11-08
+## [1.0.4] - 2020-11-08 Sean Mathews @f34rdotcom
 - [X] CORE: Tidy QA testing.
 - [X] TWILIO: Improve docs and ```twsas``` command report format as valid commands with comments.
 - [X] TWILIO: Need to add ability to trigger on common states FIRE, ALARM, DISARM, ARM, CHIME etc in new twsas command.
@@ -84,7 +91,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - [X] API: FIX: Improve FIRE and ALARM tracking issue with state toggle.
 - [X] TWILIO: TODO: Add class based command line configurable notifications to Twilio. Allow to enable/disable event messages for different event types.
 
-## [1.0.3 P2 WIP] - 2020-10-31 (no release)
+## [1.0.3 P2 WIP] - 2020-10-31 (no release) Sean Mathews @f34rdotcom
 - [X] API: New class AD2EventSearch and support functions to subscribe. I spent a few days on this. A bit of a unicorn hunt. I could write discrete API and CLI code for every possible message type from the AlarmDecoder protocol but this seems excessive. The task is the same on every message so a single CLI and API to create custom tracking of all messages using REGEX and simple state logic was what I ended up with. This new class when constructed becomes a virtual contact with OPEN/CLOSE/FAULT states. These states are updated based upon user supplied regex patterns and simple user provided logic hints.
 - [X] API: RFX message expand hex to bin for easy parsing.
 - [X] API: Added message type tracking during parse for post processing.
@@ -98,14 +105,14 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - [X] STSDK: Wired BYPASS to the bypass component contact capability.
 - [X] STSDK: remove battery fault contact from device profile in ST dev portal.
 
-## [1.0.3 P1] - 2020-10-20
+## [1.0.3 P1] - 2020-10-20 Sean Mathews @f34rdotcom
 - [X] CORE: Add command feedback to commands that require a restart to take effect.
 - [X] STSDK: Remove restart for enable. Add warnings about restarting.
 - [X] CORE: Fix: build error JSON.h issue using esp4.x.
 - [X] STSDK: Fix: Not enabling by default.
 - [X] TWILIO: Fix: mbedtls_x509_crt_parse is broken when building esp 4.x with return code -0x2180. The PEM parsing routine expects the last byte to be null but under 4.x build it is `-`. Using EMBED_TXTFILES automatically puts a \0 at the end of the block of memory and it is included in the size of the buffer. This is the setting that is used in the 3.x build component mk file.
 
-## [1.0.3] - 2020-10-19
+## [1.0.3] - 2020-10-19 Sean Mathews @f34rdotcom
 - [X] STSDK: Improve: Connect Panic Alarm and Medical Alarm buttons each requires 3 taps with 5 second timeout.
 - [X] STSDK: Improve: Connect DISARM button.
 - [X] STSDK: Improve: Connect ARM STAY, ARM AWAY buttons.
@@ -130,7 +137,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - [X] CORE: Fix deprecated stsdk api calls.
 - [X] CORE: Fix: building with espidf 4.x Currently testing only on 3.x branch but will switch to 4.x as soon as some warnings are sorted out.
 
-## [1.0.2] - 2020-10-11
+## [1.0.2] - 2020-10-11 Sean Mathews @f34rdotcom
 - [X] CORE: New: Command: ```'netmode <[W,E,N]> <ARGS>'```. If SmartThings is disabled allow control of network settings.
 - [X] CORE: Improve: Add util function ```ad2_query_key_value``` for N/V parsing. Will be used to store settings in NV in a easy to use text only way.
 - [X] CORE: Improve: Filled in empty wifi init function and added ```hal_event_handler```.
@@ -147,16 +154,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - [X] STSDK: Fix: Finish wiring smokeDetector capability and test FIRE alarm to show if smoke/clear events.
 - [X] STSDK: Tidy: Function name fix ```connection_start_task```.
 
-## [1.0.1 p1] - 2020-09-27
+## [1.0.1 p1] - 2020-09-27 Sean Mathews @f34rdotcom
 - [X] - Added support code for 'C' mode connection.
 - [X] - Refactor 'ad2source' command to allow selecting GPIO UARTS TX/RX pins.
 - [X] - New command 'ad2term' to connect directly to AD2* and stop processing.
 - [X] - Adding a ad2_* string util for splitting strings. Still needs const char * override.
 
-## [1.0.1] - 2020-09-25
+## [1.0.1] - 2020-09-25 Sean Mathews @f34rdotcom
 - [X] Code formatting and Travis CI setup on github. What a mess :)... "astyle --style=otbs"
 
-## [1.0.0B] - 2020-09-18
+## [1.0.0B] - 2020-09-18 Sean Mathews @f34rdotcom
 
 Refactor to take all dev work and clean it up into components that can be turned off / on as needed for a given project.
 Changes and tests.
@@ -178,7 +185,7 @@ Changes and tests.
 - [X] OTA: Add subscriber callback for ON_FIRMWARE_VERSION
 - [X] REFACTOR: Started adding event id string formats for human readable.
 
-## [1.0.0] - 2020-09-18
+## [1.0.0] - 2020-09-18 Sean Mathews @f34rdotcom
 
 Initial project release for testing.
 

README.md

@@ -279,66 +279,56 @@ Configure the notifications using the notification components CLI commands.
       - [c] Close output format string.
       - [f] Fault output format string.
 
-### Twilio notification commands
-  - Sets the 'Account SID' for notification.
+### Configuration for Twilio message notifications.
 
-    ```twsid {slot} {hash}```
+  - Sets the 'SID' for a given notification slot. Multiple slots allow for multiple twilio accounts.
+
+    ```twilio sid {slot} {hash}```
 
     - {slot}: [N]
-      - For default use 0. Support multiple Twilio accounts.
-    - {hash}: Twilio 'Account SID'. SendGrid 'KEY Name' or user defined N/A.
+      - For default use 0.
+    - {hash}: Twilio 'Account SID'.
 
-    Example: twsid 0 aabbccdd112233..
+    Example: twilio sid 0 aabbccdd112233..
 
-  - Sets the 'User Auth Token' for notifications.
+  - Sets the 'Auth Token' for a given notification slot.
 
-    ```twtoken {slot} {hash}```
+    ```twilio token {slot} {hash}```
 
     - {slot}: [N]
-      - For default use 0. Support multiple Twilio accounts.
-    - {hash}: Twilio 'User Auth Token'. SendGrid 'API KEY'.
-
-    Example: twtoken 0 aabbccdd112233..
-
-  - Sets the 'Type' for notification.
+      - For default use 0.
 
-    ```twtype {slot} {type}```
+    - {hash}: Twilio 'Auth Token'.
 
-    - {slot}
-      - For default use 0. Support multiple accounts.
-    - {type}: [M|C|E]
-      - Notification type [M]essage, [C]all, [E]mail.
+    Example: twilio token 0 aabbccdd112233..
 
-    Example: twtype 0 M
+  - Sets the 'From' info for a given notification slot.
 
-  - Sets the 'From' address for notification.
+    ```twilio from {slot} {phone|email}```
 
-    ```twfrom {slot} {phone} ```
+    - {phone|email}: [NXXXYYYZZZZ|[email protected]]
 
-    - {slot}: [N]
-      - For default use 0. Support multiple Twilio accounts.
-    - {phone}: [NXXXYYYZZZZ]
-      - From phone #
+  - Sets the 'To' info for a given notification slot.
 
-    Example: twfrom 0 13115552368
+    ```twilio to {slot} {phone|email}```
 
-  - Sets the 'To' address for notification.
+    - {phone|email}: [NXXXYYYZZZZ|[email protected]]
 
-    ```twto {slot} {phone}```
+  - Sets the 'Type' for a given notification slot.
 
-    - {slot}: [N]
-      - For default use 0. Support multiple Twilio accounts.
-    - {phone}: [NXXXYYYZZZZ]
-      - To phone #
+    ```twilio type {slot} {type}```
 
-    Example: twto 0 13115552368
+    - {type}: [M|C|E]
+      - Notification type [M]essage, [C]all, [E]mail.
+    Example: type 0 M
 
   - Define a smart virtual switch that will track and alert alarm panel state changes using user configurable filter and formatting rules.
 
-    ```twsas {slot} {setting} {arg1} [arg2]```
+    ```twilio switch {slot} {setting} {arg1} [arg2]```
 
     - {slot}
       - 1-99 : Supports multiple virtual smart alert switches.
+
     - {setting}
       - [N] Notification slot
         -  Notification settings slot to use for sending this events.
@@ -349,7 +339,7 @@ Configure the notifications using the notification components CLI commands.
       - [T] Message type filter.
         - {arg1}: Message type list seperated by ',' or empty to disables filter.
           - Message Types: [ALPHA,LRR,REL,EXP,RFX,AUI,KPM,KPE,CRC,VER,ERR,EVENT]
-            - For EVENT type the message will be generated by the API and not the AD2.
+            - For EVENT type the message will be generated by the API and not the AD2
       - [P] Pre filter REGEX or empty to disable.
       - [O] Open(ON) state regex search string list management.
         - {arg1}: Index # 1-8
@@ -368,91 +358,91 @@ Configure the notifications using the notification components CLI commands.
       - Configure notification profiles..
         ```console
         # Profile #0 EMail using api.sendgrid.com
-        twsid 0 NA
-        twtoken 0 Abcdefg012345....
-        twfrom 0 [email protected]
-        twto 0 [email protected]
-        twtype 0 E
+        twilio sid 0 NA
+        twilio token 0 Abcdefg012345....
+        twilio from 0 [email protected]
+        twilio to 0 [email protected]
+        twilio type 0 E
 
         # Profile #1 SMS/Text message using api.twilio.com
-        twsid 1 Abcdefg012345....
-        twtoken 1 Abcdefg012345....
-        twfrom 1 15555551234
-        twto 1 15555551234
-        twtype 1 M
+        twilio sid 1 Abcdefg012345....
+        twilio token 1 Abcdefg012345....
+        twilio from 1 15555551234
+        twilio to 1 15555551234
+        twilio type 1 M
 
         # Profile #2 Voice Twiml call using api.twilio.com
-        twsid 2 Abcdefg012345....
-        twtoken 2 Abcdefg012345....
-        twfrom 2 15555551234
-        twto 2 15555551234
-        twtype 2 C
+        twilio sid 2 Abcdefg012345....
+        twilio token 2 Abcdefg012345....
+        twilio from 2 15555551234
+        twilio to 2 15555551234
+        twilio type 2 C
         ```
       - Send notifications from profile in slot #0 for 5800 RF sensor with SN 0123456 and trigger on OPEN(ON), CLOSE(OFF) and FAULT REGEX patterns. In this example the Text or EMail sent would event contain the user defined message.
         ```console
         Twilio SmartSwitch #1 report
         # Set notification slot [N] to #0.
-        twsas 1 N 0
+        twilio switch 1 N 0
         # Set default virtual switch state [D] to 'CLOSED'(0)
-        twsas 1 D 0
+        twilio switch 1 D 0
         # Set auto reset time in ms [R] to 'DISABLED'
-        twsas 1 R 0
+        twilio switch 1 R 0
         # Set message type list [T]
-        twsas 1 T RFX
+        twilio switch 1 T RFX
         # Set pre filter REGEX [P]
-        twsas 1 P !RFX:0123456,.*
+        twilio switch 1 P !RFX:0123456,.*
         # Set 'OPEN' state REGEX Filter [O] #01.
-        twsas 1 O 1 !RFX:0123456,1.......
+        twilio switch 1 O 1 !RFX:0123456,1.......
         # Set 'CLOSED' state REGEX Filter [C] #01.
-        twsas 1 C 1 !RFX:0123456,0.......
+        twilio switch 1 C 1 !RFX:0123456,0.......
         # Set 'FAULT' state REGEX Filter [F] #01.
-        twsas 1 F 1 !RFX:0123456,......1.
+        twilio switch 1 F 1 !RFX:0123456,......1.
         # Set output format string for 'OPEN' state [o].
-        twsas 1 o RF SENSOR 0123456 OPEN
+        twilio switch 1 o RF SENSOR 0123456 OPEN
         # Set output format string for 'CLOSED' state [c].
-        twsas 1 c RF SENSOR 0123456 CLOSED
+        twilio switch 1 c RF SENSOR 0123456 CLOSED
         # Set output format string for 'FAULT' state [f].
-        twsas 1 f RF SENSOR 0123456 FAULT
+        twilio switch 1 f RF SENSOR 0123456 FAULT
         ```
       - Send notifications from profile in slot #2 in the example a Call profile when EVENT message "FIRE ON" or "FIRE OFF" are received. Use a Twiml string to define how the call is processed. This can include extensive external logic calling multiple people or just say something and hangup.
         ```console
         Twilio SmartSwitch #2 report
         # Set notification slot [N] to #2.
-        twsas 2 N 2
+        twilio switch 2 N 2
         # Set default virtual switch state [D] to 'CLOSED'(0)
-        twsas 2 D 0
+        twilio switch 2 D 0
         # Set auto reset time in ms [R] to 'DISABLED'
-        twsas 2 R 0
+        twilio switch 2 R 0
         # Set message type list [T]
-        twsas 2 T EVENT
+        twilio switch 2 T EVENT
         # Set 'OPEN' state REGEX Filter [O] #01.
-        twsas 2 O 1 FIRE ON
+        twilio switch 2 O 1 FIRE ON
         # Set 'CLOSED' state REGEX Filter [C] #01.
-        twsas 2 C 1 FIRE OFF
+        twilio switch 2 C 1 FIRE OFF
         # Set output format string for 'OPEN' state [o].
-        twsas 2 o <Response><Say>Notification alert FIRE ALARM</Say></Response>
+        twilio switch 2 o <Response><Say>Notification alert FIRE ALARM</Say></Response>
         # Set output format string for 'CLOSED' state [c].
-        twsas 2 c <Response><Say>Notification alert FIRE CLEAR</Say></Response>
+        twilio switch 2 c <Response><Say>Notification alert FIRE CLEAR</Say></Response>
         ```
       - Send notifications from profile in slot #2 in the example a Call profile when EVENT message "POWER BATTERY" or "POWER AC" are received. Use a Twiml string to define how the call is processed. This can include extensive external logic calling multiple people or just say something and hangup.
         ```console
         Twilio SmartSwitch #3 report
         # Set notification slot [N] to #2.
-        twsas 3 N 2
+        twilio switch 3 N 2
         # Set default virtual switch state [D] to 'CLOSED'(0)
-        twsas 3 D 0
+        twilio switch 3 D 0
         # Set auto reset time in ms [R] to 'DISABLED'
-        twsas 3 R 0
+        twilio switch 3 R 0
         # Set message type list [T]
-        twsas 3 T EVENT
+        twilio switch 3 T EVENT
         # Set 'OPEN' state REGEX Filter [O] #01.
-        twsas 3 O 1 POWER AC
+        twilio switch 3 O 1 POWER AC
         # Set 'CLOSED' state REGEX Filter [C] #01.
-        twsas 3 C 1 POWER BATTERY
+        twilio switch 3 C 1 POWER BATTERY
         # Set output format string for 'OPEN' state [o].
-        twsas 3 o <Response><Say>Notification alert ON MAIN AC POWER</Say></Response>
+        twilio switch 3 o <Response><Say>Notification alert ON MAIN AC POWER</Say></Response>
         # Set output format string for 'CLOSED' state [c].
-        twsas 3 c <Response><Say>Notification alert ON BATTERY BACKUP POWER</Say></Response>
+        twilio switch 3 c <Response><Say>Notification alert ON BATTERY BACKUP POWER</Say></Response>
         ```
       - Existing search verbs. ```RFX``` and others are not useful here as they can be filtered by message type ```RFX``` directly. The more useful verbs contain a modifier such as ON/OFF. TODO: Add ZONE tracking verbs and algorithm.
         ```

components/alarmdecoder-api/alarmdecoder_api.cpp

@@ -186,7 +186,7 @@ void AlarmDecoderParser::notifySubscribers(ad2_event_t ev, std::string &msg, AD2
     // convert event to human readable string and state OPEN/CLOSE/FAULT
     std::string emsg;
     if (event_str.find((int)ev) == event_str.end()) {
-        emsg = "EVENT ID %d" + std::to_string(ev);
+        emsg = "EVENT ID " + std::to_string(ev);
     } else {
         emsg = event_str[(int)ev];
     }