adding datetime field to sarracenia messages for #5

Author: petersilva

opendcs-acquisition/Compared_to_Sundew.md

@@ -66,11 +66,12 @@ So, about 10x less stuff to manage.
 * Sarracenia provides much more built-in functionality, the only code here is to interface with OpenDCS
 * The Sundew plugin had to implement AMQP, building messages manually, and various availability strategies with stuff being left around from previous attempts.
 * In the Sarracenia plugin, availability is implemented with two lines of code in producing the query selection criteria instead of an array of scripts.
-* Sundew's pxRouting table, which is also needed, was left out from the comparison (it's 104klines... would have dwarfed everything else.)
+* Sundew's pxRouting table, which is also needed, was left out from the comparison (it's 104klines... would have dwarfed everything else, would have been 100:1 in that case.)
 * the maintenance interventions in Sarracenia are straightforward, edit one file, and restart. in Sundew, one has to re-run a sort
-  of compiler to rebuild configurations as an intermediate step.
-* in the Sundew version one must have one configuration per LRGS server, the Sarracenia plugin uses a single configuration to query all three with redundancy included.
+  of compiler to rebuild configurations as an intermediate step. The configuration files to manage are also larger.
+* In the Sundew version, one has a configuration per LRGS server, the Sarracenia plugin uses a single configuration to query all three with redundancy included.
 * The Sarracenia version sanity checks PDTs, noting the ones missing from the OpenDCS reference, for later resolution.
-
+* the Sarracenia plugin includes "geometry" (GeoJSON compatible location data) and "datetime" (observation datetime) headers in the messages,
+  not present in Sundew ones.
 
 

opendcs-acquisition/INSTALL.md

@@ -232,6 +232,11 @@ input data for dcpflow callback is the initial output from getDcpMessage -e -x (
 feedback from processing would be needed to understand whether this decode is easily 
 understood by downstream consumers.
 
+* in the message that accompanies the file, the following fields are set:
+  * datetime ( e.g. 'datetime': '2023-07-16T16:16:00Z' ) the observation time from the DCS message header.
+  * geometry ( e.g. 'geometry': {'type': 'Point', 'coordinates': (47.1714, -101.2025)}  the location of the
+    ob as per the Pdt database.
+
 ## Maintenance Activities
 
 ### Robustness to single LRGS Server failure

opendcs-acquisition/config/sr3/plugins/dcpflow.py

@@ -235,7 +235,7 @@ def on_start(self):
 
         self.pdts_setup()
 
-    def generate_message( self, BulletinFile, Pdt ):
+    def generate_message( self, BulletinFile, Pdt , ob_time):
         bulletin_stat = sarracenia.stat( BulletinFile )
 
 
@@ -265,6 +265,8 @@ def generate_message( self, BulletinFile, Pdt ):
                     logger.debug( f" msg: {msg} " )
                 except Exception as ex:
                    pass
+                if ob_time:
+                    msg['datetime'] = ob_time.strftime("%Y-%m-%dT%H:%M:%SZ")
             return msg
         else:
             logger.info( f"{bf2} already received, no need to publish again. discarding {BulletinFile}" )
@@ -343,7 +345,7 @@ def gather(self):
                 FirstLine=True
                 if bf:
                      bf.close()
-                     msg=self.generate_message(BulletinFile,Pdt)
+                     msg=self.generate_message(BulletinFile,Pdt,ob_time)
                      if msg:
                          messages.append(msg)
                 continue
@@ -385,7 +387,7 @@ def gather(self):
 
         if bf:
             bf.close()
-            msg=self.generate_message(BulletinFile,Pdt)
+            msg=self.generate_message(BulletinFile,Pdt,ob_time)
             if msg:
                 messages.append(msg)
         rof.close()