v2.0.7.9016

- refactor: Shiny tab text updates - miniDOT concatenate - miniDOT reformat - Advanced (new tab) - Formatting Files
leppott · Sep 11, 2023 · ff1fd57 · ff1fd57
1 parent a86bceb
commit ff1fd57
Show file tree

Hide file tree

Showing 24 changed files with 407 additions and 76 deletions.
diff --git a/DESCRIPTION b/DESCRIPTION
@@ -1,6 +1,6 @@
 Package: ContDataQC
 Title: Quality Control (QC) of Continous Monitoring Data
-Version: 2.0.7.9015
+Version: 2.0.7.9016
 Authors@R: c(
 	person("Erik W", "Leppo", email="[email protected]",role=c("aut","cre")),
 	person("Ann","Roseberry Lincoln", role="ctb"),

diff --git a/NEWS b/NEWS
@@ -3,10 +3,20 @@ NEWS-ContDataQC
 
 <!-- NEWS.md is generated from NEWS.Rmd. Please edit that file -->
 
-    #> Last Update: 2023-09-08 15:57:43.083915
+    #> Last Update: 2023-09-11 14:24:14.581759
 
 # Version History
 
+## v2.0.7.9016
+
+2023-09-11
+
+- refactor: Shiny tab text updates
+  - miniDOT concatenate
+  - miniDOT reformat
+  - Advanced (new tab)
+  - Formatting Files
+
 ## v2.0.7.9015
 
 2023-09-08

diff --git a/NEWS.md b/NEWS.md
@@ -3,10 +3,20 @@ NEWS-ContDataQC
 
 <!-- NEWS.md is generated from NEWS.Rmd. Please edit that file -->
 
-    #> Last Update: 2023-09-08 15:57:43.083915
+    #> Last Update: 2023-09-11 14:24:14.581759
 
 # Version History
 
+## v2.0.7.9016
+
+2023-09-11
+
+- refactor: Shiny tab text updates
+  - miniDOT concatenate
+  - miniDOT reformat
+  - Advanced (new tab)
+  - Formatting Files
+
 ## v2.0.7.9015
 
 2023-09-08

diff --git a/NEWS.rmd b/NEWS.rmd
@@ -21,6 +21,15 @@ cat(paste0("Last Update: ",Sys.time()))
 
 # Version History
 
+## v2.0.7.9016
+2023-09-11
+
+* refactor: Shiny tab text updates
+    + miniDOT concatenate
+    + miniDOT reformat
+    + Advanced (new tab)
+    + Formatting Files
+
 ## v2.0.7.9015
 2023-09-08
 

diff --git a/inst/shiny-examples/ContDataQC/external/tab_1_About.R b/inst/shiny-examples/ContDataQC/external/tab_1_About.R
@@ -22,8 +22,12 @@ function() {
                                     ,includeHTML("www/RMD_HTML/App_1d_Tips.html")
                          )# tabPanel ~ QC Tips ~ END
 
+                         , tabPanel("Advanced"
+                                    ,includeHTML("www/RMD_HTML/App_1e_Advanced.html")
+                         )# tabPanel ~ Advanced ~ END
+
                          , tabPanel("Related Apps"
-                                    , includeHTML("www/RMD_HTML/App_1e_RelatedApps.html")
+                                    , includeHTML("www/RMD_HTML/App_1f_RelatedApps.html")
                          )## tabPanel ~ END
 
 

diff --git a/inst/shiny-examples/ContDataQC/external/tab_2_DataPrep.R b/inst/shiny-examples/ContDataQC/external/tab_2_DataPrep.R
@@ -6,7 +6,7 @@ function() {
            , mainPanel(
              tabsetPanel(type = "tabs"
 
-                         , tabPanel("Formatting files"
+                         , tabPanel("Formatting Files"
                                     , includeHTML("www/RMD_HTML/App_2a_FormattingFiles.html")
                          )## tabPanel ~ END
 
@@ -113,12 +113,15 @@ function() {
                                         )## tabsetPanel
                           )## tabPanel ~ END
 
+                         , tabPanel("Discrete Data"
+                                    , includeHTML("www/RMD_HTML/App_2cz_DiscreteData.html")
+                                    )## tabPanel ~ END
 
-                         , tabPanel("Organizing files"
+                         , tabPanel("Organizing Files"
                                     , includeHTML("www/RMD_HTML/App_2b_OrganizingFiles.html")
                          )## tabPanel ~ END
 
-                         , tabPanel("Naming files"
+                         , tabPanel("Naming Files"
                                     , includeHTML("www/RMD_HTML/App_2c_NamingFiles.html")
                                     # included pic in RMD to HTML
                                     # , img(src = "Fig_NamingFiles_v1_20210617.png"

diff --git a/inst/shiny-examples/ContDataQC/rmd/App_1e_Advanced.rmd b/inst/shiny-examples/ContDataQC/rmd/App_1e_Advanced.rmd
@@ -0,0 +1,39 @@
+---
+title: "Advanced"
+date: "`r format(Sys.time(), '%Y-%m-%d %H:%M:%S')`"
+always_allow_html: true
+output:
+  html_fragment:
+    toc: yes
+---
+
+```{r setup, include=FALSE}
+knitr::opts_chunk$set(results='asis', echo=FALSE, warning=FALSE, message = FALSE)
+# needed for trouble shooting
+boo_DEBUG <- FALSE
+if(boo_DEBUG==TRUE){
+  # myConfig <- file.path(system.file(package="ContDataQC"), "extdata", "config.ORIG.R")
+  # source(myConfig)
+}## IF ~ boo_DEBUG ~ END
+```
+# Advanced
+
+Some users may prefer to use the 
+<a href="https://github.com/leppott/ContDataQC" target="_blank">R code</a> 
+instead of the Shiny app. The example code in Figure 1 shows how to run the main
+functions when using the default settings in the R package. After the user 
+directs R to the correct directories and selects the operation (QCRaw, Aggregate
+or SummaryStats), there are several places in the code where the user needs to 
+make entries (e.g., SiteID, sensor type, and start/end date). R will read 
+through the file names in the directory, select files that match the criteria 
+and generate outputs for each input file that matches the criteria. 
+
+Users with intermediate to advanced-level R skills can customize the settings 
+(e.g., add new parameters, change units, change file naming and formatting 
+requirements, change the format of one of the QC reports from Word to html or 
+PDF) by editing a plain text [configuration file](Config_default.zip).  Users 
+have more flexibility using the Shiny app versus R package when it comes to 
+setting directories and file naming schemes because they can browse to the 
+desired input file(s).
+
+![Figure 1. Example code from the ContDataQC R package for running the QCRaw function. Entries highlighted in yellow show places where users can modify entries. The other colors show where linkages exist between lines of code, with comments explaining how the R code uses each entry.](RMD_Images/1.6_Fig1_Advanced_20230828.jpg){width=95%}
diff --git a/...les/ContDataQC/rmd/App_1e_RelatedApps.rmd → ...les/ContDataQC/rmd/App_1f_RelatedApps.rmd b/...les/ContDataQC/rmd/App_1e_RelatedApps.rmd → ...les/ContDataQC/rmd/App_1f_RelatedApps.rmd
diff --git a/inst/shiny-examples/ContDataQC/rmd/App_2a2_miniDOT1.Rmd b/inst/shiny-examples/ContDataQC/rmd/App_2a2_miniDOT1.Rmd
@@ -10,9 +10,37 @@ output: html_fragment
 knitr::opts_chunk$set(echo = FALSE)
 ```
 
-# miniDOT Concatenate
+# miniDOT DO Concatenate
 
-This function merges daily Dissolved Oxygen data files downloaded from PME 
-miniDot DO loggers into a single multi-column CSV file. Use if the data were not
-compiled using the PME java app on the logger. The output should be downloaded 
-into the Data0_Original folder.
+When data are initially downloaded from miniDOT sensors, there are separate 
+.txt files for each day (in this example, 325 individual files). Users have two 
+options for combining them into one CSV file: 1) the **ContDataQC Concatenate** 
+function; or 2) the **PME java app** that comes with the logger.
+
+## Concatenate Instructions
+
+* Click the ‘Browse’ button and select all the individual files that you want to combine. The status bar will tell you when the upload is complete (be patient – you may not see anything happen for the first 30 seconds or so, depending on how many files you are uploading).
+
+*	Click the ‘Concatenate’ button; a status bar will appear in the lower right corner.
+
+*	After the function is done running, a ‘Download’ button will appear. Click this button.
+
+*	You will be prompted to save a zipped file named ‘miniDOT_cat_date_time’. Save the zipped file in the Data0_Original folder.
+
+*	Unzip/extract the file(s) or else the files won’t be seen as an option to upload to the Reformat function. After file is unzipped, a new folder called ‘miniDOT_cat’ will appear. The original zipped file can be deleted if desired.
+
+## Additional Notes
+
+*	Want to try it yourself? Click [here](SiteX_20230828.zip) to 
+download miniDOT DO test files.
+
+*	Why are the DO saturation values all NA? This occurs when no inputs are 
+provided to calculate DO percent saturation.  To avoid issues with the main 
+ContDataQC QC function any field with all NA's is when the miniDOT file is
+created.
+
+## Examples
+
+Example image after performing the **Concatenate** function.
+
+![Concatenated miniDOT file](RMD_Images/miniDOT_2cat.png)
diff --git a/inst/shiny-examples/ContDataQC/rmd/App_2a3_miniDOT2.Rmd b/inst/shiny-examples/ContDataQC/rmd/App_2a3_miniDOT2.Rmd
@@ -1,7 +1,11 @@
 ---
 title: "DataPrep_miniDOT_Reformat, 3f"
 output: html_fragment
+editor_options: 
+  markdown: 
+    wrap: 72
 ---
+
 ```{r setup, include=FALSE}
 # ---
 # title: "DataPrep_miniDOT_Reformat"
@@ -10,36 +14,49 @@ output: html_fragment
 knitr::opts_chunk$set(echo = FALSE)
 ```
 
-# miniDOT Reformat
+# miniDOT DO Reformat
 
-The miniDOT reformat function  converts files to a format that is ready to run 
-through the DataQC function. This saves you time and eliminates errors that 
-sometimes occur when manually reformatting files.
+After the daily files are concatenated, run the combined file through
+the Reformat function. The function reformats the file so that it is
+ready to run through the QC report function.
 
 ## Instructions
 
-* Click the ‘Browse’ button and select the input file(s). You can run multiple 
-files at once. The status bar will tell you when the upload is complete.
+-   Click the 'Browse' button and select the input file(s). You can run
+    multiple files at once. The status bar will tell you when the upload
+    is complete.
+
+-   Click the 'Reformat miniDOT file(s)' button; a status bar will
+    appear in the lower right corner.
+
+-   After the function is done running, a 'Download' button will appear.
+    Click this button.
 
-* Click the ‘Reformat miniDOT file(s)’ button; a status bar will appear in the 
-lower right corner.
+-   You will be prompted to save a zipped file named
+    'miniDOT_reformat_date_time'. Save the zipped file in the Data1_RAW
+    folder.
 
-* After the function is done running, a ‘Download’ button will appear. Click 
-this button.
+-   Unzip/extract the file(s) or else the files won't be seen as an
+    option to upload to the QC report function (Main Functions - QC raw
+    data). After file is unzipped, a new folder called
+    'miniDOT_reformat' will appear. The original zipped file can be
+    deleted if desired.
 
-* Save the zipped file in the Data1_RAW folder
+-   See below for examples of files before and after reformatting.
 
-* Your files are now ready to be run through the QC function!
+## Additional Notes
 
-## Common errors -
+-   Want to try it yourself? Click [here](SiteX_20230828.zip) to
+    download miniDOT DO test files.
 
-*	DO percent saturation values of all NA. This occurs when no inputs are 
-provided to calculate DO percent saturation, and results in. no statistics being
-generated for DO percent saturation later in the QC process.
+-   Why are the DO saturation values all NA? This occurs when no inputs
+    are provided to calculate DO percent saturation. To avoid issues
+    with the main ContDataQC QC function any field with all NA's is when
+    the miniDOT file is created.
 
 ## Examples
 
-Example images of before and and after performing reformat.
+Example image before after performing the **Reformat** function.
 
 ![Concatenated miniDOT file](RMD_Images/miniDOT_2cat.png)
 

diff --git a/inst/shiny-examples/ContDataQC/rmd/App_2a_FormattingFiles.rmd b/inst/shiny-examples/ContDataQC/rmd/App_2a_FormattingFiles.rmd
@@ -0,0 +1,63 @@
+---
+title: "Formatting Files"
+date: "`r format(Sys.time(), '%Y-%m-%d %H:%M:%S')`"
+always_allow_html: true
+output:
+  html_fragment:
+    toc: yes
+---
+
+```{r setup, include=FALSE}
+knitr::opts_chunk$set(results='asis', echo=FALSE, warning=FALSE, message = FALSE)
+# needed for trouble shooting
+boo_DEBUG <- FALSE
+if(boo_DEBUG==TRUE){
+  # myConfig <- file.path(system.file(package="ContDataQC"), "extdata", "config.ORIG.R")
+  # source(myConfig)
+}## IF ~ boo_DEBUG ~ END
+
+# 20230911, file missing so recreated from Jen's "updates"
+```
+
+# Formatting Files
+
+Input files need to be formatted a specific way in order for the Shiny app and R package to work. Guidelines.
+
+*	Input files must be comma-delimited (CSV) (but not CSV UTF-8 files).
+
+*	The columns do not have to be in any specific order
+
+*	There are two required fields: SiteID; and “Date Time” (with Date and Time combined into one field).
+
+*	If using the default [configuration file](Config_default.zip), column names must match the capitalization, spelling and symbology shown in the table below (for example, R will not recognize ‘sensor depth ft’; it needs to be ‘Sensor Depth ft’)
+
+*	Either periods or spaces in the column headings are acceptable. For example, in the table below, air temperature is shown as ‘’Air Temp C’ but it can also be entered as ‘Air.Temp.C’.
+
+*	For units, slashes, spaces or periods (e.g., DO mg/L, DO.mg.L or DO mg L) are acceptable. Do NOT use parentheses.
+
+*	Advanced R users can add new parameters (beyond the ones show in the table below) to the configuration file.
+
+*	Optional fields include rowID, logger ID (serial number) and discrete measurements. Discrete (in situ) measurements can be entered for any parameter (just add ‘Discrete’ to the beginning of the column heading; for example, ‘Discrete Air Temp C’). The discrete data point will be included in the time series plots that are generated when the user runs the QC function. If using Onset HOBO sensors, you can save time by using the automated reformat function 
+
+
+## Data Formattting Information
+
+```{r table_1}
+library(readxl)
+library(knitr)
+library(kableExtra)
+# state directories
+table.dir  <- "tables"
+table.file <- "App_Information_Tables.xlsx"
+tab2a.dir  <- "DataPrep_Table2c"
+
+table2c <- read_excel(file.path(table.dir, table.file), sheet = tab2a.dir
+                     , na = c("NA", ""), trim_ws = TRUE, skip = 0
+                     , col_names = TRUE)  
+
+options(knitr.kable.NA = '')
+# kable(table1)
+table2c %>% 
+  kbl() %>% 
+  kable_styling(full_width = F, position = "left")
+```
diff --git a/inst/shiny-examples/ContDataQC/rmd/App_2cz_DiscreteData.rmd b/inst/shiny-examples/ContDataQC/rmd/App_2cz_DiscreteData.rmd
@@ -0,0 +1,28 @@
+---
+title: "Discrete Data"
+date: "`r format(Sys.time(), '%Y-%m-%d %H:%M:%S')`"
+always_allow_html: true
+output:
+  html_fragment:
+    toc: yes
+---
+
+```{r setup, include=FALSE}
+knitr::opts_chunk$set(results='asis', echo=FALSE, warning=FALSE, message = FALSE)
+# needed for trouble shooting
+boo_DEBUG <- FALSE
+if(boo_DEBUG==TRUE){
+  # myConfig <- file.path(system.file(package="ContDataQC"), "extdata", "config.ORIG.R")
+  # source(myConfig)
+}## IF ~ boo_DEBUG ~ END
+
+# 20230911, new tab from Jen's "updates"
+```
+
+# Discrete Data
+
+Discrete (in situ) measurements can be entered for any parameter. This needs to be done manually after the input file has been formatted (for example, after a HOBO file has been run through the Reformat function). Just add a new column into the input file and use the following naming scheme for the column heading: parameter name and units, with ‘Discrete’ at the beginning (for example, ‘Discrete Air Temp C’). The discrete data point(s) will be included in the time series plots in the Word or html QC reports that are generated when the user runs the QC function (below is an example). 
+
+Discrete measurements are important for validating your data, and in some cases, can be used to correct for sensor drift. Accuracy checks are comparisons of discrete or in situ measurements taken in the lab and/or in the field with sensor measurements from the closest date/time. The sensor measurement should be within the accuracy quoted by the manufacturer (e.g., ±0.2°C if you are using the Onset HOBO proV2 sensor). Click [configuration file](EXAMPLE_AccuracyCheckWkst.xlsx) to download an example of an accuracy check worksheet being kept by an RMN partner.
+
+![Example QC plot with discrete data](RMD_Images/2.3_Discrete_Fig1_20230828.jpg)