R UDF service prototype

• Contains documentation and vignettes (including pkgdown sites)
• Implemented using 3 strategies
• Increased usability for end-users: no need for code to be adapted for running in the cloud
• Ability to fetch and install R packages over the internet (e.g. CRAN, Github etc.)
• Query external services using suitable packages (e.g. OGC web services)
• Ability for back-ends to send supplementary files to the UDF service
• Ability to import arbitrary files required for executing UDFs over the internet
• Support for running C++ code along with R scripts using Rcpp

v0.0.3

This pre-release implements a new strategy for user-defined functions in R for Earth observation data. The RESTful web service implemented in v0.0.2 is extended to allow data represented as base64 encoded strings instead of JSON arrays in the POST body.

News: Now hosted on Docker Hub: https://hub.docker.com/r/pramitghosh/openeo.r.udf/!

Initial working draft

This release shows the use of User Defined Functions (UDFs) in R in the context of the OpenEO project. OpenEO.R.UDF is a R package and currently integrates with the implementation by @flahn in R here: https://github.com/Open-EO/openeo-r-backend/tree/feature/udf_develop.

Working example:

• Install required packages
  Install the packages "openeo-r-backend", "openeo" and "openEO.R.UDF"

   library(devtools)
   install_github("Open-EO/openeo-r-backend", ref = "feature/udf_develop")
   install_github("Open-EO/openeo-r-client")
   install_github("pramitghosh/OpenEO.R.UDF")

• Start the backend server and load demo data

   library(openEO.R.Backend)
   createServerInstance()
   openeo.server$workspaces.path = "<path/to/workspace/directory here>"
   openeo.server$initEnvironmentDefault()
   openeo.server$initializeDatabase()
  
   # Creating a user for the first time...
   # openeo.server$createUser(user_name = "<username here>", password = "<password here>")
  
   openeo.server$loadDemo() #Loads demo data
   openeo.server$startup(port = 8000) #Starts up local server on port 8000

• Login and execute process using the R client

   library(openeo)
   conn = connect(host = "http://127.0.0.1:8000/api/", user = "<registered username here>", password = "<password here>", rbackend = T) #Login using credentials
   conn %>% listProcesses() #Should show a list of available processes including `aggregate_time`
  
   # Upload UDF script file in R to the server from the local working directory
   # (e.g. <https://github.com/pramitghosh/OpenEO.R.UDF/blob/master/data/example_udf/sample_udf.R>)
   uploadUserData(conn, content = "../backend/sample_udf.R", target = "/script.R")
   #Check to make sure the file "script.R" is listed to be present on the server
   conn %>% listFiles()
  
   # Create process graph starting with
   # Sentinel-2 data -> Temporal filtering -> Applying UDF of type `aggregate_time` to find the median along the time axis for all the 13 bands
   task = collection("sentinel2_subset", id_name = "product_id") %>%
   process(process_id = "filter_daterange", prior.name = "imagery", from = "2017-05-01", to = "2017-06-20") %>%
   process(process_id = "aggregate_time", prior.name = "imagery", script = "/script.R")
   conn %>% executeTask(task = task, format = "GTiff", output_file = "out.tif") #Executes the process graph

The output file is to be found in the workspace directory that was defined. The end of processing could be observed when the message "Task result was sucessfully stored." appears on the console running the client.

Possible To-do tasks to explore later:

• Write parameters in_dim and drop_dim for run_UDF() to disk from the backend as metadata files along with the "legend" file.