Refactor teal.reporter

[donyunardi]

Summary

teal.reporter is a powerful feature that allows users to create reports when they discover interesting analyses during exploration in teal. However, as user requirements continue to evolve, the current design has shown some limitations.

Some key challenges include:

• disk space concerns related to the bookmarking feature: teal issue #1188
• limited customization due to its tight integration with teal module programming, making it difficult for users to modify cards: teal.modules.general issue #736
• reproducibility challenges: teal.reporter issue #280
• additional concerns in nestdevs-tasks issue #44

To address these limitations, we aim to refactor teal.reporter with a more flexible and scalable design. This will involve researching potential solutions, conducting proof-of-concept (PoC) evaluations, and implementing a new design that better supports these evolving requirements.

Please see related comment here.

Definition of Done

• The new design supports bookmarking without excessive disk space usage.
• Users can customize report cards without modifying teal module programming.
• Reports generated through teal.reporter are fully reproducible.
• The refactored system maintains or improves existing functionalities.
• User can disable reporter in one or multiple teal modules

Tasks

• [Research] Research and PoC new teal.reporter design #105
• Implementation phase

[gogonzo]

Current state of insightsengineering/teal.reporter#307

The new design supports bookmarking without excessive disk space usage.

Reporter has been modified and should not cause disc storage problems - we need an app to test it @m7pr

Users can customize report cards without modifying teal module programming.
User can disable reporter in one or multiple teal modules

Not started yet. It needs extra teal function to modify returned ReportDocument object returned from the module (as reactive) and pass (or don't to disable) further.
At this point example app with an explicit <module>$server wrapper is enough to demonstrate the possibility.

Reports generated through teal.reporter are fully reproducible.

@m7pr has made a step forward and split the code so that it is possible to generate an executable document.

[m7pr]

The new design supports bookmarking without excessive disk space usage.
Reporter has been modified and should not cause disc storage problems - we need an app to test it @m7pr

You can use the app from the opening comment of the PR in teal.reporter

Code

devtools::load_all('../teal.reporter')
devtools::load_all('../teal.widgets')
devtools::load_all('../teal.code')
devtools::load_all('../teal')
devtools::load_all('.')

# general data example
data <- teal_data()
data <- within(data, {
  require(nestcolor)
  CO2 <- CO2
  USArrests <- USArrests
})

app <- init(
  data = data,
  modules = modules(
    tm_a_regression(
      label = "Regression",
      response = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = "uptake",
          selected = "uptake",
          multiple = FALSE,
          fixed = TRUE
        )
      ),
      regressor = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variables:",
          choices = variable_choices(data[["CO2"]], c("conc", "Treatment")),
          selected = "conc",
          multiple = TRUE,
          fixed = FALSE
        )
      )
    ),
    tm_a_pca(
      "PCA",
      dat = data_extract_spec(
        dataname = "USArrests",
        select = select_spec(
          choices = variable_choices(
            data = data[["USArrests"]], c("Murder", "Assault", "UrbanPop", "Rape")
          ),
          selected = c("Murder", "Assault"),
          multiple = TRUE
        ),
        filter = NULL
      )
    ),
    tm_g_scatterplot(
      label = "Scatterplot Choices",
      x = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]], c("conc", "uptake")),
          selected = "conc",
          multiple = FALSE,
          fixed = FALSE
        )
      ),
      y = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]], c("conc", "uptake")),
          selected = "uptake",
          multiple = FALSE,
          fixed = FALSE
        )
      ),
      color_by = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(
            data[["CO2"]],
            c("Plant", "Type", "Treatment", "conc", "uptake")
          ),
          selected = NULL,
          multiple = FALSE,
          fixed = FALSE
        )
      ),
      size_by = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]], c("conc", "uptake")),
          selected = "uptake",
          multiple = FALSE,
          fixed = FALSE
        )
      ),
      row_facet = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]], c("Plant", "Type", "Treatment")),
          selected = NULL,
          multiple = FALSE,
          fixed = FALSE
        )
      ),
      col_facet = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]], c("Plant", "Type", "Treatment")),
          selected = NULL,
          multiple = FALSE,
          fixed = FALSE
        )
      )
    ),
    tm_g_bivariate(
      x = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]]),
          selected = "conc",
          fixed = FALSE
        )
      ),
      y = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]]),
          selected = "uptake",
          multiple = FALSE,
          fixed = FALSE
        )
      ),
      row_facet = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]]),
          selected = "Type",
          fixed = FALSE
        )
      ),
      col_facet = data_extract_spec(
        dataname = "CO2",
        select = select_spec(
          label = "Select variable:",
          choices = variable_choices(data[["CO2"]]),
          selected = "Treatment",
          fixed = FALSE
        )
      )
    )
  )
)
if (interactive()) {
  shinyApp(app$ui, app$server, enableBookmarking = "server")
}

However I do not see a lot of disk space being saved. Rather the opposite.

shiny bookmark for tm_g_bivariate that is still kept in the old fashion for old teal.reporter classes

shiny bookmark for module that is written using new teal.reporter class that needs to be stored as .rds file

[m7pr]

Deleting my previous comment, as it was moved here #106
so that we have just one version of this comment and keep track of changes

[donyunardi]

We'll continue with #106
Closing.