Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hms::as_hms() function slow down the reading of slf files #83

Open
lizihao-anu opened this issue Aug 1, 2024 · 1 comment
Open

hms::as_hms() function slow down the reading of slf files #83

lizihao-anu opened this issue Aug 1, 2024 · 1 comment
Assignees
Labels
feature a feature request or enhancement

Comments

@lizihao-anu
Copy link
Contributor

lizihao-anu commented Aug 1, 2024

This issue is linked to the modification of #77. hms::as_hms() function slow down the reading of slf files because the parquet format does not support hms::as_hms(). @Moohan found this issue and suggested a workaround like

slfhelper::read_slf_episode(
  "1920",
  col_select = !dplyr::starts_with("keytime"),
  as_data_frame = FALSE
)

We can see this Public-Health-Scotland/phslookups#9 for more technical details on how to achieve this feature.

@lizihao-anu lizihao-anu added the feature a feature request or enhancement label Aug 1, 2024
@lizihao-anu lizihao-anu self-assigned this Aug 1, 2024
@Moohan
Copy link
Member

Moohan commented Aug 1, 2024

Hey, just wanted to be clear that the slowdown I noticed was when running something like:

library(slfhelper)
highland <- read_slf_episode("1920", partnerships = "S37000016")

The other issue and solution mentioned are separate and is basically how the tidyselect helper (which should work with arrow::read_parquet don't work as they're not being passed through properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature a feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants