-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix multiple cell ids match with the same dataset id #140
Conversation
myushen
commented
May 7, 2024
…th the same dataset id
R/import_metadata_and_counts.R
Outdated
|
||
# Generate cpm from counts | ||
cli_alert_info("Generating cpm from {.path {metadata_tbl$file_id_db}}. ") | ||
unique_file_ids <- unique(metadata_tbl$file_id_db) | ||
cli_alert_info("Generating cpm from {.path {unique_file_ids}}. ") | ||
get_counts_per_million(input_sce_obj = sce_obj, output_dir = counts_path$cpm_path, hd5_file_dir = counts_path$original_path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At this point, counts_path$cpm_path
seems like it's a vector of filepaths, one for each file_id_db
in the metadata. However, get_counts_per_million
expects a single file path. Does this work in practice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this API requires users to import a SingleCellExperiment object one at a time, and each object should have the same file_id_db
derived from dataset_id
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, I see that file_id_db = .data$dataset_id |> openssl::md5() |> as.character()
, so it there will only be one file_id_db
. I think to make this clearer, you could just create file_id_db
as a separate variable that you add to the metadata data frame, and also use to generate the paths. That way you won't have to filter out the redundant copies at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for that, @multimeric; I had a discussion with @myushen and concluded that an additional argument is not needed at this stage, just good messaging and documentation, in case the user input and multiple data sets in the same single experiment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't suggesting adding a new argument. I'm just saying that we have a single file ID, which gets duplicated as a result of putting it into the data frame, and then we filter it back down again to a single value using distinct
. This can be avoided by using a separate variable:
file_id_db <- .data$dataset_id |> openssl::md5() |> as.character()
...
metadata_tbl <-
metadata_tbl |> mutate(file_id_db = file_id_db)
...
original_path <- file.path(original_dir, basename(file_id_db))
cpm_path <- file.path(cache_dir, "cpm", basename(file_id_db))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but the redundant file_if_db column will be carried in the metadata, that is what we use to do future queries. So, in the metadata, the redundancy will remain.
Or am I missing something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I realise it will be like that in the metadata. My suggestion is just to make the code simpler and clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure if the additional argument would be in the front end (cost of each additional argument is $1M) or in the back end (cost $1)
sure in the backend @myushen follow @multimeric lead.
I think @myushen intended that as modifying the front end (code interface to the user)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will add a check point at the front end and rename the function for clarification
code optimisation Co-authored-by: Michael Milton <[email protected]>
… for clarification