Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify hello-gatk #364

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

adamrtalbot
Copy link
Collaborator

@adamrtalbot adamrtalbot commented Sep 3, 2024

This PR simplifies the hello-gatk example.

  • Uses simpler data inputs which are much, much smaller enhancing portability
  • Reduces reliance on samplesheets to make handling inputs more approachable for a beginner
  • Removes tuples of sample ID to make passing between processes a bit simpler, cleans up the message a bit
  • Uses only 1 operator (collect), which makes it more obvious what each step does

Negatives:

  • doesn't include any 'interesting' operators, which might show how to use channels
  • uses a weird chromosome (chr20:coordinateStart-coordinateEnd) which is harder to develop for us

Resolves #355

… methods.

This should make it quicker and cleaner to use as training material.
@adamrtalbot adamrtalbot marked this pull request as ready for review September 3, 2024 17:52
Copy link

netlify bot commented Sep 3, 2024

Deploy Preview for nextflow-training ready!

Name Link
🔨 Latest commit 133b592
🔍 Latest deploy log https://app.netlify.com/sites/nextflow-training/deploys/66f43f11f18c360008bdea2f
😎 Deploy Preview https://deploy-preview-364--nextflow-training.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@adamrtalbot adamrtalbot marked this pull request as draft September 3, 2024 17:52
Comment on lines 38 to 44
params.reads_bam = "./data/samplesheet.csv"

// Accessory files
params.genome_reference = "${projectDir}/data/ref/ref.fasta"
params.genome_reference_index = "${projectDir}/data/ref/ref.fasta.fai"
params.genome_reference_dict = "${projectDir}/data/ref/ref.dict"
params.calling_intervals = "${projectDir}/data/intervals.list"
params.reference = "./data/ref/ref.fasta"
params.reference_index = "./data/ref/ref.fasta.fai"
params.reference_dict = "./data/ref/ref.dict"
params.calling_intervals = "./data/intervals.list"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workflow.projectDir doesn't exist in the configuration, so we just use a relative path.

We could wrap it in closures but that makes it complicated.

Copy link
Member

@mribeirodantas mribeirodantas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a quick look, it seems cool! It'd be good if someone else could try out the new version of this training.

Co-authored-by: Marcel Ribeiro-Dantas <[email protected]>
@adamrtalbot
Copy link
Collaborator Author

A small side benefit I noticed while testing, because the two containers share most of the layers downloading them is much more efficient.

@mribeirodantas mribeirodantas marked this pull request as ready for review September 4, 2024 23:15
@mribeirodantas mribeirodantas marked this pull request as draft September 4, 2024 23:15
@vdauwera
Copy link
Collaborator

vdauwera commented Sep 6, 2024

Reviewing the overall flow right now, at first and second glance everything looks really good, I like that this is simpler. We can always demonstrate more advanced stuff in later trainings.

One quick comment is I noticed the new use of checkIfExists: true but it's not formally explained/introduced anywhere in the text, would be nice to put in a short explanation (and link to docs which we should do more anyway)

Also I noticed this numbering/header issue:

Screenshot 2024-09-06 at 5 59 11 PM

@kenibrewer
Copy link
Member

@adamrtalbot Just a heads up that we'd like to merge this for our training in SF tomorrow.

@vdauwera vdauwera marked this pull request as ready for review September 25, 2024 05:51
@vdauwera vdauwera marked this pull request as draft September 25, 2024 06:25
@adamrtalbot
Copy link
Collaborator Author

@adamrtalbot Just a heads up that we'd like to merge this for our training in SF tomorrow.

I'm happy with it, if you are hit merge.

Currently needs an approval 😉

@vdauwera
Copy link
Collaborator

@adamrtalbot Just a heads up that we'd like to merge this for our training in SF tomorrow.

I'm happy with it, if you are hit merge.

Currently needs an approval 😉

I've reviewed in more detail and I feel like there's a couple of things that are a bit of a jump, so I'm making an intermediate version with most but not all of the changes

@adamrtalbot
Copy link
Collaborator Author

I've just tried removing the reference channels and it still works.

From:

ref_ch               = Channel.fromPath(params.reference, checkIfExists: true).collect()
ref_index_ch         = Channel.fromPath(params.reference_index, checkIfExists: true).collect()
ref_dict_ch          = Channel.fromPath(params.reference_dict, checkIfExists: true).collect()
calling_intervals_ch = Channel.fromPath(params.calling_intervals, checkIfExists: true).collect()

to:

ref_ch               = file(params.reference)
ref_index_ch         = file(params.reference_index)
ref_dict_ch          = file(params.reference_dict)
calling_intervals_ch = file(params.calling_intervals)

So let's do that.

In addition, we should remove checkIfExists: true.

vdauwera added a commit that referenced this pull request Sep 25, 2024
Based on Adam's branch at #364
@adamrtalbot adamrtalbot marked this pull request as ready for review September 25, 2024 09:11
adamrtalbot pushed a commit that referenced this pull request Sep 25, 2024
* Simplified version of Hello-GATK 

Based on Adam's branch at #364

* Intermediate scripts by section

* starter script for hello-gatk

* Replace bams with Adam's

* Add Adam's smaller reference files

* Delete hello-nextflow/data/samplesheet.csv

No longer needed

* Delete hello-nextflow/data/ref.tar.gz

replaced by Adam's ref files

* Delete hello-nextflow/data/intervals.list

reolaced by Adam's BED file

* Move ref under data (oops)

* convert fromPath().collect() to file()

For accessory file inputs

* Prettier fixes

whitespace

* addressing review

- add note about working directory
- remove collect mention which is no longer applicable and replace with informative statement
@vdauwera vdauwera marked this pull request as draft October 4, 2024 19:21
@vdauwera
Copy link
Collaborator

vdauwera commented Oct 4, 2024

Most of the changes in here were made in my recent PR (thanks @adamrtalbot for your hard work!) so this is no longer going to be used, but I'm leaving it open for now in case there's anything else we want to pilfer from it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix projectDir weirdness
4 participants