First nf-core/sopa release

[quentinblampey]

New release PR after recent updates (moving processes to modules, and using -params-file instead of --configfile).

The conda tests are broken for two profiles because some processes require the installation of a specific command-line tool. It works well using docker or singularity though. Do you know how I could fix this? Or maybe prevent using conda for some specific profiles (e.g., baysor and proseg).

EDIT: actually, there is also an issue with singularity on test_cellpose. This is due to a permission issue to write in the HOME directory. The lines below fixed the issue for Docker, but apparently not for Singularity...

withName: PATCH_SEGMENTATION_CELLPOSE {
    containerOptions = ''
}

Happy to hear from your review @mashehu @kenibrewer!

PR checklist

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the pipeline conventions in the contribution docs
• If necessary, also make a PR on the nf-core/sopa branch on the nf-core/test-datasets repository.
• Make sure your code lints (nf-core pipelines lint).
• Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
• Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
• Usage Documentation in docs/usage.md is updated.
• Output Documentation in docs/output.md is updated.
• CHANGELOG.md is updated.
• README.md is updated (including new tool citations and authors/contributors).

[quentinblampey]

I made some minor updates (added cell type annotation in the tests)

Also, thanks to @kenibrewer and #9, the "Files changed" are now cleaner since it doesn't contain the changes related to the TEMPLATE

[quentinblampey]

Hello, any update about the review @kenibrewer @mashehu? Happy to hear your feedback!

[mashehu]

sorry, was on vacation and parental leave, will try to come back to this this week.

[quentinblampey]

No problem @mashehu, congratulations, and I hope you had a nice parental break and holidays!

[quentinblampey]

Reverting the containerOptions indeed fixed it for Docker, but not for singularity, see here.

I couldn't find any equivalent to this page but for singularity

[quentinblampey]

I finally solved the singularity issue! For some context: cellpose uses a cache which is located in the HOME dir by default, which was not writable. I tried a bunch of different containerOptions but couldn't make it work with singularity. I ended up exporting a specific location for the cache directory directly in my process:
export CELLPOSE_LOCAL_MODELS_PATH=./cellpose_cache

I don't know if this is the cleanest way, though. I had a similar issue earlier with numba, but I fixed it in a different manner, i.e., adding this line directly in the Dockerfile:
ENV NUMBA_CACHE_DIR=/tmp/numba_cache
It worked well also, but I wonder if it's important to solve it the same way in both cases.

What do you think is best? Is it fine to keep it as is?

[mashehu]

yes, I would recommend to the same thing (and to not hard-code /tmp)

[quentinblampey]

Ok, thanks @mashehu. Cellpose is used in only one process, so it's quite easy to add the export in the corresponding script, but the numba issue will arise in every single process I defined. So should I add the export NUMBA_CACHE_DIR=... for every script, or do you have a cleaner way to do that?
I think using containerOptions = "--env NUMBA_CACHE_DIR=..." may not work since it was not working in the case of cellpose