Automating pytest migration

[GallVp]

Hi Team

As suggested by @mashehu , I am opening this draft PR.

• Here I have tried to automate pytest to nf-test conversion as much as possible. Currently it only works for pytests for modules with a single include statement, as chaining is not needed.
• If we are planning to deprecate pytest, I don't think we should merge this as it will add unnecessary code. There are too many edge cases. I am happy to create conversion PRs on modules and take care of the edge cases along the way.
• One thing that we might want to take from here are the semi-automated power assertions with nf-core modules lint --update-assertions or such. Lots of modules fail nf-test after their creation date because md5s are added for gz files or log files. If we lint for power assertions and automate some of them such as nft-bam or nft-vcf, that will encourage good practice across modules and save people from downstream pain.

[adamrtalbot]

Why not just add flag --hard to the existing command? Or indeed --force which is more consistent with the rest of nf-core tools.

e.g. --migrate_pytest --force instead of --migrate_pytest --migrate_pytest_hard

[GallVp]

Thank you @adamrtalbot

I like your idea. I have also thought of not adding any additional flag at all. The code should try to extract and convert the pytest workflows by default, but if an exception is thrown, it should catch that, and revert back to the original migration scheme. That way, we also don't need to update the documentation.

However, the problem with this PR and the functionality it implements is that it is very fragile. I am using regex which cannot cover many edge cases. A more robust approach is to use a language parser such as pyPEG (https://stackoverflow.com/questions/6571964/which-tool-to-use-to-parse-programming-languages-in-python). But then we have plans to deprecate pytest.

[GallVp]

Update!

Since a language parser using Antlr has now been implemented for Nextflow under the nextflow/language-server project, I am using that to create a more universal pytest to nf-test converter here: https://github.com/GallVp/pytest2nf-test

The converter is in its infancy but it can already tackle far more complicated cases which were out of reach by the regex approach taken by this PR. Hopefully, we'll fast-convert all the remaining pytests soon.