include databases in release?

[alexg9010]

The databases used in the tests/settings.yaml are referencing paths on our cluster:
https://github.com/BIMSBbioinfo/pigx_sarscov2_ww/blob/c2044e640766f35fb2bd38a93a1aef4dde382412/tests/settings.yaml#L8-L11

Thus there are two options to solve this issue:

• a) improve input validation to catch missing (or wrong path to) databases and inform the user about the download_databases.sh script. This would leave the downloading on the user's side but would allow us to keep the release tarball small.

• b) check if it is possible to provide minimal versions of the databases with the pipeline release:

  • use the small (0.5 GB) viral kraken2 database (https://benlangmead.github.io/aws-indexes/k2)
  • figure out whether the Taxonomy database of Krona is sufficient (https://github.com/marbl/Krona/wiki/Installing#notes-for-database-updates)

[rekado]

The primary goal of the big test is to ensure that we've got all files in the tarball, that all applications run, and that the pipeline runs to completion. The results are not all that important, because for each of the features there should be a separate validation test anyway.

[jonasfreimuth]

I would consider this resolved once we are able to get distcheck working, which will should be possible with the changes on here (which boil down to having extra rules to download databases if they are not already at the specified locations), and exposing the small database archives I've built on bimsbstatic.