lrp_3.0.7

it looks like Sniffles output were accidentally turned off in ONTWholeGenome (#247)

lrp_3.0.6

Add info on multiplexed ONT runs to Terra sample table (#232)

• Added support for multiplexed Nanopore runs

lrp_3.0.5

created dockerfile for winnowmap2 installation (#239)

lrp_3.0.4

Add the flowcell id to the PacBio sample table (#230)

• Bug fixes for automatic population and updates to Terra tables.

1. Fixed an issue where, when updating rows that include manual edits from Terra, an inconsistent sort order caused values to be randomly assigned.
2. While sample tables can be changed by simply uploading a new version, the sample set membership table seems to just add redundant entries, potentially causing downstream problems. Fixed by only uploading the changed membership entries, rather than the full table.
3. When manual edits in the sample table cause sample_set entries to be invalidated, remove the invalid sample_set entries.

• More automation updates.

1. Updated update_nanopore_tables to allow for table editing and sample_set invalidation (similar to update_pacbio_tables script).
2. Added a verbosity argument to workflow automator.

• Add fastq_pass and fast5_pass dirs to Nanopore sample table in Terra

• Fixes for replacing nan values

• Add flowcell id (CellPac.Barcode) to Terra table

lrp_3.0.3

Bug fixes for automatic population and updates to Terra tables. (#226)

• Bug fixes for automatic population and updates to Terra tables.

1. Fixed an issue where, when updating rows that include manual edits from Terra, an inconsistent sort order caused values to be randomly assigned.
2. While sample tables can be changed by simply uploading a new version, the sample set membership table seems to just add redundant entries, potentially causing downstream problems. Fixed by only uploading the changed membership entries, rather than the full table.
3. When manual edits in the sample table cause sample_set entries to be invalidated, remove the invalid sample_set entries.

• More automation updates.

1. Updated update_nanopore_tables to allow for table editing and sample_set invalidation (similar to update_pacbio_tables script).
2. Added a verbosity argument to workflow automator.

• Add fastq_pass and fast5_pass dirs to Nanopore sample table in Terra

• Fixes for replacing nan values

lrp_3.0.2

Relax Longshot task execution stringency (#234)

lrp_3.0.1

Release of 3.0 pipeline (#224)

This marks the release of the 3.0.* workflow series, intended to massively increase the usability of these workflows in Terra. This release, therefore, contains substantial, breaking changes from the 2.1.* workflow series.

First, processing is now more logically divided between flowcell, sample, and cohort levels. The majority of workflows are now intended to run in series - e.g. first flowcell-level processing, followed by sample-level processing (which presumes that flowcell-level outputs are sample-level inputs). Future improvements are dependent on this assumed behavior.

Second, all top-level workflows are now expected to return outputs that are compatible with Terra’s data model. To retain control over directory structure, we provide functionality for tasks to copy files to a destination bucket and return that storage path to the data model.

Finally, we’ve made many improvements to the workflows themselves. Those changes include:

PB Sequel II/IIe:

• Updated to CCS 6.0.0 (so that Sequel II data will be compatible with Sequel IIe on-board basecalling).
• Added extracthifi tool to subset CCS reads to only rq >= 0.99 reads, so that all subsequent tasks in Sequel II and Sequel IIe CCS datasets are processed on reads in the same way.

ONT Basecalling

• Upgraded to Guppy 4.5.4
• Improved basecalling on datasets where the sequencing_summary.txt and final_summary.txt files are not present
• Added support for demultiplexing with standard barcoding kits. Because basecalling can be single-plex or multiplexed, its output can technically be 1-to-1 or 1-to-many. That severely breaks compatibility with Terra's data model. Thus, we now only return the directory to which the results are being written. We will rely on scripts that scan input directories to populate the data model instead.

Assembly

• Added Canu assembly workflows
• Added Flye assembly workflows
• Added Hifiasm assembly workflows
• Added Quast to assembly workflows for automatic assembly performance reports
• Made Quast comparison against a reference sequence optional

Variant calling

• Added variant calling subworkflows for PBCCS, PBCLR, and ONT data
• Added DeepVariant-PEPPER to ONT calling
• Parallelized most variant calling tasks
• Allow variant calling to be turned off for whole genome workflows (useful if one wants to make CNV calls on the BAMs and doesn't need the SNV/SV calls immediately).

QC

• Added workflow to verify a BAM file against a sample fingerprint
• Replace the mosdepth Docker image to one with a newer version
• Added tools to compute yield stats based on PacBio .pbi files
• Fixes to IsoSeq output filenaming

lrp_2.1.57

Added an option to merge basecalled fastqs into a single file (#217)

• Added an option to merge basecalled fastqs into a single file

• Added output section to Guppy wdl

lrp_2.1.56

update workflow exe options to use new default exe bucket now that we have our own cromwell server (#211)

lrp_2.1.55

Optionally extract uncorrected data (#207)

• Added tool to extract uncorrected reads to a separate BAM file