Bring the MAS-ISO-seq workflows into our production pipelines #340
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonn-smith
force-pushed
the
kvg_pb_masseq
branch
3 times, most recently
from
626bd6d to
d013453
Compare
|
@SHuang-Broad here are the files that docker script generated. I'm not sure it produced helpful output, though. |
|
Thank @jonn-smith. |
SHuang-Broad
requested changes
Jun 29, 2022
bshifaw
approved these changes
Jul 1, 2022
docker/lr-transcript_utils/python/process_quant_files_into_tsv.py
Outdated
Show resolved
Hide resolved
SHuang-Broad
approved these changes
Jul 24, 2022
jonn-smith
force-pushed
the
kvg_pb_masseq
branch
2 times, most recently
from
d08819c to
3b4ef83
Compare
- Added required apt-get update to docker. - Added missing slash for finalizing files. - Updated `PBFlowcell.wdl` with some initial offline comments. - Fixed potential issue with transcript equivalence classes. - Updated longbow version to 0.5.29. - Updated subreads stats to reflect MAS-seq yield. - Variable name and other minor updates in `update_umi_positions.py` - Updated `Utils.GetRawReadGroup` to escape spaces as well. - Updated `AdjustUmiSequenceWithAdapterAlignment` to work with all models. - Updated lr-10x docker image to v0.1.18 - Updated Longbow WDL for new model names in the latest version. - Adjusted ram for `Longbow.Correct` task. - Fixed model names to reflect new longbow model name convention. - Updated to latest version of lr-transcript_utils. - Fixed problem with argument parsing in update_umi_positions_2.py, causing umis to not be corrected for libraries other than 5'. - Updated model segment names in script to collect better stats. Some stats are still broken. - Added jupyter notebook wdl. - Several fixes to PBMASIsoSeqQuantify.wdl for bugs. - Updated memory for StringTie2 quantify to 64gb (bandaid fix). - Removed contig filters for sharding so that no read is dropped. - Added post-umi correction tags to `tags to preserve` in the call to minimap2 when aligning reads in `PBFlowcell.wdl` - Updated `StringTie2.wdl` to version `2.2.1` - Commented out some unused code in `PBMASIsoSeqQuantify.wdl` - The MAS-seq node filter was filtering the node names by a specific PacBio instrument name, causing all reads from sequencers other than that one to not be counted as MAS-seq nodes and therefore not quantified. This was replaced with more robust logic (a regular expression). - Updated `umi_correction.py` to have variables for all tags. - Updated `umi_correction.py` to switch between pre-extracted and non-extracted data; updated WDL accordingly. - Updated `PBFlowcell.wdl` to only process certain MAS-seq outputs on single-cell data. - Updated `Longbow.wdl` to only process certain MAS-seq outputs on single-cell data. - Updated longbow to 0.5.33 to fix issues for bulk libraries. - Added in overrides for barcode correction to enable more flexibility. - `PBFlowcell` now default shards to 100 for MASSEQ. - `Longbow.Process` now default shards to 25. - Exposed an optional input in `Longbow.Process` to allow for custom shard widths. - Added in a script to pull down all monitoring logs for a run. - Addressed reviewer comments.
jonn-smith
force-pushed
the
kvg_pb_masseq
branch
from
3b4ef83 to
5dabacb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR enables MAS-seq production runs in
PBFlowcell.wdl, with a quantification pipelinePBMASIsoSeqQuantify.wdland a sample demultiplexing pipelinePBMASIsoSeqDemultiplex.wdl.I have made some substantial changes to PBFlowcell and other task WDLs. All WDLs validate and I have tested
PBFlowcellandPBMASIsoSeqQuantify. I haven't created a test dataset to run through the demultiplexing WDL, so that has not been tested yet.My tests have been done with a
5'library because it's what we used for the paper. I'm going to start testing with the other libraries, and some modifications will likely be necessary. My thoughts are to get this in and then quickly update it with whatever changes are necessary for other preps.