Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding GitHub Action for Variant Calling Test Run #7

Open
wants to merge 51 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
671a9b8
Adding initial version of variant calling test run
tefirman Dec 13, 2024
7f61538
Updating GitHub Action name
tefirman Dec 13, 2024
444329c
Updating miniconda setup action in vc-test-run
tefirman Dec 13, 2024
3f49680
Adding dummy change to trigger GitHub Action
tefirman Dec 13, 2024
34bc294
Replacing conda with apt in vc-test-run
tefirman Dec 13, 2024
7e322ba
Being more explicit about GATK
tefirman Dec 13, 2024
2e593cf
Being more explicit about vcf tab separation
tefirman Dec 13, 2024
a0744c5
Getting rid of validation step, already done with WOMtools
tefirman Dec 13, 2024
4666896
Fixing test run wdl path
tefirman Dec 13, 2024
9ec1f6f
Fixing ref dict filename
tefirman Dec 13, 2024
d5d025a
Removing ref alt from vc-test-run
tefirman Dec 13, 2024
7009dee
Being more explicit about chromosome name from vc-test-rur
tefirman Dec 13, 2024
2f201fa
Debugging statements for vc-test-run
tefirman Dec 13, 2024
140735a
Fixing chr naming for vc-test-run
tefirman Dec 13, 2024
2699eda
Adding cromwell config to vc-test-run
tefirman Dec 13, 2024
a2f18b6
Reordering config to vc-test-run
tefirman Dec 13, 2024
7116656
Improving test bam for vc-test-run
tefirman Dec 13, 2024
ac9e47e
Adding temporary debugging statements to vc WDL
tefirman Dec 13, 2024
c49377d
Adding temporary debugging statements to vc WDL
tefirman Dec 13, 2024
f8a2ab2
Reverting vc WDL
tefirman Dec 13, 2024
0ad1100
Adding debugging statements to vc-test-run
tefirman Dec 13, 2024
e09bfcf
Fixing debugging for vc-test-run
tefirman Dec 13, 2024
85c2335
Adding ref file check in vc-test-run
tefirman Dec 13, 2024
0dbfd38
Additional debugging in vc-test-run
tefirman Dec 13, 2024
0aae448
Adding verbosity for vc-test-run
tefirman Dec 13, 2024
52d8158
Adding verbosity for vc-test-run
tefirman Dec 13, 2024
727fecd
Adding verbosity for vc-test-run
tefirman Dec 13, 2024
d4a1559
Adding verbosity for vc-test-run
tefirman Dec 13, 2024
ac06bec
Adding verbosity for vc-test-run
tefirman Dec 13, 2024
72c6b1c
Fixing cromwell config for vc-test-run
tefirman Dec 13, 2024
2a77fde
Fixing cromwell config for vc-test-run
tefirman Dec 13, 2024
73b2ce6
Adding verbosity to vc-test-run
tefirman Dec 13, 2024
15a434b
Reverting WDL and massively simplifying the GitHub Action
tefirman Dec 13, 2024
d247e10
Adding debugging to vc-test-run
tefirman Dec 13, 2024
1095c85
Adding debugging to vc-test-run
tefirman Dec 13, 2024
d7e8db9
Fixing path issue in vc-test-run
tefirman Dec 13, 2024
3a6f52d
Being more explicit about ref vcfs in vc-test-run
tefirman Dec 13, 2024
b9f3d93
Adding debugging to vc-test-run
tefirman Dec 13, 2024
4e1ab75
Fixing cromwell call in vc-test-run
tefirman Dec 13, 2024
8de7e3e
Making bam file more realistic in vc-test-run
tefirman Dec 13, 2024
1e2d315
Adding vcf conversion step in vc WDL
tefirman Dec 13, 2024
8c95eab
Making bam file more realistic in vc-test-run
tefirman Dec 13, 2024
5967601
Adding debugging to vc-test-run
tefirman Dec 13, 2024
084b0cd
Fixing annovar command in vc WDL
tefirman Dec 13, 2024
75ee460
Small debugging fixes
tefirman Dec 13, 2024
a81a837
Consolidating steps in vc-test-run
tefirman Dec 13, 2024
abd0e1e
Using more realistic input bam
tefirman Dec 14, 2024
a5f64bd
Pulling more realistic bam from outside source
tefirman Dec 16, 2024
75901e7
Switching to genomic region that is known to have variants
tefirman Dec 16, 2024
d4cbea5
Switching to genomic region that is known to have variants
tefirman Dec 16, 2024
5d14754
Switching to genomic region that is known to have variants
tefirman Dec 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Adding verbosity for vc-test-run
tefirman committed Dec 13, 2024
commit 727fecdba9b322ef90a6668fcd647768f5a5fcc8
47 changes: 37 additions & 10 deletions .github/workflows/variant-calling-test-run.yml
Original file line number Diff line number Diff line change
@@ -238,9 +238,23 @@ jobs:

- name: Run test workflow
run: |
# Create directory for logs
# Create logs directory
mkdir -p logs

echo "=== Pre-execution Directory Check ==="
echo "Current directory: $(pwd)"
echo "Directory contents:"
ls -la

echo "=== Creating and checking Cromwell execution directory ==="
mkdir -p cromwell-executions
sudo chmod -R 777 cromwell-executions
ls -la cromwell-executions

echo "=== Checking test data accessibility ==="
ls -l test/data/
ls -l test/test-inputs.json

echo "=== Running Workflow ==="
java -Dconfig.file=cromwell.conf \
-Dcromwell.workflow.debug-mode=true \
@@ -255,23 +269,36 @@ jobs:
# Wait briefly to let Cromwell create initial files
sleep 10

echo "=== Checking Generated Scripts ==="
find cromwell-executions -name "script" -type f -exec sh -c 'echo "=== {} ==="; cat {}' \;
echo "=== Checking Task Directories ==="
find cromwell-executions -type d -exec sh -c 'echo "=== {} ==="; ls -la {}' \;

# Monitor for a bit longer
sleep 50
echo "=== Checking Generated Scripts ==="
find cromwell-executions -name "script" -type f -exec sh -c 'echo "=== {} ==="; cat {}; echo "=== Permissions ==="; ls -l {}' \;

# Monitor execution
for i in {1..12}; do
echo "=== Check $i ==="
echo "Process tree:"
pstree -p $(pgrep java)
echo "Directory state:"
find cromwell-executions -type f -mmin -1 -ls
sleep 5
done

# Kill Cromwell if still running
kill $CROMWELL_PID || true

echo "=== Final Process State ==="
ps aux | grep -E 'gatk|java|samtools'
echo "=== Final Directory State ==="
find cromwell-executions -type f -ls

echo "=== Script Execution Logs ==="
find cromwell-executions -name "stdout" -o -name "stderr" | while read -r log; do
echo "=== Task Logs ==="
find cromwell-executions -name "stdout" -o -name "stderr" -o -name "script" | while read -r log; do
echo "=== $log ==="
cat "$log" || echo "Empty/unreadable"
cat "$log" 2>/dev/null || echo "Empty/unreadable"
done

echo "=== Cromwell Log Tail ==="
tail -n 50 workflow.log

- name: Check outputs
run: |