Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

44 same json use by multiple tasks #99

Merged
merged 9 commits into from
Feb 11, 2025
Merged
66 changes: 66 additions & 0 deletions jsonTaskOrderTest/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Unit Test for JSON Input Handling & Task Order (jsonTaskOrderTest)

## Overview

The `jsonTaskOrderTest` workflow is a unit test designed to validate the behavior of JSON input handling in WDL (Workflow Description Language). It ensures that a JSON input string is correctly passed and utilized by multiple tasks while maintaining the correct execution order.
tefirman marked this conversation as resolved.
Show resolved Hide resolved

## Purpose

This workflow serves as a test case for:
- Ensuring that the same JSON input is correctly used by multiple tasks.
- Verifying that task execution order is strictly maintained.
- Ensuring that Task 2 does not execute before Task 1 completes.
- Validating proper dependency resolution between tasks.

## Workflow Components

### Workflow: `JsonOrderTest`

This workflow consists of two tasks designed to test JSON input sharing and execution order.

#### Inputs:
- `input_json`: A JSON string used as input by both tasks.

#### Outputs:
- `task1_output`: Metadata file output from the `Task1` task.
- `task2_output`: Metadata file output from the `Task2` task.

### Tasks

#### Task: `Task1`
- **Purpose**: Reads and processes the JSON input.
- **Operation**:
- Writes the JSON input to an output file.
- Confirms Task 1 execution by appending a success message.

#### Task: `Task2`
- **Purpose**: Uses the same JSON input but depends on `Task1`.
- **Operation**:
- Waits for `Task1` to complete.
- Reads the same JSON input and appends `Task1`'s output to its own result file.

### Expected Output Files

- **Task1 Output (`task1_output.txt`)**
```
Processing JSON in Task1: {"I am the text that from input.json"}
Task1 completed
```

- **Task2 Output (`task2_output.txt`)**
```
Processing JSON in Task2: {"I am the text that from input.json"}
Task2 completed after Task1
Processing JSON in Task1: {"I am the text that from input.json"}
Task1 completed
```

## Version
- **WDL Version**: 1.0

## Additional Notes
- This workflow demonstrates how multiple tasks can share the same JSON input while maintaining execution order.
- The `Task2` execution is dependent on `Task1` completing successfully.
- The unit test confirms that JSON input is processed consistently across tasks.
- Any deviation in execution order or missing outputs indicates a failure in dependency resolution.

3 changes: 3 additions & 0 deletions jsonTaskOrderTest/inputs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"jsonTaskOrderTest.input_json": "I am the text that from input.json"
}
57 changes: 57 additions & 0 deletions jsonTaskOrderTest/jsonTaskOrderTest.wdl
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
version 1.0

workflow jsonTaskOrderTest {
input {
String input_json # JSON string used as input for both tasks
}

call Task1 { input: input_json = input_json }
call Task2 { input: input_json = input_json, previous_output = Task1.output_file }

output {
File task1_output = Task1.output_file
File task2_output = Task2.output_file
}
}

task Task1 {
input {
String input_json
}

command <<<
echo "Processing JSON in Task1: ~{input_json}" > task1_output.txt
echo "Task1 completed" >> task1_output.txt
>>>

output {
File output_file = "task1_output.txt"
}

runtime {
cpu: 1
memory: "2G"
}
}

task Task2 {
input {
String input_json
File previous_output
}

command <<<
echo "Processing JSON in Task2: ~{input_json}" > task2_output.txt
echo "Task2 completed after Task1" >> task2_output.txt
cat ~{previous_output} >> task2_output.txt
>>>

output {
File output_file = "task2_output.txt"
}

runtime {
cpu: 1
memory: "2G"
}
}
5 changes: 5 additions & 0 deletions jsonTaskOrderTest/options.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"workflow_failure_mode": "ContinueWhilePossible",
"write_to_cache": false,
"read_from_cache": false
}
Loading