user-manual.md

Welcome to Ziggy!

Ziggy is "A Pipeline management system for Data Analysis Pipelines." This is the user guide and assorted documentation.

Table of Contents

1. Super-Basic Introduction

2. Prerequisites and System Requirements

3. Pipeline Architecture

4. Downloading and Building Ziggy

5. Configuring a Pipeline

   5.1. Pipeline Algorithms

   5.2. Python Pipeline Algorithms

   5.3. Other Pipeline Algorithms

   5.5. Algorithm Parameters

   5.4. The Datastore

   5.6. Remote Environments

   5.7. Pipelines and Nodes

6. Build System for your Pipeline

7. Running the Cluster

8. Using the Ziggy Console, Part 1

   8.1​. Starting a Pipeline

   8.2​. The Instances Panel

   8.3.​ Changing the Start and End Nodes

   8.4.​ Changing Algorithm Parameter Values

   8.5.​ Organizing Pipelines and Parameter Sets

   8.6. The Edit Pipeline Dialog Box

   8.7. The Datastore Control Panel

9. Intermediate Topics

   9.1. The Datastore and the Task Directory

   9.2. Setting up a Relational Database Management System (RDBMS)

10. Troubleshooting Pipeline Execution

    10.1​. Log Files in Ziggy

11. Using the Ziggy Console, Part 2: Troubleshooting

    11.1.​ Monitoring Tab

    11.2.​ Alerts Panel

    11.3.​ Log Files Display

    11.4.​ Re-Run or Resume a Failed Task

    11.5.​ Halting Tasks

12. High Performance Computing

    12.1. The Remote Execution Dialog Box

    12.2. HPC Cost Estimation

13. Cloud Computing (TBD)

14. Data Receipt

    14.1​. Execution Flow

    14.2.​ Data Receipt Display

15. Event Handler

    15.1​. Introduction to Event Handlers

    15.2​. Defining Event Handlers

    15.3​. Event Handler Examples

    15.4​. Sending Event Information to Algorithms

16. Reports and Performance Tracking (TBD)

17. Alternative User Interface Options

    17.1​. Command-Line Interface

18. Security (TBD)

19. Ziggy's Dustier Corners

    19.1​. [More on the Relational Database](more-rdbms.md)
    
    19.2​. [More on Parameter Sets](more-parameter-sets.md)
    
    19.3​. [Parameter Overrides](parameter-overrides.md)
    
    19.4​. [Redefining a Pipeline](redefine-pipeline.md)
    
    19.5. [Creating Ziggy Nicknames](nicknames.md)
    
    19.6. [Advanced Unit of Work Configurations](advanced-uow.md)
    
    19.7. [Software Version Tracking](version-tracking.md)
    
    <!-- 19.8. [Customizing Ziggy](customizing-ziggy.md) -->
    

20. Contact Us

Appendix A. The Properties File