[discussion]: How will the data flow through GenomeDK?

[martonvago]

It could be useful to specify how exactly data will move through GenomeDK.

Maybe something like:

1. Sent to GenomeDK as a file of some format

   data.xyz (cannot leave GenomeDK)

2. [shear] Transformed to a data frame

   variable (cannot leave GenomeDK)

3. [sprout] Properties extracted and edited

   properties.py, datapackage.json (committed and synced to GitHub)

4. [sprout] Data frame saved as batch Parquet file

   batch.parquet (cannot leave GenomeDK)

5. [sprout] Batch Parquet files merged

   data.parquet (cannot leave GenomeDK)