Skip to content

Commit

Permalink
Move data to brick dir
Browse files Browse the repository at this point in the history
  • Loading branch information
john-shaffer committed Dec 27, 2023
1 parent da0ef10 commit 819c11c
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 7 deletions.
9 changes: 5 additions & 4 deletions dvc.lock
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
schema: '2.0'
stages:
to-parquet:
cmd: clj -X biobricks.ecotox/build-parquet '{:db "data/ecotox_ascii" :output "data/ecotox_parquet"}'
cmd: clj -X biobricks.ecotox/build-parquet '{:db "data/ecotox_ascii" :output "brick"}'
deps:
- path: data/ecotox_ascii
md5: 3c1d6016fd057c38a9d5690fcf300c6a.dir
size: 2322245817
nfiles: 51
outs:
- path: data/ecotox_parquet
md5: 4debb30e4bf026e8d00226726d7a0c5c.dir
size: 3768141088
- path: brick
hash: md5
md5: a64c66ade4f0eefc2651bf7b03fb3d79.dir
size: 3768405317
nfiles: 48
4 changes: 2 additions & 2 deletions dvc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
# see https://dvc.org/doc/user-guide/project-structure/dvcyaml-files#pipelines-files-dvcyaml
stages:
to-parquet:
cmd: clj -X biobricks.ecotox/build-parquet '{:db "data/ecotox_ascii" :output "data/ecotox_parquet"}'
cmd: clj -X biobricks.ecotox/build-parquet '{:db "data/ecotox_ascii" :output "brick"}'
deps:
- data/ecotox_ascii
outs:
- data/ecotox_parquet
- brick
2 changes: 1 addition & 1 deletion src/biobricks/ecotox.clj
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
(write-parquet parquet-file rows))))

(comment
(let [out (fs/path "data" "ecotox_parquet")]
(let [out (fs/path "brick")]
(fs/delete-tree out)
(build-parquet {:db "data/ecotox_ascii" :output (str out)}))
)

0 comments on commit 819c11c

Please sign in to comment.