Skip to content

Commit dc1b27d

Browse files
feat: use parallel version of bzip2 to decompress gisaid snapshot
I don't know if it's any faster, but why now. The results are correct in my local testing. Locally, it does use multiple threads, but not too many. We might be bound by download speed rather then decompression though. Related: #242
1 parent 69e95d1 commit dc1b27d

File tree

2 files changed

+3
-2
lines changed

2 files changed

+3
-2
lines changed

Dockerfile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,8 @@ FROM nextstrain/base:branch-python-base
3434
RUN apt-get update && apt-get install -y --no-install-recommends \
3535
python3-netifaces \
3636
time\
37-
xz-utils
37+
xz-utils \
38+
lbzip2
3839

3940
# Install Python deps
4041
RUN python3 -m pip install pipenv

bin/fetch-from-gisaid

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,4 @@ set -euo pipefail
77
curl "$GISAID_API_ENDPOINT" \
88
--user "$GISAID_USERNAME_AND_PASSWORD" \
99
--fail --silent --show-error --location-trusted --http1.1 \
10-
| bunzip2
10+
| lbzip2 -d

0 commit comments

Comments
 (0)