-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
adding comparision log files gsutil vs gcloud storage & bucket downlo…
…ad time logs for #22
- Loading branch information
1 parent
d14f1b1
commit 13d8812
Showing
13 changed files
with
991 additions
and
156 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/512MB.zip.crdownload . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/512MB.zip.crdownload... | ||
| - [1/1 files][512.0 MiB/512.0 MiB] 100% Done 10.5 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/512.0 MiB. | ||
|
|
||
| real 0m53.731s | ||
| user 0m7.157s | ||
| sys 0m4.560s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/1_5GB/ . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/1_5GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/1_5GB/512MB.zip.crdownload... | ||
| - [2/2 files][ 1.5 GiB/ 1.5 GiB] 100% Done 10.2 MiB/s ETA 00:00:00 | ||
| Operation completed over 2 objects/1.5 GiB. | ||
|
|
||
| real 2m27.596s | ||
| user 0m20.662s | ||
| sys 0m11.477s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/1GB.zip . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/1GB.zip... | ||
| \ [1/1 files][ 1.0 GiB/ 1.0 GiB] 100% Done 10.0 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/1.0 GiB. | ||
|
|
||
| real 1m40.784s | ||
| user 0m12.795s | ||
| sys 0m8.000s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/2GB/ . | ||
|
|
||
| Copying gs://nano-data-output-asia-southeast-1-singapore/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/2GB/1GB/1GB.zip... | ||
| - [2/2 files][ 2.0 GiB/ 2.0 GiB] 100% Done 11.6 MiB/s ETA 00:00:00 | ||
| Operation completed over 2 objects/2.0 GiB. | ||
|
|
||
| real 3m15.546s | ||
| user 0m25.954s | ||
| sys 0m14.656s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/2_5GB/ . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/2_5GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/2_5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/2_5GB/512MB.zip.crdownload... | ||
| - [3/3 files][ 2.5 GiB/ 2.5 GiB] 100% Done 10.4 MiB/s ETA 00:00:00 | ||
| Operation completed over 3 objects/2.5 GiB. | ||
|
|
||
| real 4m3.119s | ||
| user 0m33.807s | ||
| sys 0m18.005s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/5GB/ . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/5GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/5GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/5GB/2GB/2GB/1GB/1GB.zip... | ||
| \ [5/5 files][ 5.0 GiB/ 5.0 GiB] 100% Done 11.0 MiB/s ETA 00:00:00 | ||
| Operation completed over 5 objects/5.0 GiB. | ||
|
|
||
| real 7m57.676s | ||
| user 1m12.466s | ||
| sys 0m33.232s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/7_5GB/ . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/2.5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/2.5GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/2.5GB/512MB.zip.crdownload... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/5GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/5GB/2GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/7_5GB/5GB/2GB/1GB.zip... | ||
| | [8/8 files][ 7.5 GiB/ 7.5 GiB] 100% Done 11.0 MiB/s ETA 00:00:00 | ||
| Operation completed over 8 objects/7.5 GiB. | ||
|
|
||
| real 12m16.677s | ||
| user 2m12.451s | ||
| sys 0m54.926s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ time gsutil -m cp -r gs://nano-data-output-asia-southeast-1-singapore/10GB/ . | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/2GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/5GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/5GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://nano-data-output-asia-southeast-1-singapore/10GB/5GB/5GB/2GB/2GB/1GB/1GB.zip... | ||
| | [10/10 files][ 10.0 GiB/ 10.0 GiB] 100% Done 10.9 MiB/s ETA 00:00:00 | ||
| Operation completed over 10 objects/10.0 GiB. | ||
|
|
||
| real 16m8.564s | ||
| user 3m8.747s | ||
| sys 1m17.473s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ |
102 changes: 102 additions & 0 deletions
102
log-bkt-download-time/bucket-europe-central2-warsaw.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| Copying gs://bucket-europe-central2-warsaw/200.txt... | ||
| \ [1/1 files][200.0 MiB/200.0 MiB] 100% Done 9.0 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/200.0 MiB. | ||
|
|
||
| real 0m21.573s | ||
| user 0m3.791s | ||
| sys 0m2.011s | ||
| Copying gs://bucket-europe-central2-warsaw/300.txt... | ||
| \ [1/1 files][300.0 MiB/300.0 MiB] 100% Done 7.9 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/300.0 MiB. | ||
|
|
||
| real 0m31.690s | ||
| user 0m4.839s | ||
| sys 0m2.442s | ||
| Copying gs://bucket-europe-central2-warsaw/512.txt... | ||
| - [1/1 files][512.0 MiB/512.0 MiB] 100% Done 7.7 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/512.0 MiB. | ||
|
|
||
| real 0m51.643s | ||
| user 0m8.841s | ||
| sys 0m4.930s | ||
| Copying gs://bucket-europe-central2-warsaw/700.txt... | ||
| | [1/1 files][700.0 MiB/700.0 MiB] 100% Done 8.8 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/700.0 MiB. | ||
|
|
||
| real 1m6.898s | ||
| user 0m9.864s | ||
| sys 0m4.382s | ||
| Copying gs://bucket-europe-central2-warsaw/1GB.zip... | ||
| - [1/1 files][ 1.0 GiB/ 1.0 GiB] 100% Done 10.1 MiB/s ETA 00:00:00 | ||
| Operation completed over 1 objects/1.0 GiB. | ||
|
|
||
| real 1m38.031s | ||
| user 0m13.176s | ||
| sys 0m7.409s | ||
| Copying gs://bucket-europe-central2-warsaw/1_5GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/1_5GB/512MB.zip.crdownload... | ||
| / [2/2 files][ 1.5 GiB/ 1.5 GiB] 100% Done 7.5 MiB/s ETA 00:00:00 | ||
| Operation completed over 2 objects/1.5 GiB. | ||
|
|
||
| real 2m25.607s | ||
| user 0m18.388s | ||
| sys 0m9.188s | ||
| Copying gs://bucket-europe-central2-warsaw/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/2GB/1GB/1GB.zip... | ||
| \ [2/2 files][ 2.0 GiB/ 2.0 GiB] 100% Done 11.0 MiB/s ETA 00:00:00 | ||
| Operation completed over 2 objects/2.0 GiB. | ||
|
|
||
| real 3m8.967s | ||
| user 0m26.759s | ||
| sys 0m13.706s | ||
| Copying gs://bucket-europe-central2-warsaw/2_5GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/2_5GB/512MB.zip.crdownload... | ||
| Copying gs://bucket-europe-central2-warsaw/2_5GB/2GB/1GB/1GB.zip... | ||
| - [3/3 files][ 2.5 GiB/ 2.5 GiB] 100% Done 8.6 MiB/s ETA 00:00:00 | ||
| Operation completed over 3 objects/2.5 GiB. | ||
|
|
||
| real 4m0.371s | ||
| user 0m32.946s | ||
| sys 0m17.103s | ||
| Copying gs://bucket-europe-central2-warsaw/5GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/5GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/5GB/2GB/2GB/1GB/1GB.zip... | ||
| \ [5/5 files][ 5.0 GiB/ 5.0 GiB] 100% Done 9.6 MiB/s ETA 00:00:00 | ||
| Operation completed over 5 objects/5.0 GiB. | ||
|
|
||
| real 7m56.219s | ||
| user 1m11.586s | ||
| sys 0m31.992s | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/2.5GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/2.5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/2.5GB/512MB.zip.crdownload... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/5GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/5GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/5GB/2GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/7_5GB/5GB/2GB/2GB/1GB.zip... | ||
| | [8/8 files][ 7.5 GiB/ 7.5 GiB] 100% Done 10.4 MiB/s ETA 00:00:00 | ||
| Operation completed over 8 objects/7.5 GiB. | ||
|
|
||
| real 11m50.116s | ||
| user 1m54.695s | ||
| sys 0m49.072s | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/2GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/5GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/5GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/5GB/2GB/2GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/5GB/2GB/2GB/1GB/1GB.zip... | ||
| Copying gs://bucket-europe-central2-warsaw/10GB/5GB/5GB/2GB/1GB/1GB.zip... | ||
| | [10/10 files][ 10.0 GiB/ 10.0 GiB] 100% Done 10.6 MiB/s ETA 00:00:00 | ||
| Operation completed over 10 objects/10.0 GiB. | ||
|
|
||
| real 15m43.523s | ||
| user 2m36.899s | ||
| sys 1m5.150s | ||
| subash@MSI:/mnt/c/SUBASH/G-CLOUD/BUCKET$ |
Oops, something went wrong.