Quantify the Download Time from a GCP Bucket

[subash-taranga]

Measuring Download Time. This snap is only for just have an idea but all files can find from
f9a8e8b
From here i will add the test, result as well as conclusion

Uploaded all logs here f9a8e8b for downloading from world wide locations as well as from Europe locations

####Creating buckets in different regions####
####In Europe we have below regions####
europe-central2(Warsaw)
europe-north1(Finland)
europe-southwest1(Madrid)
europe-west1(Belgium)
europe-west10(Berlin)
europe-west12(Turin)
europe-west2(London)
europe-west3(Frankfurt)
europe-west4(Netherlands)
europe-west6(Zurich)
europe-west8(Milan)
europe-west9(Paris)

Please note here the location is Singapore just only for an example
Further note that i have uploaded files using this way :200 MB 300MB 512MB 700MB 1GB 1.5GB 2GB 2.5GB 5GB 7.5GB 10GB
$gcloud storage cp [file name] gs://[bucket name]/

Downloading files to local machine
here use gsutil with Multi-threading & log files can be found here f9a8e8b
time gsutil -m cp -r gs://bucket-europe-north1-finland/200.txt .
time gsutil -m cp -r gs://bucket-europe-north1-finland/300.txt .
time gsutil -m cp -r gs://bucket-europe-north1-finland/512.txt .
time gsutil -m cp -r gs://bucket-europe-north1-finland/700.txt .
time gsutil -m cp -r gs://bucket-europe-north1-finland/1GB.zip .
time gsutil -m cp -r gs://bucket-europe-north1-finland/1_5GB .
time gsutil -m cp -r gs://bucket-europe-north1-finland/2GB .
time gsutil -m cp -r gs://bucket-europe-north1-finland/2_5GB .
time gsutil -m cp -r gs://bucket-europe-north1-finland/5GB .
time gsutil -m cp -r gs://bucket-europe-north1-finland/7_5GB .
time gsutil -m cp -r gs://bucket-europe-north1-finland/10GB .

Downloading files to local machine
here use gcloud storage

time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/512MB.zip.crdownload .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/1GB.zip .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/1_5GB/ .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/2GB/ .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/2_5GB/ .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/5GB/ .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/7_5GB/ .
time gcloud storage cp -r gs://nano-data-output-europe-north-1-finland/10GB/ .

After that i compare the 2 download methods gsutil Vs gcloud storage
as gcloud storage is more powerful tool for this task.
But in our case within ISP speed limit gsutil also doing better ,hence i was not able to see significant different.
logs file comparison can be seen here f9a8e8b

Concluded points for GCP Bucket Downloading
+++01+++For Single region bucket with standard storage class, no significant download time differences based on GCP bucket location. Where it doesn't matter the location is Europe or anywhere in Global. So that we can chose most cheap GCP Bucket location
+++02+++We noticed that file starting from 200MB to 10GB, download time is determine by ISP Band Width. For and example if we use 80Mbps to download 1GB file, it will take 1m 40s. Whereas if we use nearly 3 times speed ISP Band Width like 250 Mbps to download 1GB file, it will take around 30s-40s which is nearly 3 times lesser than earlier
+++03+++Based on above observation at the moment we can conclude download time depend on ISP BW

SO THE ISP INTERNET SPEED CORRELATE TO DOWNLOAD SPEED AND FINALLY DOWNLOAD SPEED CORRELATE TO DOWNLOAD TIME

IN THIS CASE ISP INTERNET SPEED IS CONSTANT(+ & - AROUND 10), HENCE DOWNLOAD SPEED IS CONSTANT. SO THE DOWNLOAD TIME IS TOTALLY DEPEND ON FILE SIZE AS SOWN IN BELOW PLOTS FROM ACTUAL TEST RESULTS

LETS MAKE SOME PLOTS FOR DOWNLOAD SPEED Vs DOWNLOAD TIME TO UNDERSTAND THIS MORE
WE CAN SEE STRONG LINEAR RELATIONSHIP
python code for seaborn plots can be find here f9a8e8b

SO THE CONCLUSION IS WHEN THE END USE ISP INTERNET SPEED IS CONSTANT AND USE STANDARD STORAGE CLASS, DOESN'T MATTER THE GCP LOCATION WHERE download cost = total GB * per GB cost
ON THE OTHER HAND , ISP SPEED CORRELATE TO DOWNLOAD SPEED. DOWNLOAD SPEED CORRELATE TO DOWNLOAD TIME AND EVENTUALLY DOWNLOAD TIME POSITIVELY CORRELATE TO GB AMOUNT
***I will create one more issue for NFS PV #30 ***