CTX-4070: Streaming download #19

VukManojlovic · 2023-06-27T15:38:25Z

No description provided.

igorperic17 · 2023-06-29T14:40:03Z

@VukManojlovic @dule1322 what's the status of this one?

dule1322 · 2023-06-30T08:35:01Z

@igorperic17 This one remains in draft until backend starts sending the "Content-Length" header on "session/export" endpoint.

igorperic17 · 2023-07-03T06:37:09Z

What's the ETA on the backend work?

dule1322 · 2023-07-03T08:08:29Z

@igorperic17 The task was in CR, it is merged now so @VukManojlovic can make necessary modifications to this PR:

Update it to use streaming download only for session/export endpoint
Compare value of "Content-Size" header field with the file locally to re-download the file it it was not fully downloaded (corrupted)

dule1322 · 2023-07-03T08:09:29Z

Update: It is merged but the backend build is not passing, so it is still not in dev environment.

…implemented checking for downloaded file corruption (if file size does not match expected size)

* develop: CTX-4079: Redesigned terminate condition for experiment worker process - Added log file output for experiment worker process for debugging purposes CTX-4251: Update "model-queue/custom" to "run" endpoint Version bump CTX-4219: Make root not private to make it available outside the class - Name starts with an underscore to mark it as internally used don't touch CTX-4083: Moved heartbeat to the metrics worker process - Renamed metrics worker process to experiment worker Version bump CTX-4102: Fix terminating metrics process on experiment process kill CTX-4166: Implemented Sequence sample import CTX-3890: Changed division to // CTX-3890: Converted wider code to be compatible with int based bounding boxes CTX-3890: Changed width and height variables from float to int Update README.md

coretex/networking/network_manager_base.py

coretex/networking/requests_manager.py

coretex/networking/network_response.py

…eaming download

coretex/networking/network_manager_base.py

Vuk Manojlovic added 2 commits June 27, 2023 17:34

CTX-4070: Changed dataset download to streaming

e0d9b0d

CTX-4070: Updated streamingDownload function

0c4d070

VukManojlovic temporarily deployed to develop June 27, 2023 15:38 — with GitHub Actions Inactive

VukManojlovic marked this pull request as draft June 28, 2023 08:37

Vuk Manojlovic added 2 commits July 11, 2023 17:25

CTX-4070: Made is so streaming download is only used for samples and …

620fac0

…implemented checking for downloaded file corruption (if file size does not match expected size)

VukManojlovic temporarily deployed to develop July 11, 2023 15:26 — with GitHub Actions Inactive

CTX-4070: Minor bug fix

028e895

VukManojlovic marked this pull request as ready for review July 11, 2023 15:28

VukManojlovic temporarily deployed to develop July 11, 2023 15:28 — with GitHub Actions Inactive