Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix for continuation behaviour on broken dataset archives due to starving download connections via HTTP-GET #6380

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix for continuation behaviour on broken dataset archives due to starving download connections via HTTP-GET #6380

wants to merge 1 commit into from

Commits on Oct 24, 2023

  1. Adding a retry mechanic for HTTP-GET downloads, which adds the follow… 

    …ing behaviour:

- Download Progress Thread checks for download size validity in case the HTTP connection starves mid download. If the check fails, a RuntimeError is thrown
- Cache Downloader code with retry mechanic monitors for an exception thrown by the download progress thread, and retries download with updated `resume_size`.
- Cache Downloader will not mark incomplete files which have thrown an exception during download, and exceeded retries, as complete.
    @RuntimeRacer
    RuntimeRacer committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    22cf86f View commit details
    Browse the repository at this point in the history