[FEATURE] Recover on download failures #844
Comments
ksafonov-db
added a commit
to ksafonov-db/databricks-sdk-py
that referenced
this issue
Jan 8, 2025
ksafonov-db
added a commit
to ksafonov-db/databricks-sdk-py
that referenced
this issue
Jan 8, 2025
Signed-off-by: Kirill Safonov <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this issue
Jan 8, 2025
## What changes are proposed in this pull request? 1. Extending Files API client to support resuming download on failures. New implementation tracks current offset in the input stream and issues a new download request from this point in case of an error. 2. New code path is enabled by 'DATABRICKS_ENABLE_EXPERIMENTAL_FILES_API_CLIENT' config parameter. ## How is this tested? Added unit tests for the new code path: `% python3 -m pytest tests/test_files.py` --------- Signed-off-by: Kirill Safonov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem Statement
Currently download using Files API client fails on network error or server timeout.
Proposed Solution
Make Files API client automatically re-connect to the server and resume the download in case of temporary failures.
The text was updated successfully, but these errors were encountered: