Skip to content

Remove storage entry if resumable storage has been enabled and the stored URL has been voided #105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 2, 2025

Conversation

pandabear
Copy link
Contributor

@pandabear pandabear commented May 22, 2025

This pull request aims to resolve the issue #103 .

This change introduces a logic handler when initializing a new Uploader instance with storage-enabled to not only clean up voided storage key entry, but also make the usage more user-friendly as no extra effort is required to initialize a resumable feature. tus-js-client was referenced https://github.com/tus/tus-js-client/blob/bef505f2fb751095af06e3a9bdf62384df136a7a/lib/upload.ts#L697-L705 to determine how this case is handled. Otherwise retain the same logic as before.

  • Add a test case that simulates the issue
  • Implement try-except special handling case to prevent voided storage key entry from causing errors during Uploader-instance initialization

@Acconut
Copy link
Member

Acconut commented May 23, 2025

Thank you very much for opening this PR!

@pandabear pandabear force-pushed the url-storage-voided-entry-handler branch from 7b61530 to 314c06b Compare May 26, 2025 07:23
Copy link
Member

@Acconut Acconut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much. The test logic looks good now, but CI is failing on Windows due to PermissionError: [Errno 13] Permission denied: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpevpkhklg'. Can you please have a look at this?

@jukuisma
Copy link

jukuisma commented Jun 2, 2025

PermissionError: [Errno 13] Permission denied: 'C:\Users\RUNNER~1\AppData\Local\Temp\tmpevpkhklg'

https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile /windows seems related.

@pandabear pandabear force-pushed the url-storage-voided-entry-handler branch from 314c06b to 997d60a Compare June 2, 2025 12:21
@pandabear
Copy link
Contributor Author

Thank you very much. The test logic looks good now, but CI is failing on Windows due to PermissionError: [Errno 13] Permission denied: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpevpkhklg'. Can you please have a look at this?

I noticed earlier with named temporary folders. Perhaps it's the use of context manager that messes up with windows logic. 3.12 introduces delete_on_close parameter, but that won't be much help for earlier Windows python versions.

Nevertheless, I updated the test to not rely on context manager. If this also fails then only thing left is to create a file during test runtime within tests-folder, like how it was done in test_filestorage.py -module.

@Acconut
Copy link
Member

Acconut commented Jun 2, 2025

I just ran CI tests again, but unfortunately it didn't seem to help so far.

@pandabear pandabear force-pushed the url-storage-voided-entry-handler branch from 997d60a to d4a5e1b Compare June 2, 2025 12:29
@pandabear
Copy link
Contributor Author

pandabear commented Jun 2, 2025

I just ran CI tests again, but unfortunately it didn't seem to help so far.

My last attempt before I ditch tempfile. My guess it's the storagethat was occupying the temporary file.

@Matoking
Copy link
Contributor

Matoking commented Jun 2, 2025

NamedTemporaryFile(delete=True, delete_on_close=False) works nicely on Wine, while reproducing the permission error with delete_on_close=True. Something to do with deletion rights that tempfile tries to request by default upon opening the file?

python/cpython#88221 (comment)

But as @pandabear stated, delete_on_close is only available on Python 3.12 and up, so probably not an option here...

Copy link
Member

@Acconut Acconut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Looks all good now!

@Acconut Acconut merged commit e786fa8 into tus:main Jun 2, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

If url_storage is enabled, BaseUploader should remove file entry upon certain failures during get_offset()
4 participants