Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZIP + afsc GCS fails to detect file if a key prefix of the same name exists #19

Open
dchoi-specificmedia opened this issue Apr 5, 2023 · 0 comments

Comments

@dchoi-specificmedia
Copy link
Contributor

dchoi-specificmedia commented Apr 5, 2023

GCS seems to support both files AND directories (prefixes) with the same string.
For example, gs://test_bucket/test_file.txt and gs://test_bucket/test_file.txt/sub_file.txt are both valid GCS objects.

In the case this occurs, it looks like the ZIP detection isn't handling this correctly.
Specifically, github.com/viant/afsc/gs.(s *storager).List will return multiple files in the case that both a file and a prefix exist at the same path, and github.com/viant/afs/zip.newStorager() checks to make sure the List only returns one file https://github.com/viant/afs/blob/master/zip/storager.go#L241.

The currently coded behavior seems to work on the assumption that a file and a directory cannot be the same name (which is valid).

This may also coincide with the way that afsc/gs is handling the List method, as it seems to assume that the URL provided can be a prefix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant