New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

listdir limited to 1000 entries #21

Open

SnapshotCiTy opened this issue Jan 31, 2017 · 1 comment

SnapshotCiTy commented Jan 31, 2017

Simple S3 is liiting the output to 1000 file when using listdir.

http://stackoverflow.com/questions/40331681/python-simples3-listdir-limit-the-output-to-1000-files

Any way / plan to make it support more than 1000 - or an unlimited number of entries?

Owner

lericson commented Jan 31, 2017

I'll copy my response here for posterity

Author of simples3 here. I'm sorry this stopped working, but that's really on Amazon. As you can see in the implementation of S3Bucket.listdir there is support for pagination and it should be seamless. If it isn't, then that's because Amazon stopped returning the truncated flag or stopped returning markers to perform pagination.

You're welcome to do the job, but I have to question Amazon on this decision. Why break existing APIs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment