Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gracefully handle throttling #47

Open
jkallay1 opened this issue Aug 15, 2019 · 2 comments
Open

Gracefully handle throttling #47

jkallay1 opened this issue Aug 15, 2019 · 2 comments
Labels
Type: Enhancement Provide a new feature or improve an existing one

Comments

@jkallay1
Copy link

Received the error:

could not import from AWS: Error retrieving EC2 tags for CloudFront Distribution [omitted] Throttling: Rate exceeded

and got a stack trace, with no output going to the output file.

There should be some way to see all of the complete resources that have been fetched so that even with throttling all resources can be fetched over time.

@xescugc
Copy link
Member

xescugc commented Aug 23, 2019

Sorry for the delay I was on vacations 🌞

That's a good idea :).

I wanted to do that with fully support by passing a "half" state/hcl and ignoring the data already fetched for example.

The other thing is that I've improved the Throttling in a pending PR which I think I'll split to just add the better support for it directly without waiting for the full PR 😄

Right now we give a small delay when a Throttling, a handmade retry basically to be able to maybe avoid it in a while haha, but sometimes it still has it after a while of retrying.

@xescugc xescugc added the Type: Enhancement Provide a new feature or improve an existing one label Aug 23, 2019
@rquadling
Copy link

The cause of the retry is not displayed until after all retries.
So, I have to wait for 90s until I get the error.
Not great.

Also, some errors are not errors (I'm raising a separate bug report for that).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Enhancement Provide a new feature or improve an existing one
Projects
None yet
Development

No branches or pull requests

3 participants