Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stripe throwing 429s with high Terraform parallelism #25

Closed
oleg-codaio opened this issue Dec 11, 2019 · 3 comments · May be fixed by #68
Closed

Stripe throwing 429s with high Terraform parallelism #25

oleg-codaio opened this issue Dec 11, 2019 · 3 comments · May be fixed by #68

Comments

@oleg-codaio
Copy link
Contributor

To speed up diffs in Terraform, we run terraform plan / apply with higher parallelism using the parallelism flag. Unfortunately, that can occasionally result in errors like this from the Stripe provider:

Error: {"code":"rate_limit","status":429,"message":"Testmode request rate limit exceeded, the rate limits in testmode are lower than livemode. You can learn more about rate limits here https://stripe.com/docs/rate-limits.","type":"invalid_request_error"}

This can happen on both testmode and live instances, depending on the parallelism. Meanwhile, we're using AWS and I suppose the rate limits there are more generous, since even having 200x parallelism isn't a problem.

Is it possible to specifically set rate limits on this module? I know this might be more of a general issue with Terraform than specifically this provider (i.e., if Terraform allowed setting the parallelism on a per-module/resource basis, see hashicorp/terraform#14258), but I wanted to log it anyway. I'm not super familiar with the implementation, but maybe we could add some kind of queueing or randomized exponential retry logic to this provider.

@franckverrot
Copy link
Owner

Thanks for filing this in. If you'd like to try an implementation I would say you could go ahead. I'm down to pairing (online or IRL) but my time is extra limited right now so I am focusing on my day job.

As to:

Is it possible to specifically set rate limits on this module?

As you said, seems like a general issue with Terraform, I'm worried about adding extra complexity here making things less debuggable/maintainable.

As a matter of fact, maybe it's something Stripe implemented for large(r) customers? Have you asked them if this rate limit was configurable by any chance?

Thanks again, I'll keep this open!

@franckverrot
Copy link
Owner

@kabesan Is this what you've done here kabesan@0792516 ? Do you think we could use this?

@derekrliang
Copy link

@franckverrot can we reopen this? I'm encountering the same issue. I checked out kabesan@0792516 and seemed to fix the issue for me.

I can make a PR to merge those changes into main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants