-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve Crates.io rate limiting response display #9119
Conversation
In rust-lang/crates.io#1596 it was added a rate limit for crate publishing endpoint connections (10 components/min). As is said in the PR. The strategy used allows to upload 10 components first and then the ratio starts to be applied. Now, if the rate is surpassed, crates.io returns a 429 HTTP response. Cargo, tries to publish all of the workspace/crate components as fast as possible (as we should expect) but when we have more than 10 components and we try to publish the 11th, usually a minute hasn't passed yet and therefore we get a HTTP-429. This made some users to experience confusing/not well explained error messages as seen in rust-lang/crates.io#1643. The limit got increased to 30 components/min. But who knows.. So this closes rust-lang#6714 while a better solution is not found.
r? @Eh2406 (rust-highfive has picked a reviewer for you, use r? to override) |
I don't know if hard coding this error message is a good plan. For one thing what happens if a user is publishing to an alternative registry? ( But the next message is also only for crates.io. So maybe I don't have all the context.) For another thing crates.io may change its policy, and so this won't be correct. Is there context I am missing? Or, can we make a more generic message? Or, can we use some data from the response? |
Hi @Eh2406 thanks for the feedback.
In respect of the context I think @carols10cents is the one who ask for. She'll know much better than me (I just tried to fix the issue and read across some of the issues and PRs related.
You're right on the crates.io check and the alternative registry. Added
In order to be tight to their policy, and be able to respond to any changes, my suggestion is to import
The error is quite specific indeed. 429 isn't a typical HTTP response code, so I don't see how we can make the error more generic.
The response does not provide any useful info AFAIK as you can see in rust-lang/crates.io#1643 |
even if
Maybe we can get crates.io to include your text in the body of the message, or some metadata in the headers. If one day Cargo is going to respond to 429 by retrying, then we will need a According to MDN it is for "Too Many Requests". So a message like "Looks like you are making too many requests of the registry. Please slow down to give the registry servers time to process. Check your registries documentation for what rates are acceptable." and if it is crates.io add " crates.io has its relevant documentation at ___" may work. |
Hi @Eh2406. Thanks a lot for the suggestions. About the import you're completely right.
That's true, and indeed good idea. Since Cargo (at least now) publishes as fast as possible, we can include this generic error message here and attach the crates.io one if at some point they implement one (I'll try to do so). In the relevant documentation section, what would you add? Since there's no mention to the maximum rate anywhere aside from the issues. So I guess that the crates.io GH repo link? Since the Cargo book doesn't have much stuff about it. Thanks! |
It would be great if they could document it someplace better then the issues. https://crates.io/policies feels like a good place, we could start with a PR to change that page as its source is just here I think. |
Filed rust-lang/crates.io#3229. Once it's closed and we know which link can we refer to, I think we're good to go and address your suggestions 😃 |
As it was said in rust-lang/crates.io#3229 we can close this PR since the error is being returned from crates.io now in the headers and we already display them. So closing this. @Eh2406 I think that as you said, we can close #9119 . |
Thank you for all the work tracking down all the implications. While this PR did not get merged, your contribution did close a lot of issues! |
In rust-lang/crates.io#1596 it was added
a rate limit for crate publishing endpoint connections (10 components/min).
As is said in the PR. The strategy used allows to upload 10 components
first and then the ratio starts to be applied.
Now, if the rate is surpassed, crates.io returns a 429 HTTP response.
Cargo, tries to publish all of the workspace/crate components as fast
as possible (as we should expect) but when we have more than 10
components and we try to publish the 11th, usually a minute hasn't
passed yet and therefore we get a HTTP-429.
This made some users to experience confusing/not well explained error
messages as seen in rust-lang/crates.io#1643.
The limit got increased to 30 components/min. But who knows..
So this closes #6714 while
a better solution is not found.