feat: add fixed_window bucket option

[panga]

Add fixed_window bucket option when enabled refill at specified interval instead of granular.

Used for enforcing a rate limit policy that does not allow allow request rate to go over a the size in a given interval.

Examples

Default behavior

buckets = {
  ip: {
    size: 1000,
    per_second: 1000
  }
}

Result: Refill 1 token each 1 millisecond.

New fixed_window: true

buckets = {
  ip: {
    size: 1000,
    per_second: 1000,
    fixed_window: true
  }
}

Result: Refill 1000 tokens each 1000 milliseconds.

[santiagoaguiar]

Elegant :).

This could still allow for something like 1 req at second 0, 999 req at second 0.9, and 1000 at second 1; I guess some aggregations could still show that as 2000 req 'per second' since you got 2000 requests in less than a second interval.

If we truly wanted to avoid more than 1k reqs per second, we would need to define a size 1 fill rate 1000, but customers would complain that sending two reqs at same time will return 429...

I think the fixed_window behavior will be easier to explain to customers though, but it could allow for even higher bursts than our current implementation.

[panga]

This implementation does not allow take more tokens than the specified size in the given interval.

In the example above it won't show as 2000 req per second as the bucket would only refill at second 1.9, so you will only be able to do next 1000 requests after second 1.9 making it impossible to use more than 1000 in less than 1 second while the current implementation allows 1999.

[panga]

What you're describing here is that in less than 1s interval a spike of request can happens at the edge of the interval and use more tokens. We do not have a way around that without degradation of user experience. The difference of this new feature is that a penalty will be applied and next request will need to wait for next refill and the remaining and reset headers will be consistent with the behavior.

[pfreixes]

@panga how are we gonna test this? or what are the plans? seems that ideally we would like to test this before merging it, right?