Memory consumption increase after migrating to 2.0.0

[larissaleite]

Hi!
After updating to version 2.0.0 on the 23rd of September las tweek, I've noticed a big increase in the memory consumption of my application:

Is that something that you are aware? Do you have any clues about what could be causing it and potential solutions?
Thank you 💙

[seandstewart]

Hey @larissaleite -

The default connection pool in aioredis is now quite large, have you tried using a BlockingConnectionPool with a smaller number of connections? Say 100 or so?

[larissaleite]

Thank you for your reply! So this is the default maximum number of connection?
I am creating the redis client just using from_url(REDIS_URL), but I could try using a BlockingConnectionPool

[larissaleite]

I made this change yesterday (where you can see 10/06 16:00) but didn't see any noticeable effect, the behavior is quite similar to the previous day:

Would you have any other suggestion to try?
Thanks!

[Andrew-Chen-Wang]

@larissaleite have you resolved this issue? Having a little bit of info would be nice to reproduce, for instance:

tl;dr assuming this is for a web server, you should close your ConnectionPool from the Redis instance after every request.

1. Are you running this on a web server where you're creating 1 Redis instance per request? As in, does each request to a web server create a new Redis instance?
2. Is this on production? (I guess based on the metrics). There may be confounding factors such as too many requests by your db servers.
3. How many open Redis connections are there? At peak? At minimum? On average?
4. Are you running PubSub or just Redis?

A little bit of code to reproduce would be helpful since I'm not aware of a memory spike.

Assuming this is for a website, what it looks like to me is you open a bunch of Redis instances but do not close these connections after each request which may lead to your web server failing, restarting (if you're using some daemon manager), and having your memory drop down to base level.

Because you're using from_url, you're creating separate ConnectionPools (at least that's what it seems like), and you aren't closing these connections (which may be our fault due to __del__).

I would assume a fix would be, after every request and assuming this is a web server, close the Redis connections or Connection Pool manually. This should stabilize and amortize the memory usage.

[larissaleite]

Thanks @Andrew-Chen-Wang for your reply, sorry I could not come back to this earlier.

Are you running this on a web server where you're creating 1 Redis instance per request? As in, does each request to a web server create a new Redis instance?
Is this on production? (I guess based on the metrics). There may be confounding factors such as too many requests by your db servers.

We actually create the Redis instance at app startup, so it's not created per request. And yep, this is production.

redis_client = Redis(connection_pool=BlockingConnectionPool(max_connections=200)).from_url(REDIS_URL)

How many open Redis connections are there? At peak? At minimum? On average?

The amount of connections is relatively stable, actually, but at peak we have around ~800 connections, whereas during lower traffic the minimum is 50.

Are you running PubSub or just Redis?

Just Redis.

Because you're using from_url, you're creating separate ConnectionPools (at least that's what it seems like), and you aren't closing these connections (which may be our fault due to del).
I would assume a fix would be, after every request and assuming this is a web server, close the Redis connections or Connection Pool manually. This should stabilize and amortize the memory usage.

Thanks, I will investigate this a bit better. Since I don't recreate the redis_client instance, as mentioned before, we simply reuse it and run a redis_client.get(key) per request. This pretty much sums up our usage.

[larissaleite]

Hi @Andrew-Chen-Wang just reporting back after we've finally managed to nail down the problem. The issue in the end was not related to aioredis, but to an update in uvicorn/uvloop that happened around the same time when we migrated to aioredis 2.0. Sorry for the false alarm and thanks for your assistance on this! 🙂