Autoscaling and Distribution

[artur-ma]

Describe the issue

I belive this wrappers has all the plugins to solve the distribution issue on Aurora Mysql autscaling, we have an issue that new replicas are not picked up when additng new replicas, it usually happens on traffic spikes and there are no "idle" connections to close and ropen

Using this package, we see that the host monitoring "find" new replicas, but "read write split" plug, does not not use them, as it has the read replica cached

We didnt find any example in the docs that explains how to confgure the wrapper correctly for such case

Thanks

Links

[karenc-bq]

Thank you for raising this.

1. Could you please elaborate what currently happens when you there is a traffic spike? If there are no idle connections to close and reopen, is the application just returning a connection error to you? For this scenario, do you expect the AWS Advanced NodeJS Wrapper to reconnect to a new replica when there's an error. Or do you expect the Wrapper to always connect to the new replicas instead of the cached replica?
2. Are you using any connection pools? The Wrapper has Internal Connection Pool available, does this address the issue you are seeing?

[artur-ma]

@karenc-bq Thanks for the response,

Could you please elaborate what currently happens when you there is a traffic spike?

We have autoscaling configured on our Aurora Mysql cluster, so it scales up new read replicase because existing ones are overloaded, so what happens is the existing one stays on 98% CPU, but new scaled ones are using ~20-30% CPU, because they bearly have new connections

The reason for that is because pods in k8s are scaled faster that relicas in Aurora, so we have a lot of new pods, but still only one running replica of aurora, when new replicase are set up and running, I would expec even distribution between all replicas (the wrapper will identify the new replicas and send queries there, without waiting for existing connections to be closed because of idle time)

[karenc-bq]

Hi @artur-ma, thank you for the clarification. For visibility, we are working on a fix for read/write splitting plugin reusing the cached reader connection forever. However, based on the information provided we believe internal connection pool would better suit your use case. Could you please give it a try and let us know if any issues arises? Thank you.