Add support for AWS Secrets Manager as an external key value store

[jonathanfoster]

Is your feature request related to a problem? Please describe.

I'm an enterprise Tyk Cloud customer and I have a ton of APIs that require auth tokens for upstream API authorization. I use AWS Secrets Manager for external secrets, but this isn't natively supported by Tyk. Instead I have to run External Secrets Operator to inject secrets as environment variables, which requires me to store all secret key-value pairs in one external secret (e.g., TYK_GW_SECRETS=key:value,key2:value2). There's no way for me to allow individual API owners to manage their own secrets.

Describe the solution you'd like

A solution would be to natively support AWS Secrets Manager as an external key-value store.

Describe alternatives you've considered

The current workaround is to use External Secrets Operator to map secrets from AWS to environment variables in Tyk. This has the limitation of only supporting one secret with all key-value pairs.

Another option I've consider is installing Vault and have it integrate with AWS. This would require an enterprise license for Vault though.

[jonathanfoster]

I started looking into implementing this myself and found I can inject individual secrets as environment variables named TYK_SECRET_${key}, there's no need to combine all secrets into key-value pairs in TYK_GW_SECRETS. This certainly makes secrets more manageable with ESO, but it would still be nice if Secrets Manager was natively supported.

The TYK_SECRET_${key} feature wasn't 100% clear after initially reading the KV store docs so maybe there's an opportunity to clarify. Actually, I would say that's true for the KV store docs in general. It's been tough to wrap my head around what notation is required for what location and whether the reference supports a path or uses the hardcoded tyk-apis path.

[andyo-tyk]

Hi @jonathanfoster,

Thanks for raising this ticket. We are reviewing the enhancements we made to KV storage earlier this year and looking towards the future possibilities (which include improving support for other storage options).

In terms of the documentation, it's a tricky subject to cover concisely at the moment, due to some details of the implementation, but we have tried to be as clear as possible what notation is supported in different scenarios.
If you've got some specific questions or suggestions we could look to clarify further.

Thanks for your support.