Support for encrypted values in Declarative Configuration

[erikgb]

User story

As a Kong DB-less Cluster Administrator I can encrypt selective values (secrets) in the Declarative Configuration, so that the configuration file can be stored in Git without locking down the Git repository.

Background

We are considering a DB-less Kong Cluster setup because of the truly declarative configuration. But without any support for encrypting secrets in the configuration file, it will have very limited usability - at least for us, as our goal is to have automated CI/CD pipelines managing clusters.

Managing secrets is always problematic, but I think we all can agree that managing one secret well is easier than managing hundreds of them.

Suggestions

There is already a couple of solutions out there, that we actually use, that can serve as inspiration:

• https://github.com/voxpupuli/hiera-eyaml; Puppet deployments
• https://github.com/ulisesbocchio/jasypt-spring-boot; Spring Boot application deployments

[p0pr0ck5]

Rather than store the whole config in source control, it’s saner to store a config skeleton in source control and then use a template rendered to generate the final config. This allows better separation of function and data. You can use something like Vault to store secrets and consul-template to render the config.

[erikgb]

@p0pr0ck5 : There are a lot of possible workarounds for this issue, but without native support for this in Kong you will always have a greater risk of exposing exposing secrets to unauthorized subjects. I.e. a Kong ("admin") UI would have to be strictly open for authorized users (admins). This application would need access to the Kong Admin API, and in the Admin API the secrets would be available in cleartext.

Related issue: #1237

[erikgb]

And to have a declarative configuration containing secrets under source control management, will require some kind of templating. It is only the Kong cluster nodes that need access to the secrets in cleartext - if you consider the consumer and upstream service as outside the system.

Of course, if you have a Vault infrastructure up and running, I would rather go in that direction. But I would guess not all users of Kong do have that...... I am not trying to get "spaceship control software" security here. Just something with pretty good security, which is better than nothing.

I do not know the internal architecture of Kong, but I would imagine something like this could be possible - if the Kong administrator enabled this new feature:

• Kong configuration properties containing secrets are always stored in an encrypted format in the database (that will actually resolve Storing sensitive information in database #1237). For the DB Less configuration it becomes the responsability of the Kong administrator to encrypt the secrets in the configuration - if he wants to.
• The Admin API REST service will make sure that all configuration properties containing secrets are encrypted on read (GET). If the configuration is stored in a database there will be no transformation, since the secrets would already be encrypted. When reading the DB Less configuration (from file), through the Admin API, I think the Admin REST service should encrypt any secret configuration property that isn't already encrypted. Or maybe not, just leave it up to the Kong administrator... ("shit in, shit out"). ;-)
• The Admin API REST service should accept both encrypted and cleartext secrets on configuration inserts/updates (POST/PUT), but the inserted/updated security configuration is always encrypted (if not already encrypted) before it is flushed to the database.
• It is only the Kong runtime that would be allowed to decrypt the secrets. Since Kong is already caching it's configuration, it shouldn't be a big performance overhead. The Kong plugins will (of course) get the relevant secrets in cleartext. I.e. to perform basic auth in a plugin architecture you need the password in cleartext....

[p0pr0ck5]

cc @bungle your recent work on DAO transformations may be relevant here ^

[erikgb]

Thanks @p0pr0ck5! Looking at @bungle latest work, I consider #5047 to be highly relevant. And it seems like you by this PR, already made a fundament for implementing what I am looking for - or at least something similar.

Just a small disclaimer: I am not a Lua programmer, nor an expert on security. So please bear with me. ;-) But I hope you appreciate input/ideas from regular "users" as me, and I would be happy to contribute to discussions and reviews on a general level....

[erikgb]

Some suggestions from me, to hopefully push this issue forward:

The eyaml format never got standardized, and I do not know if anyone ever tried. ;-) But I think the eyaml-format can be a good choice for supporting encrypted values in the Kong declarative file format. There is actually some tool support for eyaml out there: Visual Studio plugin, Atom package ++. Tool support is not very relevant in this context, but selecting a format with some existing tool support will make the declarative format (with encrypted values) more user-friendly.

The encryption method(s) will have to be decided, but starting with PKCS#7 (default in eyaml), an example may look like:

consumers:
- username: my-user
  keyauth_credentials:
  - key: my-key

will turn into:

consumers:
- username: my-user
  keyauth_credentials:
  - key:  >
    ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv
    NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh
    jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y
    l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd
    /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm
    IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==]