make Secret API ready for production

[seeflood]

What would you like to be added:
Layotto already has the feature of Secret API, see #343
However, there are still many problems to be solved before it can be used in a production environment, including:

• Demos and docs. see Add demo and document for secret API #460

• Let the components of other APIs refer to the secret component in the configuration file.
  For example, currently if users want to use the redis component for the state API, they should write the password of redis in the configuration file, which is insecure.
  To solve this problem, we should allow users to "refer to" the secret component in the configuration and have the secret component automatically obtain the password.
  See https://docs.dapr.io/operations/components/component-secrets/

• Access control. For example, we should retrict which keys can be accessed by app1, see https://docs.dapr.io/developing-applications/building-blocks/secrets/secrets-scopes/

Why is this needed:
To make Secret API ready for production

---

chinese:

虽然 Layotto 支持了 secret API，见 #343
但是距离在生产环境用，还需要:

• 文档补充. Add demo and document for secret API #460
• 让其他API的组件在配置文件中可以引用密文。比如，当前用户想用 state API的redis组件时，必须在配置文件中配置 redis 的密码，这不安全。作为解决方案，可以允许用户在配置里”引用“secret 组件，由 secret 组件自动获取密码。见 https://docs.dapr.io/operations/components/component-secrets/
• 密钥的访问控制，比如限制app1能访问的密钥的白名单、黑名单等，见 https://docs.dapr.io/developing-applications/building-blocks/secrets/secrets-scopes/ 这个我们也可以配置在Layotto的json配置文件里

[ZLBer]

@seeflood anyone do this ? if not assign me.

[seeflood]

@ZLBer No one working on it yet.
Need your help 🚀

[ZLBer]

@seeflood our meta can not parse such data:

"metadata”:{
  "redisPassword":{
    "secretKeyRef":{
    	"name":"redis-secret"
        "key":  "redis-password"
}
}
}

so i think we can change the structure of this：

"metadata”:{
"secretKeyRef":"redisPassword",
"secretKeyRef-name":"redis-secret",
"secretKeyRef-key":"redis-password"
}
}

our meta:

dapr meta:

[seeflood]

@ZLBer that's really a problem :(
The existing key-value string pair is not expressive.
For example, if we use the structure you proposed:

"metadata”:{
  "secretKeyRef":"redisPassword",
  "secretKeyRef-name":"redis-secret",
  "secretKeyRef-key":"redis-password"
}

It seems ok. But if we want to add a new secret reference here, e.g. a redisToken,the structure will be:

"metadata”:{
  "secretKeyRef":"redisPassword",
  "secretKeyRef-name":"redis-secret",
  "secretKeyRef-key":"redis-password",  

  "secretKeyRef":"redisToken",
  "secretKeyRef-name":"redis-secret",
  "secretKeyRef-key":"redis-token"
}

It won't work!

[seeflood]

In addition to "secret reference", we might need a "configuration reference" ,see #500
So we have to solve this problem now.

• Solution A:

Make incompatible changes:

"metadata”:[
  {
  "name": "redisPassword",
  "secretKeyRef:{
      "name":"redis-secret",
      "key":"redis-password"
    }
  },{
  "name": "redisUsername",
  "value": "admin"
  }
]

We have to warn and help existing users refactor their configs.

• Solution B:

Add a new field called "properties":

"properties”:[
  {
  "name": "redisPassword",
  "secretKeyRef:{
      "name":"redis-secret",
      "key":"redis-password"
    }
  },{
  "name": "redisUsername",
  "value": "admin"
  }
]

[zhenjunMa]

I prefer Solution A, cause our project is still at an early stage, therefore, maintainability is more important than compatibility

[ZLBer]

@seeflood @zhenjunMa
Agree with A, add the configuration reference field, config can not only be obtained from SecretStore, but also obtained from the configuration center or istio control panel according to #500, which greatly enhances the scalability of the configuration.
赞同A, 增加 配置引用 这个字段，不止可以从SecretStore获取，按照 #500 里面说的，也可以从配置中心、istio控制面板获取相关配置，大大增强配置的可扩展性。

[seeflood]

agreed.
Then we should make this incompatible change in v0.5, and release v0.5 as soon as possible, so that we won't affect more users.

[ZLBer]

@seeflood @zhenjunMa 我觉得我们可以在meta之外加一个refs字段，初始化的时候把refs里的字段注入到meta里就可以了，我觉得这样的改动是最小的了。不然改动真的挺大的，配置解析、配置文件、每种组件的config都得改。

[seeflood]

@ZLBer 恩这也是一种方案

不然改动真的挺大的，配置解析、配置文件、每种组件的config都得改。

如果用方案A，不需要改组件的代码呀，runtime 层在启动时把ref翻译成 map[string]string 结构，然后再调组件的 Init 接口初始化。
比如 PubSub.Init 接收的参数还是 map[string]string

方案A 涉及的改动点是:

• 改配置解析的逻辑（runtime把ref翻译成 map[string]string、调组件的 Init 接口）
• 所有json文件。这个可以手改，也可以写段代码自动改（读json 文件、改结构、序列化覆盖掉旧的json文件）
• 文档里的配置项描述

[ZLBer]

@seeflood 这里面所有的config也都得改吧? 现在的这个大config是json直接解析过来的。或者可以自己自定义一个json解析器将ref解析成 map[string]string? （没写过

[seeflood]

当前讨论进展见:
#500 (comment)

  "state": {
    "redis": {
      "metadata": {
        "redisHost": "localhost:6380",
        "redisPassword": ""
      },
      "secretRef": [
        {
          "name": "xxx",
          "key": "yyy"
        }
      ]
    }
  },

如果用这个结构的话，就可以向下兼容啦，看看合适么