Before we begin be sure to download and install confd.
confd supports the following backends:
- etcd
- consul
- vault
- environment variables
- redis
- zookeeper
- dynamodb
- rancher
- ssm (AWS Simple Systems Manager Parameter Store)
This guide assumes you have a working etcd, or consul server up and running and the ability to add new keys.
etcdctl set /myapp/database/url db.example.com
etcdctl set /myapp/database/user rob
curl -X PUT -d 'db.example.com' http://localhost:8500/v1/kv/myapp/database/url
curl -X PUT -d 'rob' http://localhost:8500/v1/kv/myapp/database/user
vault mount -path myapp generic
vault write myapp/database url=db.example.com user=rob
export MYAPP_DATABASE_URL=db.example.com
export MYAPP_DATABASE_USER=rob
redis-cli set /myapp/database/url db.example.com
redis-cli set /myapp/database/user rob
[zk: localhost:2181(CONNECTED) 1] create /myapp ""
[zk: localhost:2181(CONNECTED) 2] create /myapp/database ""
[zk: localhost:2181(CONNECTED) 3] create /myapp/database/url "db.example.com"
[zk: localhost:2181(CONNECTED) 4] create /myapp/database/user "rob"
First create a table with the following schema:
aws dynamodb create-table \
--region <YOUR_REGION> --table-name <YOUR_TABLE> \
--attribute-definitions AttributeName=key,AttributeType=S \
--key-schema AttributeName=key,KeyType=HASH \
--provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1
Now create the items. The attribute value value must be of type string:
aws dynamodb put-item --table-name <YOUR_TABLE> --region <YOUR_REGION> \
--item '{ "key": { "S": "/myapp/database/url" }, "value": {"S": "db.example.com"}}'
aws dynamodb put-item --table-name <YOUR_TABLE> --region <YOUR_REGION> \
--item '{ "key": { "S": "/myapp/database/user" }, "value": {"S": "rob"}}'
This backend consumes the Rancher metadata service. For available keys, see the Rancher Metadata Service docs.
aws ssm put-parameter --name "/myapp/database/url" --type "String" --value "db.example.com"
aws ssm put-parameter --name "/myapp/database/user" --type "SecureString" --value "rob"
The confdir is where template resource configs and source templates are stored.
sudo mkdir -p /etc/confd/{conf.d,templates}
Template resources are defined in TOML config files under the confdir.
/etc/confd/conf.d/myconfig.toml
[template]
src = "myconfig.conf.tmpl"
dest = "/tmp/myconfig.conf"
keys = [
"/myapp/database/url",
"/myapp/database/user",
]
Source templates are Golang text templates.
/etc/confd/templates/myconfig.conf.tmpl
[myconfig]
database_url = {{getv "/myapp/database/url"}}
database_user = {{getv "/myapp/database/user"}}
confd supports two modes of operation daemon and onetime. In daemon mode confd polls a backend for changes and updates destination configuration files if necessary.
confd -onetime -backend etcd -node http://127.0.0.1:2379
confd -onetime -backend consul -node 127.0.0.1:8500
ROOT_TOKEN=$(vault read -field id auth/token/lookup-self)
confd -onetime -backend vault -node http://127.0.0.1:8200 \
-auth-type token -auth-token $ROOT_TOKEN
confd -onetime -backend dynamodb -table <YOUR_TABLE>
confd -onetime -backend env
confd -onetime -backend redis -node 192.168.255.210:6379
or if you want to connect to a specific redis database (4 in this example):
confd -onetime -backend redis -node 192.168.255.210:6379/4
confd -onetime -backend rancher -prefix /2015-07-25
Note: The metadata api prefix can be defined on the cli, or as part of your keys in the template toml file.
Output:
2014-07-08T20:38:36-07:00 confd[16252]: INFO Target config /tmp/myconfig.conf out of sync
2014-07-08T20:38:36-07:00 confd[16252]: INFO Target config /tmp/myconfig.conf has been updated
The dest configuration file should now be in sync.
cat /tmp/myconfig.conf
Output:
# This a comment
[myconfig]
database_url = db.example.com
database_user = rob
confd -onetime -backend ssm
In this example we will use confd to manage two nginx config files using a single template.
etcdctl set /myapp/subdomain myapp
etcdctl set /myapp/upstream/app2 "10.0.1.100:80"
etcdctl set /myapp/upstream/app1 "10.0.1.101:80"
etcdctl set /yourapp/subdomain yourapp
etcdctl set /yourapp/upstream/app2 "10.0.1.102:80"
etcdctl set /yourapp/upstream/app1 "10.0.1.103:80"
curl -X PUT -d 'myapp' http://localhost:8500/v1/kv/myapp/subdomain
curl -X PUT -d '10.0.1.100:80' http://localhost:8500/v1/kv/myapp/upstream/app1
curl -X PUT -d '10.0.1.101:80' http://localhost:8500/v1/kv/myapp/upstream/app2
curl -X PUT -d 'yourapp' http://localhost:8500/v1/kv/yourapp/subdomain
curl -X PUT -d '10.0.1.102:80' http://localhost:8500/v1/kv/yourapp/upstream/app1
curl -X PUT -d '10.0.1.103:80' http://localhost:8500/v1/kv/yourapp/upstream/app2
/etc/confd/conf.d/myapp-nginx.toml
[template]
prefix = "/myapp"
src = "nginx.tmpl"
dest = "/tmp/myapp.conf"
owner = "nginx"
mode = "0644"
keys = [
"/subdomain",
"/upstream",
]
check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/service nginx reload"
/etc/confd/conf.d/yourapp-nginx.toml
[template]
prefix = "/yourapp"
src = "nginx.tmpl"
dest = "/tmp/yourapp.conf"
owner = "nginx"
mode = "0644"
keys = [
"/subdomain",
"/upstream",
]
check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/service nginx reload"
/etc/confd/templates/nginx.tmpl
upstream {{getv "/subdomain"}} {
{{range getvs "/upstream/*"}}
server {{.}};
{{end}}
}
server {
server_name {{getv "/subdomain"}}.example.com;
location / {
proxy_pass http://{{getv "/subdomain"}};
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}