Skip to content

Commit

Permalink
Update documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
debovema authored Nov 28, 2018
1 parent cca0674 commit c3522ca
Showing 1 changed file with 133 additions and 10 deletions.
143 changes: 133 additions & 10 deletions consul/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,52 @@
# Centralized configuration management for Flogo with a Consul KV Store
# Centralized configuration management for Flogo with a Consul key/value store

> **WARNING**: This contribution is in an experimental state and uses a patched version of _TIBCOSoftware/flogo-lib_.
## About properties resolution in Flogo

> **WARNING**: This description is true *only in this experimentation* until
[TIBCOSoftware/flogo-lib#255](https://github.com/TIBCOSoftware/flogo-lib/pull/255) is merged.

Properties are defined in the *flogo.json* configuration file in the *properties* array. For instance:
```json
"properties": [
{
"name": "log.message",
"type": "string",
"value": "Default message"
}
]
```

Properties values can be resolved by different methods:
* **Environment variables**: simplest method when deploying the application in containers (Docker, Kubernetes, ...)
* **Profile files**: best method to define different *profiles* of properties such as *dev*, *integration*, *prod*
* **External resolver**: can be any system supported by a Flogo contribution (as this one for Consul).
The contribution must be installed in the application using standard ``` flogo install``` command
* **Default**: use the value specified in the *flogo.json* configuration file of the application

The defined order for properties resolution is as follows:

> Environment variable **>** profile file (JSON) **>** external (Consul) **>** default value
* This priority order is statically defined.
* Each method is called only if it is enabled (see [Configuration](#configuration)).
* Each property uses the value from the first method succeeding.
* Resolution is performed at the start of the Flogo engine and properties are then cached during the lifetime of the
application.

## Requirements

* Go
* Go dep
* Flogo, at least v0.5.7
* Docker, for testing purpose only
* jq, to simplify the creation of the project only

## Usage

Let's demonstrate the resolution of properties with this Consul-based contribution (and with built-in methods too).

### Create a Flogo project

1. create a simple Flogo project and enter in its directory
Expand Down Expand Up @@ -34,29 +77,38 @@ EOF
flogo ensure
```

5. add a property ```message``` and use it in the log activity
5. add a property ```log.message``` and use it in the log activity
```
cat flogo.json | jq '. + {"properties": [{"name": "message", "type": "string", "value": "Default message"}]}' | jq '.resources[].data.tasks[].activity |= . + {"mappings": {"input": [{"type": "assign", "value": "$property.message", "mapTo": "message"}]}}' | jq '.resources[].data.tasks[].activity.input |= del(.message)' > flogo.json.tmp && mv flogo.json.tmp flogo.json
cat flogo.json | \
jq '. + {"properties": [{"name": "message", "type": "string", "value": "Default message"}]}' | \
jq '.resources[].data.tasks[].activity |= . + {"mappings": {"input": [{"type": "assign", "value": "$property[log.message]", "mapTo": "message"}]}}' | \
jq '.resources[].data.tasks[].activity.input |= del(.message)' \
> flogo.json.tmp && mv flogo.json.tmp flogo.json
```

6. build the application
```
flogo build -e
```

### Prepare a Consul KV Store
### Prepare a Consul key-value store

1. start a single-host development-mode Consul agent
1. start a single-host development-mode Consul agent with Docker (for development purpose only)
```
docker run -d --name=dev-consul -e CONSUL_BIND_INTERFACE=eth0 -p 8500:8500 consul
```

2. add a value for the property ```message``` in Consul
2. add a value for the property ```log.message``` in Consul
```
docker exec -t dev-consul consul kv put flogo/simple-config/message "Consul message"
docker exec -t dev-consul consul kv put flogo/simple-config/log/message "Consul message"
```

The key is prefixed by ```flogo``` and by the application name (```simple-config```).
* The key is prefixed by ```flogo``` and by the application name (```simple-config```).

* Notice that the property is using the
[standard way of grouping properties](https://tibcosoftware.github.io/flogo/development/flows/property-bag/#grouping-of-properties)
with a dot separator. All dots in properties names are replaced by forward slashes before resolving in Consul,
leveraging the hierarchical paradigm of the Consul key/value store.

### Run & test

Expand All @@ -81,8 +133,8 @@ INFO [activity-flogo-log] - Default message

1. set following environment variables
```
export CONSUL_HTTP_ADDR=127.0.0.1:8500
export FLOGO_APP_PROPS_VALUE_RESOLVER=consul
export CONSUL_HTTP_ADDR=127.0.0.1:8500 # optional as it is the default value
export FLOGO_APP_CONFIG_EXTERNAL=consul # enable the Consul external property resolver
```

2. run the application
Expand All @@ -94,3 +146,74 @@ You should see a log like:
```
INFO [activity-flogo-log] - Consul message
```

#### with Consul, overriding with a profile file

1. create a JSON profile file with an overridden value for ```log.message```
```
echo '{"log.message": "Profile file message"}' | jq '.' > profile.json
```

2. set following environment variables
```
export CONSUL_HTTP_ADDR=127.0.0.1:8500 # optional as it is the default value
export FLOGO_APP_CONFIG_EXTERNAL=consul # enable the Consul external property resolver
export FLOGO_APP_CONFIG_PROFILES=profile.json # override with a property value in a JSON profile file
```

3. run the application
```
./bin/simple-config
```

You should see a log like:
```
INFO [activity-flogo-log] - Profile file message
```

The property value in the profile file has overridden Consul.
The Consul API is not even called since the value of the property was resolved successfully by a resolver with a higher
priority.

#### with Consul, overriding with an environment variable

1. set following environment variables
```
export CONSUL_HTTP_ADDR=127.0.0.1:8500 # optional as it is the default value
export FLOGO_APP_CONFIG_EXTERNAL=consul # enable the Consul external property resolver
export FLOGO_APP_CONFIG_PROFILES=profile.json # override with a property value in a JSON profile file
export LOG_MESSAGE="Env var message" # override with an (canonical) environment variable
```

2. run the application
```
./bin/simple-config
```

You should see a log like:
```
INFO [activity-flogo-log] - Env var message
```

The environment variable has overridden Consul.
The Consul API is not even called since the value of the property was resolved successfully by a resolver with a higher
priority.

## Configuration

The configuration is set by a very limited set of environment variables. Some (three) are builtin in the Flogo engine,
the others are defined by external resolvers such as this current one.

### Flogo level

| Name | Default value | Required | Example value |
|----------------------------|---------------|-------------------------------|----------------------------------------------------|
| FLOGO_APP_CONFIG_ENV_VARS | true | no | false, to disable environment variables resolution |
| FLOGO_APP_CONFIG_PROFILES | ø | for "file" resolver only | "default.json", "common.json,prod.json" |
| FLOGO_APP_CONFIG_EXTERNAL | ø | for "external" resolvers only | "consul", "etcd", ... |

### Consul level

| Name | Default value | Required | Example value |
|-------------------|-----------------|----------|-------------------------|
| CONSUL_HTTP_ADDR | 127.0.0.1:8500 | no | consul.company.com:8500 |

0 comments on commit c3522ca

Please sign in to comment.