database

Database

This example demonstrates reading data from a database, transforming it to CloudEvents and sending to OpenMeter.

This is a rather common use case when a system already collects some sort of data or log and you want to send that as usage data to OpenMeter for further processing.

In this example, we will read data from a fake chat service's database (automatically seeded with random data) that charges customers based on the number of messages and/or message length. Benthos will read messages from a message log table and send the calculated usage to OpenMeter.

The example also demonstrates that certain business logic can also be implemented during the transformation (for example: users on the enterprise plan do not get charged for message length).

Databases featured in this example:

• Postgres
• Clickhouse

Tip

Check out the supported database drivers in the Benthos documentation.

Table of Contents

• Prerequisites
• Launch the example
• Checking events
• Cleanup
• Advanced configuration
• Production use

Prerequisites

This example uses Docker and Docker Compose, but you are free to run the components in any other way.

Check out this repository if you want to run the example locally:

git clone https://github.com/openmeterio/benthos-openmeter.git
cd benthos-openmeter/examples/database

Create a new .env file and add the details of your OpenMeter instance:

cp .env.dist .env
# edit .env and fill in the details

Tip

Tweak other options in the .env file to change the behavior of the example.

Create the following meters in OpenMeter with the following details:

Description	Event type	Aggregation	Value property	Group by (optional)
All message lenghts	chat-message	SUM	$.message_length	- plan: $.plan
Billable message lengths	chat-message	SUM	$.message_length_billable	- plan: $.plan
Message count	chat-message	COUNT	-	- plan: $.plan

Tip

Read more about creating a meter in the general examples README.

Launch the example

Decide which database you want to use:

export COMPOSE_PROFILES=SELECTED_DATABASE

Available profiles:

• postgres
• clickhouse

Launch the example (database, event collector and seeder):

docker compose up -d

Checking events

Read more in the general examples README.

Cleanup

Stop containers:

docker compose down -v

Advanced configuration

Check out the configuration files and the Benthos documentation for more details.

Production use

We are actively working on improving the documentation and the examples. In the meantime, feel free to contact us in email or on Discord.

We are more than happy to help you set up OpenMeter in your production environment.