Skip to content

Pipeline: output: kafka-rest: style #1946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 23 additions & 21 deletions pipeline/outputs/kafka-rest-proxy.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,42 @@
# Kafka REST Proxy
# Kafka REST proxy

The **kafka-rest** output plugin, allows to flush your records into a [Kafka REST Proxy](http://docs.confluent.io/current/kafka-rest/docs/index.html) server. The following instructions assumes that you have a fully operational Kafka REST Proxy and Kafka services running in your environment.
The _Kafka rest_ (`kafka-rest`) output plugin lets you flush your records into a [Kafka REST Proxy](http://docs.confluent.io/current/kafka-rest/docs/index.html) server. The following instructions assume you have an operational Kafka REST Proxy and Kafka services running in your environment.

## Configuration Parameters
## Configuration parameters

This plugin supports the following parameters:

| Key | Description | default |
| :--- | :--- | :--- |
| Host | IP address or hostname of the target Kafka REST Proxy server | 127.0.0.1 |
| Port | TCP port of the target Kafka REST Proxy server | 8082 |
| Topic | Set the Kafka topic | fluent-bit |
| Partition | Set the partition number \(optional\) | |
| Message\_Key | Set a message key \(optional\) | |
| Time\_Key | The Time\_Key property defines the name of the field that holds the record timestamp. | @timestamp |
| Time\_Key\_Format | Defines the format of the timestamp. | %Y-%m-%dT%H:%M:%S |
| Include\_Tag\_Key | Append the Tag name to the final record. | Off |
| Tag\_Key | If Include\_Tag\_Key is enabled, this property defines the key name for the tag. | \_flb-key |
| Workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
| `Host` | IP address or hostname of the target Kafka REST Proxy server. | `127.0.0.1` |
| `Port` | TCP port of the target Kafka REST Proxy server. | `8082` |
| `Topic` | Set the Kafka topic. | `fluent-bit` |
| `Partition` | Optional. Set the partition number. | _none_ |
| `Message_Key` | Optional. Set a message key. | _none_ |
| `Time_Key` | The `Time_Key` property defines the name of the field that holds the record timestamp. | `@timestamp` |
| `Time_Key_Format` | Defines the format of the timestamp. | `%Y-%m-%dT%H:%M:%S` |
| `Include_Tag_Key` | Append the Tag name to the final record. | `Off` |
| `Tag_Key` | If `Include_Tag_Key` is enabled, this property defines the key name for the tag. | `_flb-key` |
| `Workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |

### TLS / SSL

The Kafka REST Proxy output plugin supports TLS/SSL.
For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).

## Getting Started
## Get started

In order to insert records into a Kafka REST Proxy service, you can run the plugin from the command line or through the configuration file:
To insert records into a Kafka REST Proxy service, you can run the plugin from the command line or through the configuration file.

### Command Line
### Command line

The **kafka-rest** plugin, can read the parameters from the command line in two ways, through the **-p** argument \(property\), e.g:
The Kafka REST plugin can read the parameters from the command line through the `-p` argument (property):

```shell
fluent-bit -i cpu -t cpu -o kafka-rest -p host=127.0.0.1 -p port=8082 -m '*'
```

### Configuration File
### Configuration file

In your main configuration file append the following:

Expand All @@ -45,8 +47,8 @@ In your main configuration file append the following:
pipeline:
inputs:
- name: cpu
tag: cpu
tag: cpu

outputs:
- name: kafka-rest
match: '*'
Expand Down Expand Up @@ -74,4 +76,4 @@ pipeline:
```

{% endtab %}
{% endtabs %}
{% endtabs %}
2 changes: 2 additions & 0 deletions vale-styles/FluentBit/Headings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ exceptions:
- gRPC
- I
- InfluxDB
- Kafka
- Kafka REST
- Kinesis
- Kubernetes
- LaunchDarkly
Expand Down