From 0104434438d52af7fcb987d14967f433e61dfb1f Mon Sep 17 00:00:00 2001 From: Patrik Egyed <8093632+pregnor@users.noreply.github.com> Date: Sat, 13 May 2023 23:16:58 +0200 Subject: [PATCH] Replace kafka-topics --zookeeper flags with --bootstrap-server flags due to Kafka 3.0.0 deprecation (#110) * fix(test,bench): replaced zk flag w/ bootstrap-srv From https://kafka.apache.org/documentation/#upgrade_300_notable > The --zookeeper option was removed from the > kafka-topics and kafka-reassign-partitions > command line tools. Please use > --bootstrap-server instead. So all in our docs regarding kafka-topics commands we need to replace `--zookeeper zookeeper-client.zookeeper:2181` with `--bootstrap-server kafka-all-broker.kafka:29092` * Update docs/benchmarks/_index.md Co-authored-by: Robert Fekete --------- Co-authored-by: Darren Lau Co-authored-by: Robert Fekete --- docs/benchmarks/_index.md | 18 +++++++++++++++--- docs/test.md | 10 +++++++++- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/docs/benchmarks/_index.md b/docs/benchmarks/_index.md index c0b6e8d..1b155f4 100644 --- a/docs/benchmarks/_index.md +++ b/docs/benchmarks/_index.md @@ -125,11 +125,23 @@ How to setup the environment for the Kafka Performance Test. 1. Exec into this client and create the `perftest, perftest2, perftes3` topics. + For internal listeners exposed by a headless service (`KafkaCluster.spec.headlessServiceEnabled` is set to `true`): + + ```bash + + kubectl exec -it kafka-test -n kafka bash + ./opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-headless.kafka:29092 --topic perftest --create --replication-factor 3 --partitions 3 + ./opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-headless.kafka:29092 --topic perftest2 --create --replication-factor 3 --partitions 3 + ./opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-headless.kafka:29092 --topic perftest3 --create --replication-factor 3 --partitions 3 + ``` + + For internal listeners exposed by a regular service (`KafkaCluster.spec.headlessServiceEnabled` set to `false`): + ```bash kubectl exec -it kafka-test -n kafka bash - ./opt/kafka/bin/kafka-topics.sh --zookeeper zookeeper-client.zookeeper:2181 --topic perftest --create --replication-factor 3 --partitions 3 - ./opt/kafka/bin/kafka-topics.sh --zookeeper zookeeper-client.zookeeper:2181 --topic perftest2 --create --replication-factor 3 --partitions 3 - ./opt/kafka/bin/kafka-topics.sh --zookeeper zookeeper-client.zookeeper:2181 --topic perftest3 --create --replication-factor 3 --partitions 3 + ./opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-all-broker.kafka:29092 --topic perftest --create --replication-factor 3 --partitions 3 + ./opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-all-broker.kafka:29092 --topic perftest2 --create --replication-factor 3 --partitions 3 + ./opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-all-broker.kafka:29092 --topic perftest3 --create --replication-factor 3 --partitions 3 ``` Monitoring environment is automatically installed. To monitor the infrastructure we used the official Node Exporter dashboard available with id `1860`. diff --git a/docs/test.md b/docs/test.md index dc389fc..1b4c99f 100644 --- a/docs/test.md +++ b/docs/test.md @@ -22,8 +22,16 @@ Topic creation by default is enabled in Apache Kafka, but if it is configured ot - To create a sample topic from the CLI you can run the following: + For internal listeners exposed by a headless service (`KafkaCluster.spec.headlessServiceEnabled `set to `true`): + + ```bash + kubectl -n kafka run kafka-topics -it --image=ghcr.io/banzaicloud/kafka:2.13-3.1.0 --rm=true --restart=Never -- /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-headless.kafka:29092 --topic my-topic --create --partitions 1 --replication-factor 1 + ``` + + For internal listeners exposed by a regular service (`KafkaCluster.spec.headlessServiceEnabled` set to `false`): + ```bash - kubectl -n kafka run kafka-topics -it --image=ghcr.io/banzaicloud/kafka:2.13-3.1.0 --rm=true --restart=Never -- /opt/kafka/bin/kafka-topics.sh --zookeeper zookeeper-client.zookeeper:2181 --topic my-topic --create --partitions 1 --replication-factor 1 + kubectl -n kafka run kafka-topics -it --image=ghcr.io/banzaicloud/kafka:2.13-3.1.0 --rm=true --restart=Never -- /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka-all-broker.kafka:29092 --topic my-topic --create --partitions 1 --replication-factor 1 ``` After you have created a topic, produce and consume some messages: