Here's a README for your Kafka Streams-based email sending application, based on the provided zip file:
This project is a Kafka Streams-based application that processes real-time email requests and enriches them with user information. It demonstrates the use of Kafka Streams to join a KStream of email requests with a GlobalKTable of user information.
- Introduction
- Features
- Kafka Streams Components
- Setup and Installation
- Running the Application
- Example Data
- Contributing
- License
The Email Sending Application demonstrates the use of Kafka Streams to process and join real-time data streams. It reads email request messages from a Kafka topic, enriches these messages with user information from another Kafka topic, and processes the enriched data.
- Processes email requests in real-time.
- Enriches email requests with user information.
- Demonstrates the use of Kafka Streams KStream and GlobalKTable for data processing.
- Handles real-time data joins.
KStream is a Kafka topic's continuous stream of records. This application uses KStream to process the email requests.
GlobalKTable is similar to KTable but stores all partitions' data locally. This is useful for enriching streams with data that needs to be globally accessible.
- Java 11
- Kafka and Zookeeper
- Docker (for Kafka and Zookeeper setup)
-
Clone the repository:
git clone https://github.com/your-username/email-sending-application.git cd email-sending-application
-
Set up Kafka and Zookeeper using Docker:
docker-compose up -d
-
Build the application using Gradle:
./gradlew clean build
-
Start the Kafka Streams application:
./gradlew run
-
Produce sample data to Kafka topics: Use the provided
OrderProducer
andUserProducer
classes to send sample data to theOrderStream
andUserTableTopic
topics.
- Run shell on kafka in docker:
docker-compose exec kafka /bin/sh
- Create topics:
kafka-topics.sh --create --topic EmailRequestsTopic --zookeeper zookeeper:2181 --partitions 1 --replication-factor 1 kafka-topics.sh --create --topic UserTableTopic --zookeeper zookeeper:2181 --partitions 1 --replication-factor 1