Skip to content

Latest commit

 

History

History
18 lines (15 loc) · 962 Bytes

README.md

File metadata and controls

18 lines (15 loc) · 962 Bytes
Raw

ullala

ullala is a demo that showcase using kafka as a data pipeline and log aggregation through the ELK stack

Setting up

  1. Download kafka command line tools: scripts/download-kafka.sh
  2. Run Kafka, Zookeeper, Elasticsearch and Logstash: docker-compose up
  3. Set kafka on your host file to point to the running kafka instance (docker-machine ip default on mac)
  4. Setup topics scripts/setup-topics.sh
  5. Install OpenCV for capturer, brew install opencv on mac.
  6. Symlink opencv into capturer, for mac you can run scripts/symlink-opencv-bew.sh
  7. Install capturer dependencies pip install -r capturer/requirements.txt
  8. Run Capturer: KAFKA_HOSTS=KAFKA:9092 capturer/capturer.py
  9. Install processor dependencies cd processor && npm insall
  10. Run processor: cd processor && KAFKA_HOSTS=kafka:9092 node index.js
  11. Install pusher dependencies cd pusher && npm install
  12. Run pusher `cd pusher && KAFKA_HOSTS=kafka:9092 node index.js