Skip to content
/ ullala Public
forked from albertein/ullala

ullala is a demo made originally for GDLjs to showcase kafka as data pipeline and the ELK stack

Notifications You must be signed in to change notification settings

gdljs/ullala

 
 

Repository files navigation

ullala

ullala is a demo that showcase using kafka as a data pipeline and log aggregation through the ELK stack

Setting up

  1. Download kafka command line tools: scripts/download-kafka.sh
  2. Run Kafka, Zookeeper, Elasticsearch and Logstash: docker-compose up
  3. Set kafka on your host file to point to the running kafka instance (docker-machine ip default on mac)
  4. Setup topics scripts/setup-topics.sh
  5. Install OpenCV for capturer, brew install opencv on mac.
  6. Symlink opencv into capturer, for mac you can run scripts/symlink-opencv-bew.sh
  7. Install capturer dependencies pip install -r capturer/requirements.txt
  8. Run Capturer: KAFKA_HOSTS=KAFKA:9092 capturer/capturer.py
  9. Install processor dependencies cd processor && npm insall
  10. Run processor: cd processor && KAFKA_HOSTS=kafka:9092 node index.js
  11. Install pusher dependencies cd pusher && npm install
  12. Run pusher `cd pusher && KAFKA_HOSTS=kafka:9092 node index.js

About

ullala is a demo made originally for GDLjs to showcase kafka as data pipeline and the ELK stack

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 70.7%
  • Python 18.9%
  • Shell 7.6%
  • HTML 2.8%