Skip to content

fellipeh/vagrant_hadoop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

vagrant_hadoop

Introduction

Vagrant project to create a cluster of 4 virtual machines with Hadoop v2.4.1 and Spark v1.0.1.

  1. node1 : HDFS NameNode + Spark Master
  2. node2 : YARN ResourceManager + JobHistoryServer + ProxyServer
  3. node3 : HDFS DataNode + YARN NodeManager + Spark Slave
  4. node4 : HDFS DataNode + YARN NodeManager + Spark Slave

Requeriments

  1. VirtualBox
  2. Vagrant
  3. Git clone this project,
  4. Run vagrant up to create the VM.
  5. Run vagrant ssh to get into your VM.
  6. Run vagrant destroy when you want to destroy and get rid of the VM.

Web Interface ( Monitoring )

You can check the following URLs to monitor the Hadoop daemons.

  1. [NameNode] (http://10.211.55.101:50070/dfshealth.html)
  2. [ResourceManager] (http://10.211.55.102:8088/cluster)
  3. [JobHistory] (http://10.211.55.102:19888/jobhistory)
  4. [Spark] (http://10.211.55.101:8080)

About

Vagrant config for Hadoop and Monitor tools

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages