Objective:

To generate possible visualization(line,charts, etc..) of hadoop data stored in hbase-0.94.7 using crux-aggregation reporting tool.

Prerequisite:

1. Install Java
2. Install pseudo-cluster hadoop. Refer https://docs.google.com/document/d/1v-J19xwJn-Pw9F8OCgLn04dqKwYkGIOxyqRAIGpmHk0/edit?pli=1
3. Install hbase-0.94.7.Refer https://docs.google.com/document/d/1ohAR8BLLX6b4M7-Rw1zCXtj2MFEYzLXilSkiWg-KNLI/edit?pli=1 I prefer to install hbase-0.94.7 because the tar file required to setup crux can be downloaded from https://github.com/mkamithkumar/crux-tar-for-hbase-0.94.7
4. Install Mysql
5. Install Apache tomcat. Refer https://gist.github.com/mkamithkumar/5668122

Setup Crux: - - Download crux from https://github.com/sonalgoyal/crux and extract it to /usr/local(or any of your desired directory)
- Login to your mysql in terminal
        hduser@Hadoop:~$ mysql -uroot -hlocalhost -p'your_mysql_root_password'

1. Create database for crux in MySQL
mysql>create databse crux;
mysql>use crux;
2. Create schema by running ${YOUR_CRUX_HOME}/db/schema.sql file in MySQL prompt,
mysql>source ${YOUR_CRUX_HOME}/db/schema.sql This creates the schema required for saving the report definitions

- Generate crux.jar and crux.war file:

1. If your are using hbase-0.94.7 , then download crux.war and crux.tar from https://github.com/mkamithkumar/crux-tar-for-hbase-0.94.7
2. If your using some other version of hbase check whether it is available in https://github.com/sonalgoyal/crux/downloads
3. If no other go, buil your own crux.war and crux.tar using Maven

Configure Maven: - Install Maven - Update hibernate.properties file which is present inside ${CRUX_HOME}/ with your MySQL host, port, dbname, testDbName, user and password. - Download struts2-fullhibernatecore-plugin-2.2.2-GA.jar from http://code.google.com/p/full-hibernate-plugin-for-struts2/downloads/detail?name=struts2-fullhibernatecore-plugin-2.2.2-GA.jar&can=2&q= - Add the jar file to your local repository by the following steps:

Generate crux.jar and crux.tar file:

• In terminal change directory to ${YOUR_CRUX_HOME}/
hduser@Hadoop:~$ cd ${YOUR_CRUX_HOME}
  
• Run the follwing maven command hduser@Hadoop:/usr/local/crux-aggregation$ mvn install:install-file -DgroupId=com.google.code -DartifactId=struts2 -fullhibernatecore-plugin -Dversion=2.2.2-GA-Dpackaging=jar -Dfile=${PATH_TO_struts2-fullhibernatecore-plugin-2.2.2-GA.jar}</br/>
• The above will add the truts2-fullhibernatecore-plugin-2.2.2-GA.jar file to your maven repository(by default .m2 directory)
• Now build maven to generate crux.jar and crux.jar file hduser@Hadoop:/usr/local/crux-aggregation$ mvn install
  
• Now go to ${YOUR_CRUX_HOME}/target folder you can find crux.jar and crux.war file
• Copy crux.jar to ${YOUR_HBASE_HOME}/lib
  
• Copy crux.war to tomcat/webapps
  
• Start hbase
  • Go to ${YOUR_HBASE_HOME} then enter bin/start-hbase.sh to start hbase hduser@Hadoop:/usr/local/hbase$ sudo bin/start-hbase.sh
    
  • Then start hbase shell by entering hbase shell hduser@Hadoop:/usr/local/hbase$ sudo bin/hbase shell
    
• Start tomcat
  • Go to ${YOUR_CATALINA_HOME} then enter bin/startup.sh to start tomcat server hduser@Hadoop:/usr/local/tomcat$ sudo bin/startup.sh
    
  • Now your crux is setup goto http://localhost:8080/crux and define your connection, mapping and report.

Setup Eclipse Project:

• Download bse-stock-hadoop-crux-reporting git hub repository and extract it
• Open Eclipse and import BSEStock project into workspace
• Resolve the build path issues by adding all jar files from ${YOUR_HADOOP_HOME} and ${YOUR_HADOOP_HOME}/lib
• Add hbase jar from ${YOUR_HBASE_HOME} and zookeeper jar from ${YOUR_HBASE_HOME}/lib to project build path
• Add the jar files in the dependency folder to your eclipse build path
• Create a table in hbase
hbase(main):006:0> create 'stockDataComposite','price'
• Copy the BSEStockdata.txt file from the dependency folder to HDFS
hadoop dfs -copyFromLocal $HOME/Downloads/BSEStockData.txt hdfs://localhost:54310/user/hduser/projects/input/bsestock
• Change the input file path in StockDriver.java to your hdfs path of input file
• Run StockDriver.java as Hadoop Job
• Open crux web ui http://localhost:8080/crux
• Give Connection Name as HbaseConnection and HBase Zookeeper Location as localhost:2181
• Add a mapping by clicking Mapping link with Mapping Name as BSEStockMapping(or your girlfriend's name)
• Add Row Key Alias

  	<ol>
  		<li><em>Alias:</em> <b>stockId</b><br/><em>Length:</em> <b>6</b><br/><em>Value Type:</em> <b>String</b></li>
  		<li><em>Alias:</em> <b>date</b><br/><em>Length:</em> <b>8</b><br/><em>Value Type:</em> <b>Long</b></li>
  	</ol>
  </li>
  <li>Add Column Alias
  	<ol>
  		<li><em>Column Family:</em> <b>price</b><br/><em>Qualifier:</em> <b>close</b><br/><em>Alias:</em> <b>closePrice</b><br/><em>Value Type:</em> <b>Float</b></li>
  		<li><em>Column Family:</em> <b>price</b><br/><em>Qualifier:</em> <b>open</b><br/><em>Alias:</em> <b>openPrice</b><br/><em>Value Type:</em> <b>Float</b></li>
  		<li><em>Column Family:</em> <b>price</b><br/><em>Qualifier:</em> <b>low</b><br/><em>Alias:</em> <b>lowPrice</b><br/><em>Value Type:</em> <b>Float</b></li>
  		<li><em>Column Family:</em> <b>price</b><br/><em>Qualifier:</em> <b>high</b><br/><em>Alias:</em> <b>highPrice</b><br/><em>Value Type:</em> <b>Float</b></li>
  	</ol>
  </li>
  <li>After adding Row Key and Column Aliases save this mapping.</li>
  <li> Add Report by Clicking Report link  with <em>Report Name</em> as <b>BSEStock report<b>(or your second girlfriend's name) </li>
  <li> Select the mapping your have saved just now from the Mapping drop down</li>
  <li> Select your desired Report Type and Reap the fruit of this project. Enjoy have fun..!!!</li> 
  <li>All info above are taken from http://nubetech.co/crux-visualizing-and-querying-big-data-saved-in-hbase</li>