Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBaseInputFormat tests and fix #1269

Open
wants to merge 3 commits into
base: titan11
Choose a base branch
from

Conversation

sjudeng
Copy link

@sjudeng sjudeng commented Feb 19, 2016

Reproduce and resolve issue #1268.

@titan-cla
Copy link

Hi @sjudeng, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

@sjudeng sjudeng changed the title Issue 1268 HBaseInputFormat tests and fix Feb 20, 2016
@graben1437
Copy link
Contributor

You might want to check pull request 1218 which makes the same changes plus cleans up a lot of old Titan back level support.

@sjudeng
Copy link
Author

sjudeng commented Feb 29, 2016

Thanks @graben1437. I'm having trouble testing on that branch. See question in #1218.

@sjudeng sjudeng mentioned this pull request Mar 1, 2016
@sjudeng
Copy link
Author

sjudeng commented Mar 2, 2016

@graben1437, unfortunately issue #1268 is not resolved in #1218. To reproduce the error, you can cherry pick the first commit from this PR, which adds HBaseInputFormat tests.

git remote add graben1437 https://github.com/graben1437/titan.git
git fetch graben1437
git checkout hbase-support1
git merge titan11
git remote add ngageoint https://github.com/ngageoint/titan.git
git fetch ngageoint
git cherry-pick 4b163aa4c7a253052f6090d94e0be6caf4e316fa
mvn clean install -DskipTests
mvn verify -Phadoop2 -Dit.test="**/HBaseInputFormatIT.java" -DfailIfNoTests=false -rf :titan-hadoop-2
Running com.thinkaurelius.titan.hadoop.HBaseInputFormatIT
...
testReadGraphOfTheGods(com.thinkaurelius.titan.hadoop.HBaseInputFormatIT)  Time elapsed: 40.625 sec  <<< ERROR!
java.lang.IllegalStateException: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.ClassCastException: org.apache.hadoop.hbase.mapreduce.TableInputFormatBase$1 cannot be cast to org.apache.hadoop.hbase.mapreduce.TableRecordReader
    at com.thinkaurelius.titan.hadoop.formats.hbase.HBaseBinaryInputFormat.createRecordReader(HBaseBinaryInputFormat.java:47)
    at com.thinkaurelius.titan.hadoop.formats.util.GiraphInputFormat.createRecordReader(GiraphInputFormat.java:53)

The error is resolved in the second commit in this PR.

git cherry-pick 30c59eaaee5929c46f02353aef12554ea952a9dd
mvn clean install -DskipTests
mvn verify -Phadoop2 -Dit.test="**/HBaseInputFormatIT.java" -DfailIfNoTests=false -rf :titan-hadoop-2
Running com.thinkaurelius.titan.hadoop.HBaseInputFormatIT
...
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants