Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Try to use jgo for classpath instead of shaded fat jar
Needs - mvn on classpath - jgo from [resolve-only](scijava/jgo#37) pull request (Python >= 3.7) Currently fails with: ``` Unable to convert into Paintera dataset: Job aborted due to stage failure: Task 1 in stage 0.0 failed 4 times, most recent failure: Lost task 1.3 in stage 0.0 (TID 736, 10.36.110.15, executor 69): java.lang.NoSuchMethodError: org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream.<init>(Ljava/io/Ou at org.janelia.saalfeldlab.n5.GzipCompression.getOutputStream(GzipCompression.java:89) at org.janelia.saalfeldlab.n5.DefaultBlockWriter.write(DefaultBlockWriter.java:49) at org.janelia.saalfeldlab.n5.DefaultBlockWriter.writeBlock(DefaultBlockWriter.java:83) at org.janelia.saalfeldlab.n5.N5FSWriter.writeBlock(N5FSWriter.java:133) at org.janelia.saalfeldlab.n5.imglib2.N5LabelMultisets.saveLabelMultisetBlock(N5LabelMultisets.java:308) at org.janelia.saalfeldlab.n5.imglib2.N5LabelMultisets.saveLabelMultisetBlock(N5LabelMultisets.java:337) at org.janelia.saalfeldlab.label.spark.convert.ConvertToLabelMultisetType.lambda$convertToLabelMultisetType$dce05d2$1(ConvertToLabelMultisetType.java:226) at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1040) at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.TraversableOnce$class.reduceLeft(TraversableOnce.scala:185) at scala.collection.AbstractIterator.reduceLeft(Iterator.scala:1336) at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$14.apply(RDD.scala:1015) at org.apache.spark.rdd.RDD$$anonfun$reduce$1$$anonfun$14.apply(RDD.scala:1013) at org.apache.spark.SparkContext$$anonfun$33.apply(SparkContext.scala:2130) at org.apache.spark.SparkContext$$anonfun$33.apply(SparkContext.scala:2130) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:109) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ``` Probably dependency issue between spark and N5 (or other libraries) that is not an issue with shaded fat jar.
- Loading branch information