I am running EMR hive query on S3 and it fails saying "Map operator initialization failed"
I tried to set HADOOP_CLASSPATH as below, still no luck.
set HADOOP_CLASSPATH=/usr/lib/hive/lib/*;
Also, adding below jar,
add jar /usr/hive/json-serde-1.3.7-jar-with-dependencies.jar . This jar file is present at the given path.
I am not sure why it is complaining as below:
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
The log is as below:
Status: Failed
Vertex failed, vertexName=Map 10, vertexId=vertex_1515176779844_0076_2_02, diagnostics=[Task failed,
taskId=task_1515176779844_0076_2_02_000031, diagnostics=[TaskAttempt 0
failed, info=[Error: Error while running task ( failure ) :
attempt_1515176779844_0076_2_02_000031_0:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:286)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.SerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 35 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1515176779844_0076_2_02_000031_1:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:286)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.SerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 35 more
], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) :
attempt_1515176779844_0076_2_02_000031_2:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:286)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.SerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 35 more
], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
attempt_1515176779844_0076_2_02_000031_3:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:286)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.SerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 35 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:159, Vertex vertex_1515176779844_0076_2_02 [Map 10]
killed/failed due to:OWN_TASK_FAILURE]
Vertex killed, vertexName=Reducer 7, vertexId=vertex_1515176779844_0076_2_04, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:12, Vertex
vertex_1515176779844_0076_2_04 [Reducer 7] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1515176779844_0076_2_06, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex
vertex_1515176779844_0076_2_06 [Reducer 2] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Reducer 4, vertexId=vertex_1515176779844_0076_2_08, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:7, Vertex
vertex_1515176779844_0076_2_08 [Reducer 4] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Reducer 3, vertexId=vertex_1515176779844_0076_2_07, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:7, Vertex
vertex_1515176779844_0076_2_07 [Reducer 3] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Reducer 5, vertexId=vertex_1515176779844_0076_2_10, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:14, Vertex
vertex_1515176779844_0076_2_10 [Reducer 5] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Reducer 11, vertexId=vertex_1515176779844_0076_2_09, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:12, Vertex
vertex_1515176779844_0076_2_09 [Reducer 11] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Map 6, vertexId=vertex_1515176779844_0076_2_03, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:165, Vertex
vertex_1515176779844_0076_2_03 [Map 6] killed/failed due
to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Map 1, vertexId=vertex_1515176779844_0076_2_05, diagnostics=[Vertex received
Kill while in RUNNING state., Vertex did not succeed due to
OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:139, Vertex
vertex_1515176779844_0076_2_05 [Map 1] killed/failed due
to:OTHER_VERTEX_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:8
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
vertexName=Map 10, vertexId=vertex_1515176779844_0076_2_02,
diagnostics=[Task failed, taskId=task_1515176779844_0076_2_02_000031,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running
task ( failure ) :
attempt_1515176779844_0076_2_02_000031_0:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:184)
... 14 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:286)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.SerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 35 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) :
attempt_1515176779844_0076_2_02_000031_1:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
I am trying to migrate a OWL data into GRAKN using the migration script as described here. I started my GRAKN engine and then executed ./migration.sh owl -i /home/file1.owl -k grakn but I get this error:
Migrating data /home/file1.owl using Grakn Engine localhost:4567 into graph grakn
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: Graph Factory [ai.grakn.factory.TitanInternalFactory] is not valid
at ai.grakn.factory.FactoryBuilder.newFactory(FactoryBuilder.java:101)
at ai.grakn.factory.FactoryBuilder.getGraknGraphFactory(FactoryBuilder.java:82)
at ai.grakn.factory.FactoryBuilder.getFactory(FactoryBuilder.java:62)
at ai.grakn.factory.GraknSessionImpl.configureGraphFactoryRemote(GraknSessionImpl.java:153)
at ai.grakn.factory.GraknSessionImpl.configureGraphFactory(GraknSessionImpl.java:134)
at ai.grakn.factory.GraknSessionImpl.getConfiguredFactory(GraknSessionImpl.java:90)
at ai.grakn.factory.GraknSessionImpl.open(GraknSessionImpl.java:79)
at ai.grakn.migration.owl.Main.runOwl(Main.java:64)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.Collections$2.tryAdvance(Collections.java:4717)
at java.util.Collections$2.forEachRemaining(Collections.java:4725)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at ai.grakn.migration.owl.Main.main(Main.java:52)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at ai.grakn.factory.FactoryBuilder.newFactory(FactoryBuilder.java:99)
... 19 more
Caused by: java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager
at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:55)
at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:473)
at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:407)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1325)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:94)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:84)
at com.thinkaurelius.titan.core.TitanFactory$Builder.open(TitanFactory.java:139)
at ai.grakn.factory.TitanInternalFactory.configureGraph(TitanInternalFactory.java:113)
at ai.grakn.factory.TitanInternalFactory.newTitanGraph(TitanInternalFactory.java:88)
at ai.grakn.factory.TitanInternalFactory.buildTinkerPopGraph(TitanInternalFactory.java:84)
at ai.grakn.factory.TitanInternalFactory.buildTinkerPopGraph(TitanInternalFactory.java:60)
at ai.grakn.factory.AbstractInternalFactory.getTinkerPopGraph(AbstractInternalFactory.java:141)
at ai.grakn.factory.AbstractInternalFactory.getTinkerPopGraph(AbstractInternalFactory.java:135)
at ai.grakn.factory.AbstractInternalFactory.getGraph(AbstractInternalFactory.java:110)
at ai.grakn.factory.AbstractInternalFactory.open(AbstractInternalFactory.java:95)
at ai.grakn.factory.SystemKeyspace.loadSystemOntology(SystemKeyspace.java:114)
at ai.grakn.factory.FactoryBuilder.newFactory(FactoryBuilder.java:107)
at ai.grakn.factory.FactoryBuilder.getGraknGraphFactory(FactoryBuilder.java:82)
at ai.grakn.factory.AbstractInternalFactory.getSystemFactory(AbstractInternalFactory.java:80)
at ai.grakn.factory.AbstractInternalFactory.<init>(AbstractInternalFactory.java:74)
at ai.grakn.factory.TitanInternalFactory.<init>(TitanInternalFactory.java:64)
... 24 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:44)
... 44 more
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:580)
at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.<init>(AstyanaxStoreManager.java:291)
... 49 more
Is there any mistake in the command I am executing to do the migration? What can I do to resolve this issue?
UPDATE: I was able to resolve the above issue but now I am getting following error in trying to start the graql shell:
Exception in thread "graql-session-0" java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
at java.util.HashMap$ValueIterator.next(HashMap.java:1466)
at ai.grakn.graph.internal.AbstractGraknGraph.loadOntologyCacheIntoTransactionCache(AbstractGraknGraph.java:337)
at ai.grakn.graph.internal.AbstractGraknGraph.getConceptLog(AbstractGraknGraph.java:307)
at ai.grakn.graph.internal.AbstractGraknGraph.buildType(AbstractGraknGraph.java:450)
at ai.grakn.graph.internal.AbstractGraknGraph.getTypeByLabel(AbstractGraknGraph.java:548)
at ai.grakn.graph.internal.AbstractGraknGraph.getMetaConcept(AbstractGraknGraph.java:608)
at ai.grakn.engine.session.GraqlSession.getTypes(GraqlSession.java:371)
at ai.grakn.engine.session.GraqlSession.sendTypes(GraqlSession.java:348)
at ai.grakn.engine.session.GraqlSession.lambda$new$48(GraqlSession.java:105)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
I am using storm 0.9.3.
I am running a topology in python, in which spouts read a URL off the Kafka queue and then pass it to a next bolt which fetches that page using python requests module.
Here is my topology definition in java.
public static void main(String[] args) throws Exception {
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("URLSpout", new URLSpout(), 1);
builder.setBolt("ScrapeBolt", new ScrapeBolt(), 30).shuffleGrouping("URLSpout");
Config conf = new Config();
//conf.setDebug(true);
conf.setMessageTimeoutSecs(50);
conf.setNumWorkers(3);
conf.setMaxSpoutPending(50);
conf.put(Config.TOPOLOGY_RECEIVER_BUFFER_SIZE, 8);
conf.put(Config.TOPOLOGY_TRANSFER_BUFFER_SIZE, 32);
conf.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 16384);
conf.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 16384);
StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
}
After the topology is submitted, it runs fine, but after some time the topology halts. When checked the logs, I found these errors.
2015-06-09T17:34:49.733+0530 b.s.util [ERROR] Async loop died!
java.lang.RuntimeException: pid:5736, name:URLSpout exitCode:-1, errorString:
at backtype.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:178) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.spout.ShellSpout.nextTuple(ShellSpout.java:91) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
Caused by: java.lang.RuntimeException: Unknown command received: error
at backtype.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:173) ~[storm-core-0.9.3.jar:0.9.3]
... 5 common frames omitted
2015-06-09T17:34:49.734+0530 b.s.d.executor [ERROR]
java.lang.RuntimeException: pid:5736, name:URLSpout exitCode:-1, errorString:
at backtype.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:178) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.spout.ShellSpout.nextTuple(ShellSpout.java:91) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
Caused by: java.lang.RuntimeException: Unknown command received: error
at backtype.storm.spout.ShellSpout.querySubprocess(ShellSpout.java:173) ~[storm-core-0.9.3.jar:0.9.3]
... 5 common frames omitted
2015-06-09T17:34:49.745+0530 b.s.util [ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) [storm-core-0.9.3.jar:0.9.3]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$fn__3812$fn__3813.invoke(worker.clj:456) [storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$mk_executor_data$fn__3274$fn__3275.invoke(executor.clj:240) [storm-core-0.9.3.jar:0.9.3]
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:473) [storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
And
java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
(Unable to capture error stream)
at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:101) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:318) ~[storm-core-0.9.3.jar:0.9.3]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
2015-06-09T17:34:49.779+0530 o.a.s.z.s.NIOServerCnxnFactory [ERROR] Thread Thread[Thread-37,5,main] died
java.lang.RuntimeException: java.lang.InterruptedException
at backtype.storm.util$wrap_in_runtime.invoke(util.clj:44) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.zookeeper$exists_node_QMARK_$fn__1668.invoke(zookeeper.clj:102) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:98) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.zookeeper$mkdirs.invoke(zookeeper.clj:114) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__1915.mkdirs(cluster.clj:119) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2372.report_error(cluster.clj:397) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$throttled_report_error_fn$fn__3221.invoke(executor.clj:180) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$fn__3441$fn$reify__3486.reportError(executor.clj:738) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.task.OutputCollector.reportError(OutputCollector.java:223) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.task.ShellBolt.die(ShellBolt.java:283) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.task.ShellBolt.access$400(ShellBolt.java:69) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:341) ~[storm-core-0.9.3.jar:0.9.3]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_45-internal]
Caused by: java.lang.InterruptedException: null
at java.lang.Object.wait(Native Method) ~[na:1.8.0_45-internal]
at java.lang.Object.wait(Object.java:502) ~[na:1.8.0_45-internal]
at org.apache.storm.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.curator.RetryLoop.callWithRetry(RetryLoop.java:107) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:157) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148) ~[storm-core-0.9.3.jar:0.9.3]
at org.apache.storm.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.zookeeper$exists_node_QMARK_$fn__1668.invoke(zookeeper.clj:101) ~[storm-core-0.9.3.jar:0.9.3]
... 11 common frames omitted
2015-06-09T17:34:49.780+0530 b.s.t.ShellBolt [ERROR] Halting process: ShellBolt died.
And
2015-06-09T18:13:15.392+0530 b.s.s.ShellSpout [ERROR] Halting process: ShellSpout died.
java.lang.RuntimeException: subprocess heartbeat timeout
at backtype.storm.spout.ShellSpout$SpoutHeartbeatTimerTask.run(ShellSpout.java:255) [storm-core-0.9.3.jar:0.9.3]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45-internal]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45-internal]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45-internal]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45-internal]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45-internal]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45-internal]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
Can someone,Please help me resolving this?
You're hitting https://issues.apache.org/jira/browse/STORM-796 which is fixed via Storm 0.9.5.
Since Storm 0.9.4 and 0.9.5 contains only bugfixes, so you can upgrade your Storm version from 0.9.3 to 0.9.5 safely.
I am geting an exception in the eclipse logs hen I try to access my registry entries in the Registry Browser perspective (and when I try to import registry entries into a registry project).
the exception is as follows: (any assistance would be much appreciated)
!MESSAGE java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.abdera.parser.stax.FOMParser
!STACK 0
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.abdera.parser.stax.FOMParser
at org.apache.abdera.util.Discover.locate(Discover.java:56)
at org.apache.abdera.util.Discover.locate(Discover.java:23)
at org.apache.abdera.util.AbderaConfiguration.newParserInstance(AbderaConfiguration.java:320)
at org.apache.abdera.Abdera.newParser(Abdera.java:224)
at org.apache.abdera.Abdera.getParser(Abdera.java:156)
at org.apache.abdera.protocol.client.AbstractClientResponse.<init>(AbstractClientResponse.java:57)
at org.apache.abdera.protocol.client.CommonsResponse.<init>(CommonsResponse.java:44)
at org.apache.abdera.protocol.client.AbderaClient.execute(AbderaClient.java:796)
at org.apache.abdera.protocol.client.AbderaClient.get(AbderaClient.java:235)
at org.wso2.carbon.registry.app.RemoteRegistry.get(RemoteRegistry.java:162)
at org.wso2.developerstudio.eclipse.greg.base.core.Registry.get(Registry.java:825)
at org.wso2.developerstudio.eclipse.greg.manager.remote.views.ResourceInfoViewer.updateGeneralInfo(ResourceInfoViewer.java:269)
at org.wso2.developerstudio.eclipse.greg.manager.remote.views.ResourceInfoViewer.updateInfo(ResourceInfoViewer.java:224)
at org.wso2.developerstudio.eclipse.greg.manager.remote.views.ResourceInfoViewer.updateMe(ResourceInfoViewer.java:334)
at org.wso2.developerstudio.eclipse.greg.manager.remote.views.RegistryBrowserView$48.widgetSelected(RegistryBrowserView.java:2391)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4134)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1458)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1481)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1466)
at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1271)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3980)
at org.eclipse.swt.widgets.Display.applicationNextEventMatchingMask(Display.java:4869)
at org.eclipse.swt.widgets.Display.applicationProc(Display.java:5239)
at org.eclipse.swt.internal.cocoa.OS.objc_msgSendSuper(Native Method)
at org.eclipse.swt.widgets.Widget.callSuper(Widget.java:221)
at org.eclipse.swt.widgets.Widget.mouseDownSuper(Widget.java:1093)
at org.eclipse.swt.widgets.Tree.mouseDownSuper(Tree.java:2052)
at org.eclipse.swt.widgets.Widget.mouseDown(Widget.java:1085)
at org.eclipse.swt.widgets.Control.mouseDown(Control.java:2538)
at org.eclipse.swt.widgets.Tree.mouseDown(Tree.java:2007)
at org.eclipse.swt.widgets.Display.windowProc(Display.java:5493)
at org.eclipse.swt.internal.cocoa.OS.objc_msgSendSuper(Native Method)
at org.eclipse.swt.widgets.Widget.callSuper(Widget.java:221)
at org.eclipse.swt.widgets.Widget.windowSendEvent(Widget.java:2102)
at org.eclipse.swt.widgets.Shell.windowSendEvent(Shell.java:2284)
at org.eclipse.swt.widgets.Display.windowProc(Display.java:5557)
at org.eclipse.swt.internal.cocoa.OS.objc_msgSendSuper(Native Method)
at org.eclipse.swt.widgets.Display.applicationSendEvent(Display.java:5002)
at org.eclipse.swt.widgets.Display.applicationProc(Display.java:5151)
at org.eclipse.swt.internal.cocoa.OS.objc_msgSend(Native Method)
at org.eclipse.swt.internal.cocoa.NSApplication.sendEvent(NSApplication.java:128)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3616)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1022)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:916)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:86)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:585)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:540)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.abdera.parser.stax.FOMParser
at org.apache.abdera.util.Discover.getClass(Discover.java:235)
at org.apache.abdera.util.Discover.load(Discover.java:210)
at org.apache.abdera.util.Discover.locate(Discover.java:47)
... 64 more
Caused by: java.lang.ClassNotFoundException: org.apache.abdera.parser.stax.FOMParser
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
at org.apache.abdera.util.Discover.getClass(Discover.java:231)
... 66 more
Someone let me know how to create Client object by DynamicClientFactory with Binding files.
this is a sample code i have tested. It occured an exception.
String wsdlURL = "http://localhost/test?wsdl";
ClassLoader loader = Thread.currentThread().getContextClassLoader();
JaxWsDynamicClientFactory factory = JaxWsDynamicClientFactory.newInstance();
List<String> bindingFiles = new ArrayList<String>();
File bindingFile = new File("custom.xml");
try {
bindingFiles.add(bindingFile.getCanonicalPath().toString());
} catch (IoException e) {
e.printStackTrace();
}
Client client = factory.createClient(wsdlURL, loader, bindingFiles);
below is custom.xml.
<bindings xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
wsdlLocation="http://localhost/test?wsdl"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"
xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
jaxb:version="2.0" jaxb:extensionBindingPrefixes="xjc"
xmlns="http://java.sun.com/xml/ns/jaxws">
<enableWrapperStyle>true</enableWrapperStyle>
<jaxb:globalBindings>
<xjc:simple/>
</jaxb:globalBindings>
</bindings>
when i run above sample code, i got below exception..
java.lang.reflect.UndeclaredThrowableException
at $Proxy18.bind(Unknown Source)
at org.apache.cxf.endpoint.dynamic.DynamicClientFactory.createClient(DynamicClientFactory.java:318)
at org.apache.cxf.endpoint.dynamic.DynamicClientFactory.createClient(DynamicClientFactory.java:252)
at DynamicWebServiceClient_test.test(DynamicWebServiceClient_test.java:93)
at DynamicWebServiceClient_test.main(DynamicWebServiceClient_test.java:54)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.cxf.common.util.ReflectionInvokationHandler.invoke(ReflectionInvokationHandler.java:52)
... 5 more
Caused by: java.lang.reflect.UndeclaredThrowableException
at $Proxy20.fatalError(Unknown Source)
at com.sun.tools.internal.xjc.api.impl.s2j.SchemaCompilerImpl.fatalError(SchemaCompilerImpl.java:279)
at com.sun.tools.internal.xjc.util.ErrorReceiverFilter.fatalError(ErrorReceiverFilter.java:71)
at com.sun.xml.internal.xsom.impl.parser.ParserContext$2.fatalError(ParserContext.java:196)
at com.sun.xml.internal.xsom.impl.parser.NGCCRuntimeEx.unexpectedX(NGCCRuntimeEx.java:484)
at com.sun.xml.internal.xsom.impl.parser.state.NGCCHandler.unexpectedEnterElement(NGCCHandler.java:179)
at com.sun.xml.internal.xsom.impl.parser.state.Schema.enterElement(Schema.java:401)
at com.sun.xml.internal.xsom.impl.parser.state.NGCCRuntime.startElement(NGCCRuntime.java:243)
at org.xml.sax.helpers.XMLFilterImpl.startElement(XMLFilterImpl.java:527)
at com.sun.tools.internal.xjc.util.SubtreeCutter.startElement(SubtreeCutter.java:93)
at com.sun.tools.internal.xjc.reader.ExtensionBindingChecker.startElement(ExtensionBindingChecker.java:133)
at org.xml.sax.helpers.XMLFilterImpl.startElement(XMLFilterImpl.java:527)
at com.sun.tools.internal.xjc.reader.xmlschema.parser.IncorrectNamespaceURIChecker.startElement(IncorrectNamespaceURIChecker.java:102)
at org.xml.sax.helpers.XMLFilterImpl.startElement(XMLFilterImpl.java:527)
at com.sun.tools.internal.xjc.reader.xmlschema.parser.CustomizationContextChecker.startElement(CustomizationContextChecker.java:177)
at org.xml.sax.helpers.XMLFilterImpl.startElement(XMLFilterImpl.java:527)
at com.sun.tools.internal.xjc.reader.internalizer.DOMForestScanner$LocationResolver.startElement(DOMForestScanner.java:130)
at com.sun.xml.internal.bind.unmarshaller.DOMScanner.visit(DOMScanner.java:228)
at com.sun.xml.internal.bind.unmarshaller.DOMScanner.scan(DOMScanner.java:111)
at com.sun.tools.internal.xjc.reader.internalizer.DOMForestScanner.scan(DOMForestScanner.java:78)
at com.sun.tools.internal.xjc.reader.internalizer.DOMForestScanner.scan(DOMForestScanner.java:86)
at com.sun.tools.internal.xjc.reader.internalizer.DOMForestParser.parse(DOMForestParser.java:88)
at com.sun.tools.internal.xjc.ModelLoader$XMLSchemaParser.parse(ModelLoader.java:251)
at com.sun.xml.internal.xsom.impl.parser.NGCCRuntimeEx.parseEntity(NGCCRuntimeEx.java:326)
at com.sun.xml.internal.xsom.impl.parser.ParserContext.parse(ParserContext.java:113)
at com.sun.xml.internal.xsom.parser.XSOMParser.parse(XSOMParser.java:154)
at com.sun.xml.internal.xsom.parser.XSOMParser.parse(XSOMParser.java:143)
at com.sun.tools.internal.xjc.ModelLoader.createXSOM(ModelLoader.java:505)
at com.sun.tools.internal.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:226)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.cxf.common.util.ReflectionInvokationHandler.invoke(ReflectionInvokationHandler.java:52)
... 39 more
Caused by: java.lang.RuntimeException: Fatal error compiling schema from WSDL at {http://localhost/test?wsdl}: Unexpected <jaxws:bindings> appears at line 18 column 62
at org.apache.cxf.endpoint.dynamic.DynamicClientFactory$InnerErrorListener.fatalError(DynamicClientFactory.java:739)
... 44 more
Caused by: org.xml.sax.SAXParseException: Unexpected <jaxws:bindings> appears at line 18 column 62
at com.sun.xml.internal.xsom.impl.parser.NGCCRuntimeEx.unexpectedX(NGCCRuntimeEx.java:477)
... 34 more
add this :
System.setProperty("javax.xml.transform.TransformerFactory",
"org.apache.xalan.processor.TransformerFactoryImpl");
and "(xalan-2.7.1.jar" and serializer-2.7.1.jar") jar