In attempting to load an AOT-compiled class from a non-default classpath, I receive the following exception:
Traceback (innermost last):
File "test.jy", line 10, in ?
at clojure.lang.Namespace.<init>(Namespace.java:34)
at clojure.lang.Namespace.findOrCreate(Namespace.java:176)
at clojure.lang.Var.internPrivate(Var.java:149)
at aot_demo.JavaClass.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
java.lang.ExceptionInInitializerError: java.lang.ExceptionInInitializerError
I'm able to reproduce this with the following trivial project.clj:
(defproject aot-demo "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.3.0"]]
:aot [aot-demo.core])
...and src/aot_demo/core.clj defined as follows:
(ns aot-demo.core
(:gen-class
:name aot_demo.JavaClass
:methods [#^{:static true} [lower [java.lang.String] java.lang.String]]))
(defn -lower [str] (.toLowerCase str))
The following Jython script is then sufficient to trigger the bug:
#!/usr/bin/jython
import java.lang.Class
import java.net.URLClassLoader
import java.net.URL
import os
customLoader = java.net.URLClassLoader(
[java.net.URL('file://%s/target/aot-demo-0.1.0-SNAPSHOT-standalone.jar'
% (os.getcwd()))])
java.lang.Class.forName('aot_demo.JavaClass', True, customLoader)
However, the exception does not occur if the test script is started with the uberjar already in the CLASSPATH variable.
What's going on here? I'm trying to write a plugin for the BaseX database in Clojure; the above accurately represents how their plugin-loading mechanism works for the purpose of providing a SSCE for this problem.
The plugin-loading mechanism used by BaseX should be extended to modify the current thread's context classloader. For the given sample code (in Jython), this would look something like the following:
currentThread = java.lang.Thread.currentThread()
oldLoader = currentThread.getContextClassLoader()
currentThread.setContextClassLoader(customLoader)
try:
cls = java.lang.Class.forName('aot_demo.JavaClass', True, customLoader)
finally:
currentThread.setContextClassLoader(oldLoader)
Related
I am creating a spark context by using-
(ns something
(:require [flambo.conf : conf]
[flambo.api :as f]))
(def c (-> (conf/spark-conf)
(conf/master "spark://formcept008.lan:7077")
(conf/app-name "clustering"))) ;; app-name
(def sc (f/spark-context c))
Then I am creating an RDD-
(f/parallelize sc DATA)
Now when I am performing some action on this data, like (f/take rdd 3) etc, I am getting an error-
17/11/28 14:35:00 ERROR Utils: Exception encountered
org.apache.spark.SparkException: Failed to register classes with Kryo
at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:129)
at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:274)
at org.apache.spark.serializer.KryoSerializerInstance.(KryoSerializer.scala:259)
at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:175)
at org.apache.spark.rdd.ParallelCollectionPartition$$anonfun$readObject$1.apply$mcV$sp(ParallelCollectionRDD.scala:79)
at org.apache.spark.rdd.ParallelCollectionPartition$$anonfun$readObject$1.apply(ParallelCollectionRDD.scala:70)
at org.apache.spark.rdd.ParallelCollectionPartition$$anonfun$readObject$1.apply(ParallelCollectionRDD.scala:70)
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1273)
at org.apache.spark.rdd.ParallelCollectionPartition.readObject(ParallelCollectionRDD.scala:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: flambo.kryo.BaseFlamboRegistrator
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$5.apply(KryoSerializer.scala:124)
at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$5.apply(KryoSerializer.scala:124)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:124)
... 27 more
17/11/28 14:35:00 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.IllegalStateException: unread block data
at java.io.ObjectInputStream$BlockDataInputStream.setBlockDataMode(ObjectInputStream.java:2449)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1385)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Any thoughts on that, please.
flambo doesn't seem to be in your classpath somehow which is why you're getting:
java.lang.ClassNotFoundException: flambo.kryo.BaseFlamboRegistrator
Are you running this from the REPL or are you using a lein or boot task?
if you're using leiningen check your classpath (lein classpath) and dependency tree (lein deps :tree)
Also, it never hurts to do a lein clean to make sure your target folder isn't causing issues
Stack trace analysis:
The Failed to register classes with Kryo is caused because flambo.kryo.BaseFlamboRegistrator is missing
Solved.
Add all the jars of your project in spark-configuration by using -
(conf/jars (map #(.getPath % (.getURLs(java.lang.ClassLoader/getSystemClassLoader))))
It will register all the classes.
Since this issue is resolved, so closing it.
use code like the following to generate a class Greeting.
project.clj
(defproject greeting "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.7.0"]]
:aot [greeting.core]
)
src/greeting/core.clj
(ns greeting.core
(:gen-class
:name Greeting
:init create
:constructors {[String] []}
:methods [(greet [String] String)]
:state data))
(defn -create
"Construct instance with a String."
[s]
[[] ;; super class args
s])
(defn -greet
"Return greeting based on the constructed data."
[this n]
(str (.data this) " " n "!"))
these code are from http://www.coderanch.com/t/601586/clojure/Calling-Clojure-Java-code. can be called from a java class.
now compile to jar by
lein uberjar
and import the standalone jar to WSO2 AS server, and got the following error
Error: java.lang.ExceptionInInitializerError at clojure.lang.Namespace. (Namespace.java:34) at clojure.lang.Namespace.findOrCreate(Namespace.java:176) at clojure.lang.Var.internPrivate(Var.java:151) at Greeting.(Unknown Source) at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:278) at
org.apache.axis2.description.java2wsdl.DefaultSchemaGenerator.(DefaultSchemaGenerator.java:140) at
org.apache.axis2.deployment.util.Utils.fillAxisService(Utils.java:453) at
org.apache.axis2.deployment.ServiceBuilder.populateService(ServiceBuilder.java:397) at
org.apache.axis2.deployment.ServiceGroupBuilder.populateServiceGroup(ServiceGroupBuilder.java:101) at
org.apache.axis2.deployment.repository.util.ArchiveReader.buildServiceGroup(ArchiveReader.java:109) at
org.apache.axis2.deployment.repository.util.ArchiveReader.processServiceGroup(ArchiveReader.java:143) at
org.apache.axis2.deployment.ServiceDeployer.deploy(ServiceDeployer.java:82) at
org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136) at
org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807) at
org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144) at
org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377) at
org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254) at
org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371) at
org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59) at
org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67) at
org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:79) at
org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:124) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by:
java.io.FileNotFoundException: Could not locate clojure/core__init.class or clojure/core.clj on classpath. at clojure.lang.RT.load(RT.java:449) at
clojure.lang.RT.load(RT.java:412) at clojure.lang.RT.doInit(RT.java:454) at
clojure.lang.RT.(RT.java:330) ... 30 more
Is there something I missed? or how to get rid of it?
For Axis2 use separate Class Loader for each service, and Clojure Class Loader use context class loader by default, so when the jar file is loading, and the Clojue object in AOT-class will be loaded by Clojure class loader, and the Clojure class loader, the context class loader, do not have information about the jar's path, and so core__init.class could not be found.
to fix it, by using the axis2's costum deployer, overload the depoly method,
and setting the context class loader the same as the service class loader.
this answer give much help to me.
and some other reference could give a hint.thanks all!
In api docs for nrepl-middleware wrap-javadoc it says that it 'accepts local-paths a space separate list of paths'
How do I set this?
I've tried the following in my profiles.clj
{:user {:plugins []
:jvm-opts ["-Xmx4G"]
:injections [(require 'clojure.repl)]
:warn-on-reflection true
:dependencies [[ritz/ritz-nrepl-middleware "0.7.0"]]
:repl-options {:nrepl-middleware
[ritz.nrepl.middleware.javadoc/wrap-javadoc :local-paths "/usr/local/share/javadocs/7/docs/api"]
}}}
But that causes the following exception when I execute nrepl-jack-in
Starting nREPL server...
Mark set
error in process sentinel: nrepl-server-sentinel: Could not start nREPL server: Reflection warning, NO_SOURCE_PATH:1:483 - reference to field getLocalPort can't be resolved.
Exception in thread "main" java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IFn
at clojure.core$comp$fn__4166.doInvoke(core.clj:2347)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.tools.nrepl.server$default_handler.doInvoke(server.clj:89)
at clojure.lang.RestFn.invoke(RestFn.java:436)
at user$eval1192.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:6619)
at clojure.lang.Compiler.eval(Compiler.java:6609)
at clojure.lang.Compiler.eval(Compiler.java:6582)
at clojure.core$eval.invoke(core.clj:2852)
at clojure.main$eval_opt.invoke(main.clj:308)
at clojure.main$initialize.invoke(main.clj:327)
at clojure.main$null_opt.invoke(main.clj:362)
at clojure.main$main$fn__6661.invoke(main.clj:440)
at clojure.main$main.doInvoke(main.clj:437)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
Subprocess failed
If I wrap :local-paths "..." in a map I don't get any javadoc middleware
If you look at the example of the nrepl-middleware on its github page, you will find that the vector to :nrepl-middleware sbould be a vector of middleware functions where as you are passing it a string path and that's why the exception "String cannot be cast to IFn".
As far as the local-paths is concerned that's something which is send by the nrepl client in the java-doc request itself. You can set local-paths in Emacs by calling:
(setq nrepl-ritz-javadoc-local-paths (list "/usr/local/share/javadocs/7/docs/api"))
I'm trying to learn Clojure, for this I installed the latest Clojure and Leiningen, so I created a new lein project and add the :main helloclojure.core entry in to project.clj but when I try to run the project I get the following error:
Exception in thread "main" java.lang.ClassNotFoundException: helloclojure.core
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at clojure.lang.DynamicClassLoader.findClass(DynamicClassLoader.java:61)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at clojure.lang.RT.classForName(RT.java:2039)
at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:207)
at user$eval12.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:6511)
at clojure.lang.Compiler.eval(Compiler.java:6501)
at clojure.lang.Compiler.eval(Compiler.java:6477)
at clojure.core$eval.invoke(core.clj:2797)
at clojure.main$eval_opt.invoke(main.clj:297)
at clojure.main$initialize.invoke(main.clj:316)
at clojure.main$null_opt.invoke(main.clj:349)
at clojure.main$main.doInvoke(main.clj:427)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
But I have no idea what is wrong.
Here is my project.clj file:
(defproject helloclojure "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:main helloclojure.core
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.4.0"]])
Any help will be much appreciated.
When you add the :main keyword to your project.clj like this, it is expected to point to a namespace that includes a function named -main. This namespace should generate a real Java class, and you can do this using the :gen-class keyword in your namespace definition.
So for this to work I would expect you to have a source file at:
src/helloclojure/core.clj
that contains at least this content:
(ns helloclojure.core
(:gen-class))
(defn -main [& args]
(println args))
I'm trying to write a Bukkit plugin in Clojure.
Generally a Bukkit plugin needs to override the Plugin class, have an onEnable() and a onDisable() method.
There are also a few other requires, like the result being in a JAR file and a plugin.yml defined, but I have all that.
(ns net.jonnay.watershipdown.WatershipDown
(:import org.bukkit.Bukkit
org.bukkit.plugin.Plugin)
(:gen-class
:name net.jonnay.watershipdown.WatershipDown
:extends org.bukkit.plugin.Plugin
:methods [ [onEnable [] void]
[onDisable [] void] ]
)
)
(set! *warn-on-reflection* true)
(defn debug-to-mc-log [^String msg]
(let [logger (Bukkit/getLogger)]
(. logger info (str "(DEBUG) " msg))))
(defn -onEnable []
(debug-to-mc-log "Enabled Watership down. Super Clojure Powers!"))
(defn -onDisable []
(debug-to-mc-log "Disabled Watership down."))
My code compiles just fine, but when I try to load it in the server, I get this exception:
13:29:20 [SEVERE] Could not load 'plugins/watershipdown-0.1.jar' in folder 'plugins':
java.lang.ClassFormatError: Duplicate method name&signature in class file net/jonnay/watershipdown/WatershipDown
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at org.bukkit.plugin.java.PluginClassLoader.findClass(PluginClassLoader.java:41)
at org.bukkit.plugin.java.PluginClassLoader.findClass(PluginClassLoader.java:29)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.bukkit.plugin.java.JavaPluginLoader.loadPlugin(JavaPluginLoader.java:131)
at org.bukkit.plugin.SimplePluginManager.loadPlugin(SimplePluginManager.java:285)
at org.bukkit.plugin.SimplePluginManager.loadPlugins(SimplePluginManager.java:200)
at org.bukkit.craftbukkit.CraftServer.loadPlugins(CraftServer.java:156)
at org.bukkit.craftbukkit.CraftServer.<init>(CraftServer.java:132)
at net.minecraft.server.ServerConfigurationManager.<init>(ServerConfigurationManager.java:52)
at net.minecraft.server.MinecraftServer.init(MinecraftServer.java:148)
at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:407)
at net.minecraft.server.ThreadServerApplication.run(SourceFile:465)
Are onEnable and onDisable defined in Plugin?
If so, they might be causing the issue.
The genClass documentation specifically says that you shouldn't redeclare inherited methods.