I can create an uberjar that is composed of lots of class files, originally Scala, Java, Clojure. The problem I have is that when I run java -jar my-server.jar it crashes with:
No configuration setting found for key 'akka.version'
This is to be expected and has a maven solution. The yellow writing on the accepted answer here is basically Akka saying "you shouldn't build uberjars with Akka jars in them, as then Akka won't be able to find its .conf files."
I am trying this as a lein solution:
:pom-plugins [[org.apache.maven.plugins/maven-shade-plugin 2.2]]
I have a local maven repository (by this I mean not the ~/.m2 one, but a local one that is used to introduce non-Clojars jars into the lein build). Maybe I need to lein deploy localrepo1 for the akka jars again to pick up this new setting - Nope - that didn't help.
Here's some of the stack trace to make it clear where the problem comes from:
Exception in thread "main" com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.version'
at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:124)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:145)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:151)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164)
at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:206)
at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:169)
at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:505)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:109)
at com.seasoft.comms.MyPLCActorHolder.createRefToLocalActor(MyPLCActorHolder.scala:39)
Edit I've now looked inside the jar files. There are two akka jar files that both have a reference.conf in them. These files are not correctly merged because (unsurprisingly) lein uberjar doesn't understand the nesting of the property key/values within them.
Specifically the reference.conf in akka-actor_2.11-2.3.9.jar has akka.version = "2.3.9", but this entry has not made it to the merged reference.conf. I altered the uberjar and that fixed the problem, of course giving me the next merge problem. So the fix here is to manually do the merge myself.
And the better fix would be to write a little merging program (with two functions: a predicate and merge) and get it into akka so that people who write build tools can just use it...
Manually merging reference.conf from the two jars and then manually altering the uberjar (overwriting the existing reference.conf) did the trick. The merged file is here.
I believe this problem will occur anytime there are property files that come from different jars that have the same name. For instance log4j.properties also needed to be overwritten.
I changed spark version to avoid this kind of problem. Spark 2.1.0 does not depend on Akka, and does not have the Akka reference.conf problem.
Related
I'm trying to build a static site with stasis and serve my assets with Optimus. Pictures reside under /resources/public/imgs/ . I can serve individual pictures after loading them as follows:
(optimus.assets/load-assets "public"
["/imgs/pic1.jpg"
"/imgs/pic2.jpg"])
The following attempt to serve pictures by regex does not work though:
(optimus.assets/load-assets "public"
[#"/imgs/.*\.jpg"])
I'm getting No files matched regex /imgs/.*\.jpg, which seems implausible.
I've done some digging around the Optimus code and may have found the culprit. When called with a regex the optimus.assets/load-assets function starts building the paths from the return value of (optimus.class-path/file-paths-on-class-path), which – in my case – consists of only the following:
optimus.class-path/file-paths-on-class-path
=> ("boot/" "boot/tag-release.properties" "boot/bin/" "boot/bin/ParentClassLoader.class" "Boot.class" "META-INF/" "META-INF/MANIFEST.MF")
Since resources isn't a subdir of any of these directories I'm not surprised that I don't get a match. So maybe my question ultimately is why I'm getting only these directories here? Is it because I'm using Boot rather than Leiningen which is presupposed by the tutorials on Optimus?
EDIT:
It has definitely something to do with Boot or at least the way I've set it up. Following Alan Thompson's advice I've created a minimal Leiningen project – load-assets worked flawlessly. The very same setup with Boot however does not. Ultimately it boils down to (System/getProperty "java.class.path" ".") returning wildly different things: Boot gives me "/home/phylax/bin/boot", i.e. my boot binary, whereas in Leiningen it gives me a plethora of directories in my actual project… any idea as to what I'm doing wrong? How can I setup Boot to work with Optimus?
Many thanks for any guidance you may give me on this
Oliver
I have a clojure script that I've been running for several years now.
Recently I had to do a reinstall of leiningen and java 8.
When I run the code I now get multiple StackOverflowErrors on third party libraries.
E.g.
StackOverflowError org.apache.commons.math3.util.FastMath.cosQ (FastMath.java:1850)
and
StackOverflowError clojure.lang.Numbers$DoubleOps.combine (Numbers.java:571)
I would post code but it happens now at multiple points in my code and on third party libraries??
without a stack trace and some code, a specific anser will be hard to find, so here is a general method for these things:
Part 1 "what was running before":
recreate your old environment, by digging through the logs, reverting to a backup etc. then run:
lein deps :tree 2>&1 > old-lein-dependencies
the 2>&1 part ensures that the version range and conflict warnings
are included in the output.
Mark down the leiningen version and java version:
lein version
Part 2: "what is running now"
Repeat the steps and record the same information as before:
lein deps :tree 2>&1 > new-lein-dependencies
lein version
java -version
Part 3: Diff and Compare
pick through all the differences
diff -u old-lein-dependencies new-lein-dependencies
there will be a big block of differences at the top where lein prints all the important warnings. The final clue is almost always here, though it's often not easy to recognise up right away.
Part 4: Do Science
go through every version change, starting from the initial configuration by pinning the versions in the project.clj until you find the change that breaks things. A convenient way to pin these is with the :managed-dependencies block in the project.clj file. It looks something like this:
:managed-dependencies [[http-kit "2.3.0-alpha4"]]
and repeat the process of switching out versions till you get a handle on where the change was introduced. For me this has almost always been the result of using a version range in a dependency rather than a specific version. I'm not too enthusiastic about version ranges anymore :-/
So I could not compare the previous setup as it was on a machine that was wiped clean.
I found that an error had crept into one of the math formulas executed by the tool which basically called itself repeatedly resulting in the stack overflow error.
After defining variables, functions, etc., can you save what you have done on the REPL too an text .clj file?
most people work with the repl through an editor such ad Eclipse/Emacs/vim and that editor has the ability to save the repl, though without some diligence on the developers part this will likely be an incomplete record of what happened. Some of the state of the repl may have come from loading files etc which will be in a different state.
So the short answer is typically not.
In Linux (mine = Ubuntu 16.04.2 LTS) if you are using lein then check for .lein (hidden directory) and look for repl-history. You should find the commands that you have typed or pasted into the REPL. This can be a source for later edit - I use geany...
I am answering the parenthetical part of your question. For me, the Clojure REPL is very useful for testing functions and proving out concepts that take no more than a few lines. I will often put hooks in a module that is not the main, just so I can load a file and run it through a couple of functions. I can also do this from main using the same mindset; that is write a debug function.
I found the Eclipse plugin to be quite useful, but I do not use it much these days, mostly Vim and running the module with one or more special functions and running the main. I don't know of any way to save REPL state.
I'd like to create a new Gradle project without any sources. I'm going to put there some configuration files and I want to generate a zip file when I build.
With maven I'd use the assembly plugin. I'm looking for the easiest and lightest way to do this with Gradle. I wonder if I need to apply the java plugin even if I don't have any sources here, just because it provides some basic and useful tasks like clean, assemble and so on. Generating a zip is pretty straightforward, I know how to do that, but I don't know where and how to put the zip generation within the gradle world.
I've done it manually until now. In other words, for projects where all I want to do is create some kind of distro and I need the basic lifecycle tasks like assemble and clean, I've simply created those tasks along with the needed dependencies.
But there is the 'base' plugin (mentioned under "Base plugins" of the "Standard Gradle Plugins" in the user's guide) that seems to fit the bill nicely for this functionality. Note though that the user guide mentions that this and the other base plugins are not yet considered part of the Gradle API and are not really documented.
The results are pretty much identical to yours, the only difference being that there are no confusing java specific tasks that always remain UP-TO-DATE.
apply plugin: 'base'
task dist(type: Zip) {
from('solr')
into('solr')
}
assemble.dependsOn(dist)
Sample run:
$ gradle clean assemble
:clean
:dist
:assemble
BUILD SUCCESSFUL
Total time: 2.562 secs
As far as I understood, it might sound strange but looks like I need to apply the java plugin in order to create a zip file. Furthermore it's handy to have available some common tasks like for example clean. The following is my build.gradle:
apply plugin: 'java'
task('dist', type: Zip) {
from('solr')
into('solr')
}
assemble.dependsOn dist
I applied the java plugin and defined my dist task which creates a zip file containing a solr directory with the content of the solr directory within my project. The last line is handy to have the task executed when I run the common gradle build or gradle assemble, since I don't want to explicitly call the dist task.
This way if I work with multiple projects I just need to execute gradle build on the parent to generate all the artifacts, including the configuration zip.
Please let me know if you have better solutions and add your own answer!
You could just use the groovy plugin and use ant. I did something like this. I do also like javanna's answer.
task jars(dependsOn: ['dev_jars']) << {
def fromDir = file('/database-files/non_dev').listFiles().sort()
File dist = new File("${project.buildDir}/dist")
dist.mkdir()
fromDir.each { File dir ->
File destFile = new File("${dist.absolutePath}" + "/" + "database-connection-" + dir.name + ".jar")
println destFile.getAbsolutePath()
ant.jar(destfile:destFile, update:false, baseDir:dir)
}
}
I am trying to go through the process of creating a jar file from a simple clojure file. Below is my clojure code:
(ns app.first (:gen-class))
(refer 'clojure.core)
(defn -main [& args] (println "this program worked!"))
I am using these instructions to create the jar file: http://en.wikibooks.org/wiki/Clojure_Programming/Tutorials_and_Tips
I see the error "java.io.FileNotFoundException: Could not locate app/hello__init.class or app/hello.clj on classpath: (NO_SOURCE_FILE:0)" when I try to complete the (compile 'app.first) step.
The only difference between my attempt and the link is the name of my file (first.clj instead of hello.clj).
Can anyone see where I am going wrong? Or for that matter, all I want to do is learn how to create a jar from a clojure file, so if anyone knows of a better/easier way to do that, let me know.
It's better to use Leiningen for such tasks - it allows to maintain dependencies, and packs all necessary components into jar file
I'm rusty on this, but I heard about other people with similar problems.
I think it's helpful to remember that the classpath you indicate points to the root of your class tree, and package names end up creating subdirectories within that tree. Awkwardly stated, but I hope you get the idea. Thus, I think you need to do some kind of gymnastics with creating directories to match the "app.first" -> "/app/first" hierarchy.
Sorry, that's as close as I come to a sensible and useful answer. Hope this helps you.
EDIT:
The Prime Directive of Computer Science: It only works if you do everything right! I spent almost 10 minutes fiddling with this but was finally successful.
Here's what I needed to do to get your program to compile:
created a directory app, and within that, first.clj with your code.
checked for the *compile-path* by doing (pr *compile-path) within Clojure. It said "classes".
created a second directory classes parallel to app.
in the shell, did export CLASSPATH=.:./classes
in Clojure, did (compile 'app.first)
... and I found a bunch of class files in classes. JARring those should be a snap.
I found it very helpful to run (doc compile) because that reminded me of the requirement to have a directory to satisfy the requirement for a *compile-path*.