clojure classpath(?) troubles loading code and dependencies with leiningen - clojure

I'm a Clojure and JVM beginner, and the build tools are a bit confusing to me, so forgive the stupidity I'm about to carry out. I don't actually know what a "classpath" is... I just want a Lisp with lots of libraries!
But I'm trying to do a write a simple little bit of text-mining code in Clojure, and I don't seem to be able to get things working.
There are two problems, and I think they're related, both probably to this classpath thing from javaland that none of the introductory Clojure books seem to explain.
I started the project with lein new and my project.clj has the following dependencies:
:dependencies [[org.clojure/clojure "1.6.0"]
[clj-fuzzy "0.1.8"]
[snowball-stemmer "0.1.0"]
[net.mikera/core.matrix "0.50.0"]])
and the relevant ns call from my core.clj is as follows:
(ns blabber.core
(:require
[clojure.string :refer [lower-case split]]
[clojure.walk :refer [keywordize-keys]]))
My directory structure is the default setup leiningen uses for a new library.
First problem:
I gather from this prior SO that if things are set up right, I shouldn't need to call (load-file "src/blabber/core.clj") in the repl before calling (use 'blabber.core) in order to get at code I just wrote. (I'm firing up the lein repl from the src/blabber directory in bash). But when I fire up the repl, use fails unless I call load-file first. So does that mean my classpath is set up wrong somehow? And how does one fix such a thing?
Second problem:
Ok, so now I'm trying to actually use core.matrix. First line of my core.clj changed to:
(ns blabber.core
(:require
[clojure.string :refer [lower-case split]]
[clojure.walk :refer [keywordize-keys]]
[clojure.core.matrix :refer [dataset]]))
When I first started the repl with this project.clj but without the :require to core.matrix, leiningen happily fetched core.matrix: Retrieving net/mikera/core.matrix/0.50.0/core.matrix-0.50.0.jar from clojars
However, after including the :require to core.matrix in the actual code and starting up the repl again, at the repl I call (load-file "src/blabber/core.clj"), and get the following error:
CompilerException java.io.FileNotFoundException: Could not locate clojure/core/matrix__init.class or clojure/core/matrix.clj on classpath: , compiling:(/Users/pauliglot/github/blabber/src/blabber/core.clj:1:1)
I'm using Leiningen 2.5.1 on Java 1.8.0_45. And I'm just following the getting started guide for core.matrix.
So what happened? Why can't the JVM find any of my stuff? Save me? Thanks!
If it helps, here's a dump of the output of lein classpath
/Users/pauliglot/github/blabber/test:/Users/pauliglot/github/blabber/src:/Users/pauliglot/github/blabber/dev-resources:/Users/pauliglot/github/blabber/resources:/Users/pauliglot/github/blabber/target/classes:/Users/pauliglot/.m2/repository/clj-fuzzy/clj-fuzzy/0.1.8/clj-fuzzy-0.1.8.jar:/Users/pauliglot/.m2/repository/snowball-stemmer/snowball-stemmer/0.1.0/snowball-stemmer-0.1.0.jar:/Users/pauliglot/.m2/repository/org/clojure/tools.macro/0.1.5/tools.macro-0.1.5.jar:/Users/pauliglot/.m2/repository/clojure-complete/clojure-complete/0.2.3/clojure-complete-0.2.3.jar:/Users/pauliglot/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20140226-71326067/google-closure-library-third-party-0.0-20140226-71326067.jar:/Users/pauliglot/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar:/Users/pauliglot/.m2/repository/org/clojure/google-closure-library/0.0-20140226-71326067/google-closure-library-0.0-20140226-71326067.jar:/Users/pauliglot/.m2/repository/org/mozilla/rhino/1.7R4/rhino-1.7R4.jar:/Users/pauliglot/.m2/repository/org/json/json/20090211/json-20090211.jar:/Users/pauliglot/.m2/repository/net/mikera/core.matrix/0.50.0/core.matrix-0.50.0.jar:/Users/pauliglot/.m2/repository/com/google/javascript/closure-compiler/v20131014/closure-compiler-v20131014.jar:/Users/pauliglot/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/pauliglot/.m2/repository/org/clojure/clojurescript/0.0-2202/clojurescript-0.0-2202.jar:/Users/pauliglot/.m2/repository/args4j/args4j/2.0.16/args4j-2.0.16.jar:/Users/pauliglot/.m2/repository/org/clojure/tools.nrepl/0.2.6/tools.nrepl-0.2.6.jar:/Users/pauliglot/.m2/repository/com/google/guava/guava/15.0/guava-15.0.jar:/Users/pauliglot/.m2/repository/org/clojure/clojure/1.6.0/clojure-1.6.0.jar:/Users/pauliglot/.m2/repository/org/clojure/tools.reader/0.8.3/tools.reader-0.8.3.jar:/Users/pauliglot/.m2/repository/org/clojure/data.json/0.2.3/data.json-0.2.3.jar
Edit
Here's the directory tree:
| blabber
|-doc
|-resources
|-src
|---blabber
|-target
|---classes
|-----META-INF
|-------maven
|---------blabber
|-----------blabber
|---stale
|-test
and all my code is in blabber/src/blabber/core.clj

A classpath is a Java concept; Leiningen should set up the classpath correctly. What I suspect is that there's a mismatch between the directories, filenames, or namespace declarations. The file path should be <name passed to 'lein new'>/src/blabber/core.clj. It wasn't clear from what you wrote whether that's the case. Try running lein deps first, too, but you shouldn't need to do that. This and lein repl are usually run from the directory, i.e. the project root directory. You might need :source-paths or :main in project.clj, but I don't think so.
(use 'clojure.core.matrix) should work. Works for me, e.g. with
(defproject blabber "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.8.0"]
[net.mikera/core.matrix "0.50.0"]]).

Related

Dependencies downloaded but still cannot require, clojure

This kills me... I cannot require anything other than built-in native deps. No hiccup, no http-kit etc... Even if I can find them on my hard drive in .m2/repository
lein new myapp,
add [markdown-clj 0.9.91] to project.clj,
add (ns metapp.core
(:require [markdown-clj :as mark]) )
lein run
Retrieving markdown-clj/markdown-clj/0.9.91/markdown-clj-0.9.91.pom from clojars
Retrieving markdown-clj/markdown-clj/0.9.91/markdown-clj-0.9.91.jar from clojars
Exception in thread "main" java.io.FileNotFoundException: Could not locate quote/markdown_clj__init.class or quote/markdown_clj.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.,
lein deps is not returning anything
Somebody knows what's wrong please? This stacktrace isn't very helpful, so Lein can fetch deps, but doesn't know how to require them?
EDIT: Running Linux Mint 18.0, clojure is in /home/denis/clojure-1.8.0 and is called by alias java -cp /home/denis/clojure-1.8.0/clojure-1.8.0.jar clojure.main. Directory tree in myapp is /home/denis/prg/cljr/myapp
SOLUTION:
Thank you guyz, but now feel like an idiot.
So to summarize for future-comers, in project.clj, to ask for a dependency "X" doesn't mean you should "require" it as "X". You must require it the way the author specified in documentation, for example [http-kit "2.2.0"] in project.clj is required as follows
(ns metapp.core (:require [org.httpkit.client :as http] ).
Second, the way you require inside your code is not the same as the way you require in REPL, for example, this works in yourapp.core (require [stuff.core as stuff]). You could also write it like this, it works too (ns yourns (:require [stuff.core :as stuff]). But this synthaxe doesnt work: (:require [stuff.core :as stuff]).
In REPL however, it's different story! I must use (:require '[stuff.core]) if it is an added dependency, or (:require 'clojure.string) if it is a built-in library! Note that something like (require '[http.async.core]) doesn't work because it is not built-in. So if you checked the documentation https://clojuredocs.org/clojure.core/require which only shows built-in examples, like me, you are doomed.
Also for built-in library like clojure.string you can use simply (require 'clojure.string), yup, the one which didn't worked with dependencies. Have fun guys! LOOOOONNG journey ahead, clojure is only language so far I needed to spend 4 days figuring out how to IMPORT modules (poke Python, it only took 30 seconds), hope it worth it!
You should require markdown.core. From the documentation for that project:
(ns foo
(:use markdown.core))
In your case:
(ns metapp.core
(:require [markdown.core :as mark]))
should work.
Not realising that the name of the library and the namespaces that make up the library are different things is something that is easy to be tripped up by.
markdown-clj is just a name of a package. But when you require something, you need to specify a module, not a package. Most of the packages have core module so the proper usage would be:
(:require [markdown-clj.core :as mark])

Distributing a simple library via clojars

I have implemented a hyphenation algorithm (at namespace hyphenator-clj.core), defined it as org.clojars.nikonyrh.hyphenator-clj 0.1.0 at defproject and pushed it to Clojars. Uberjar seems to have files like core__init.class, core.clj and core.class.
However when I try to use it as a dependency on an other project I get this error:
$ lein uberjar
Retrieving org/clojars/nikonyrh/hyphenator-clj/org.clojars.nikonyrh.hyphenator-clj/0.1.0/org.clojars.nikonyrh.hyphenator-clj-0.1.0.pom from clojars
Retrieving org/clojars/nikonyrh/hyphenator-clj/org.clojars.nikonyrh.hyphenator-clj/0.1.0/org.clojars.nikonyrh.hyphenator-clj-0.1.0.jar from clojars
Compiling example.core
java.io.FileNotFoundException: Could not locate org/clojars/nikonyrh/hyphenator_clj__init.class or org/clojars/nikonyrh/hyphenator_clj.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name., compiling:(core.clj:1:1)
Exception in thread "main" java.io.FileNotFoundException: Could not locate org/clojars/nikonyrh/hyphenator_clj__init.class or org/clojars/nikonyrh/hyphenator_clj.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name., compiling:(core.clj:1:1)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3657)
at clojure.lang.Compiler.compile1(Compiler.java:7474)
at clojure.lang.Compiler.compile1(Compiler.java:7464)
at clojure.lang.Compiler.compile(Compiler.java:7541)
at clojure.lang.RT.compile(RT.java:406)
at clojure.lang.RT.load(RT.java:451)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__5677.invoke(core.clj:5893)
...
Must I change my project's folder structure so that it matches the expected org/clojars/nikonyrh/hyphenator_clj__init.class, or can I somehow override the current behavior? If there is a good tutorial about this out there I would be happy to read it.
Basically I would like to get this example project to work. project.clj:
(defproject example "0.0.1-SNAPSHOT"
:description ""
:dependencies [[org.clojure/clojure "1.8.0"]
[org.clojars.nikonyrh.hyphenator-clj "0.1.0"]]
:javac-options ["-target" "1.6" "-source" "1.6" "-Xlint:-options"]
:aot [example.core]
:main example.core)
src/example/core.clj:
(ns example.core
(:require [org.clojars.nikonyrh.hyphenator-clj :as h])
(:gen-class))
(defn -main [& argv] (doseq [arg argv] (println (h/hyphenate arg :hyphen \-))))
I'm suspecting I also have the english.txt in a wrong directory, as it isn't contained in the uberjar but resource files are an other topic.
You would probably be better off not using a hyphen in the namespace hyphenator-clj. Why not just use hyphenator? But if you do I suspect that the name of the directory should have an underscore in it rather than a hyphen, so be: hyphenator_clj.
If fixing that issue doesn't help then another thing to look out for, which I can't see from your question, is where exactly is core.clj in the directory structure, and does the project.clj reflect that? For instance is the path for the namespace hyphenator-clj.core in a src directory off the root of your project? The root of your project being defined as where the project.clj is located.
Something else that would be good to see in the question is whether you can get the program to work just locally, without having packed it up into an uberjar and shipped it to clojars. My guess would be that it does work locally, but it would help for that to be stated.
Okay taking a look at your links now. You might like to read a working project deployed to clojars, that has hypens in its name, for instance here. The first difference you might notice is the project name you use is rather long: org.clojars.nikonyrh.hyphenator-clj. It should just be hyphenator-clj. Also I would recommend having an identifier ending in "-SNAPSHOT" as that project does.
But taking a look at the bigger picture, a great idea you suggested in the comments is to test without Clojars being in the mix at all. To do this use lein install on the library you want to use from another lein project.
Ahaa at least I understand the process a bit better now, basically my require has to match the structure of the used JAR file, which may be very different from the project's name. For example cc.qbits/spandex is actually required as qbits.spandex.
The english.txt dependency was fixed by moving it to resources folder, deploying the new version to Clojars and importing the dependency as it exists in the JAR:
(ns example.core
(:require [hyphenator-clj.core :as h])
(:gen-class))
(defn -main [& argv] (doseq [arg argv] (println (h/hyphenate arg :hyphen \-))))

lein - how to use a downloaded library

Let's say I find a cool clojure library like https://github.com/clojurewerkz/buffy
Now I want to use it. And it only lives on github.
How do I do this? I would love a full start to finish hello world example.
I've read about compiling it as a jar and using that, or using :dependencies in my project.clj but so far no examples have been complete, and I'm new.
For example in python I'd git clone the repo into the root of my working tree and any file could just say import buffy
I just learned this five minutes ago. I wanted to use the clojure-csv library, here's what I did using Leiningen
Make a new leiningen project
lein new app csvtest
Now I have a folder called csvtest/. In the csvtest/project.clj file, edit the dependencies section to add the clojure-csv github path and a version. I'm not quite sure how the versioning works to be honest:
:dependencies [[org.clojure/clojure "1.5.1"]
[clojure-csv/clojure-csv "2.0.1"]]
Now run lein deps to automagically download any unresolved dependencies for your project
$ lein deps
...
Retrieving clojure-csv/clojure-csv/2.0.1/clojure-csv-2.0.1.pom from clojars
...
Now edit csvtest/src/csvtest/core.clj. Make it look like this:
Edit: (Based on sveri's comment I changed :use closjure-csv.core to the :require line)
(ns csvtest.core
(:gen-class)
(:require [clojure-csv.core :refer [parse-csv]]))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println (parse-csv "1,2,3,hello,world")))
You have to add the (:require [clojure-csv.core :refer [parse-csv]]) line to your ns, and in main it calls parse-csv, one of the functions provided by the library. From reading, it looks like :use is depreciated in favor of :require.
Note: For anyone coming from python (like me), it looks like doing :use clojure-csv.core is akin to from closure_csv import *, which is bad practice in python as well. And (:require [clojure-csv.core :refer [parse-csv]]) is like from clojure_csv import parse_csv
Run the project with lein run. My output was
$ lein run
([1 2 3 hello world])
I hope that helps!
You can download/use it from Clojars.
Clojars link:
https://clojars.org/clojurewerkz/buffy
To use it with Leiningen, add this to your dependencies(on project.clj file):
[clojurewerkz/buffy "1.0.0-beta4"]
After that, you can run lein deps from your project root folder in order to download dependencies.
To use dependencies directly from Github repos, you can check out this: https://github.com/tobyhede/lein-git-deps
Using lein (which is Clojure's build tool) you have a project.clj file. if you do lein new project-name you get a new project named "project-name", with a project.clj file in it. look at that file, you will see a :dependencies entry in it as in this example:
:dependencies [[org.clojure/clojure "1.5.1"]]
all you must do to include buffy, is look at their github site, and find out which version to use, then you add it to the :dependecies list:
:dependencies [[org.clojure/clojure "1.5.1"]
[clojurewerkz/buffy "1.0.0-beta3"]]
now you can simply do lein deps which will download the dependencies and once you written your code lein run to actually run it.
read a little about lein and check out its sample project.clj file to get a better understanding of how you should use clojure.

Leiningen and Clojure dependencies

I'm fairly new to clojure but I've been having trouble finding good resources and examples online, so hopefully someone could point me in the right direction. I've started a project with lein, project.clj looks like this:
(defproject scratch "1.0"
:description ""
:main scratch.core
:dependencies [
[org.clojure/clojure "1.3.0"]
[org.clojars.jyaan/slick "247.1"]
])
and src/scratch/core.clj looks like this:
(ns scratch.core
(:import org.newdawn.slick))
(defn -main [] (println "hello world"))
As far as I can tell this is right, but when I try to run lein run I get a ClassNotFoundException.
I did a jar tf on the lib/slick-247.1.jar file and confirmed it has a directory structure which would indicate that it has that namespace (org/newdawn/slick/etc...). I'm sure it's a simple mistake but I can't see what it is, does anyone have any ideas?
I think the issue is that you are trying to import the whole package, like a "import org.newdawn.slick.*" in Java. In Clojure you cannot do this, but you have to import each class that you want to use.
The shortest that you can get is:
(:import (java.io BufferedReader Bits BufferedWriter))

Standalone clojure app

I'm a beginner with clojure, only starting it yesterday.
I have gathered that a simple way to create a standalone app is with leiningen lein new foo.
I tried to create a hello world test project with leiningen. I added :main and :aot directives to project.clj, added :gen-class to the core.clj file and tried lein run, but I get errors about class definition not found.
Exception in thread "main" java.lang.NoClassDefFoundError:
Caused by: java.lang.ClassNotFoundException:
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
The core.clj file
(ns test.core
(:gen-class))
(defn -main [& args] (println "Hello main"))
And the project.clj file
(defproject test "1.0.0-SNAPSHOT"
:description "FIXME: write description"
:main test.core
:aot [test.core]
:dependencies [[org.clojure/clojure "1.2.1"]])
Edit: After further testing, it seems like copying the project to my desktop works as is, which I think points to that the environment on my laptop is somehow borked, but I don't know how.
The environment on desktop is clojure from repositories and leiningen from AUR. On laptop the clojure is from clojure.org and leining is from github.
[UPDATE April 2013]
Leiningen 2, which has been officially released for some time, includes the concept of project templates. By default, Leiningen provides an app template that provides what you need out of the box. Try:
lein new app my-project
You will see that Leiningen creates the familiar project template, but also includes:
The default namespace of my-project.core as the :main entry in your project.clj file
The :gen-class form in the namespace declaration of my-project.core
A default -main function in the my-project.core namespace
For those who cannot yet use Leiningen 2, the lein-newnew plugin provides an equivalent experience under Leiningen 1.
[/UPDATE]
To build a project that, when run, prints "Hello World!", you'd do as follows (revision of your process above):
Setup
lein new my-project
cd my-project
lein deps
You should now have a basic structure in place and the Clojure jar in your lib folder.
Write a Function
Now edit src/my_project/core.clj with your editor of choice, adding the following below the (ns ...) form:
(defn -main []
(println "Hello World!"))
This function is inside your my-project.core namespace. To ensure this gets run as your main, let's add a gen-class parameter to your namespace definition at the top, so that it now looks like this at the top of core.clj:
(ns my-project.core
(:gen-class :main true))
So all together, your core.clj file looks like this:
(ns my-project.core
(:gen-class :main true))
(defn -main []
(println "Hello World!"))
Configure it as the Main Function
Once you've got src/my_project/core.clj edited as above, you need to tell Leiningen (the build tool) where the "main" function for your project lives. Here's an example defproject form that does this:
(defproject my-project "1.0.0-SNAPSHOT"
:description "My Project"
:dependencies [[org.clojure/clojure "1.2.1"]]
:main my-project.core)
Now the -main function inside my-project.core becomes the entry-point for your program.
Run It
You can now have two options for running this project:
Use lein run at the command-line while at the root of your my-project project
Create a standalone jar file by running lein uberjar. You can then run the resultant jar file by running java -jar my-project-1.0.0-SNAPSHOT-standalone.jar
Figured it out. I had the latest leiningen from git, which was borked somehow. I checked out the 1.6.1 tag and ran self-install, and now it works.
I missed it, You named your project test, you can't do that change the name to something else it will work.
You say above
lein new foo
what you mean is
lein new test