Distributing a simple library via clojars - clojure

I have implemented a hyphenation algorithm (at namespace hyphenator-clj.core), defined it as org.clojars.nikonyrh.hyphenator-clj 0.1.0 at defproject and pushed it to Clojars. Uberjar seems to have files like core__init.class, core.clj and core.class.
However when I try to use it as a dependency on an other project I get this error:
$ lein uberjar
Retrieving org/clojars/nikonyrh/hyphenator-clj/org.clojars.nikonyrh.hyphenator-clj/0.1.0/org.clojars.nikonyrh.hyphenator-clj-0.1.0.pom from clojars
Retrieving org/clojars/nikonyrh/hyphenator-clj/org.clojars.nikonyrh.hyphenator-clj/0.1.0/org.clojars.nikonyrh.hyphenator-clj-0.1.0.jar from clojars
Compiling example.core
java.io.FileNotFoundException: Could not locate org/clojars/nikonyrh/hyphenator_clj__init.class or org/clojars/nikonyrh/hyphenator_clj.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name., compiling:(core.clj:1:1)
Exception in thread "main" java.io.FileNotFoundException: Could not locate org/clojars/nikonyrh/hyphenator_clj__init.class or org/clojars/nikonyrh/hyphenator_clj.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name., compiling:(core.clj:1:1)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3657)
at clojure.lang.Compiler.compile1(Compiler.java:7474)
at clojure.lang.Compiler.compile1(Compiler.java:7464)
at clojure.lang.Compiler.compile(Compiler.java:7541)
at clojure.lang.RT.compile(RT.java:406)
at clojure.lang.RT.load(RT.java:451)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__5677.invoke(core.clj:5893)
...
Must I change my project's folder structure so that it matches the expected org/clojars/nikonyrh/hyphenator_clj__init.class, or can I somehow override the current behavior? If there is a good tutorial about this out there I would be happy to read it.
Basically I would like to get this example project to work. project.clj:
(defproject example "0.0.1-SNAPSHOT"
:description ""
:dependencies [[org.clojure/clojure "1.8.0"]
[org.clojars.nikonyrh.hyphenator-clj "0.1.0"]]
:javac-options ["-target" "1.6" "-source" "1.6" "-Xlint:-options"]
:aot [example.core]
:main example.core)
src/example/core.clj:
(ns example.core
(:require [org.clojars.nikonyrh.hyphenator-clj :as h])
(:gen-class))
(defn -main [& argv] (doseq [arg argv] (println (h/hyphenate arg :hyphen \-))))
I'm suspecting I also have the english.txt in a wrong directory, as it isn't contained in the uberjar but resource files are an other topic.

You would probably be better off not using a hyphen in the namespace hyphenator-clj. Why not just use hyphenator? But if you do I suspect that the name of the directory should have an underscore in it rather than a hyphen, so be: hyphenator_clj.
If fixing that issue doesn't help then another thing to look out for, which I can't see from your question, is where exactly is core.clj in the directory structure, and does the project.clj reflect that? For instance is the path for the namespace hyphenator-clj.core in a src directory off the root of your project? The root of your project being defined as where the project.clj is located.
Something else that would be good to see in the question is whether you can get the program to work just locally, without having packed it up into an uberjar and shipped it to clojars. My guess would be that it does work locally, but it would help for that to be stated.
Okay taking a look at your links now. You might like to read a working project deployed to clojars, that has hypens in its name, for instance here. The first difference you might notice is the project name you use is rather long: org.clojars.nikonyrh.hyphenator-clj. It should just be hyphenator-clj. Also I would recommend having an identifier ending in "-SNAPSHOT" as that project does.
But taking a look at the bigger picture, a great idea you suggested in the comments is to test without Clojars being in the mix at all. To do this use lein install on the library you want to use from another lein project.

Ahaa at least I understand the process a bit better now, basically my require has to match the structure of the used JAR file, which may be very different from the project's name. For example cc.qbits/spandex is actually required as qbits.spandex.
The english.txt dependency was fixed by moving it to resources folder, deploying the new version to Clojars and importing the dependency as it exists in the JAR:
(ns example.core
(:require [hyphenator-clj.core :as h])
(:gen-class))
(defn -main [& argv] (doseq [arg argv] (println (h/hyphenate arg :hyphen \-))))

Related

Can I start Gorilla REPL from an uberjar?

I play with this great Gorilla REPL powered project ( https://bitbucket.org/probprog/anglican-examples/ to be specific), and want to use it under certain restricted circumstances.
Is there a way to produce an uberjar that can be started using only a JVM?
Well, I know how to create an uberjar for this project, but can I start a Gorilla REPL from it? If not what do I have to add and how do I start it?
EDITED Note on Juraj's answer:
I added a start file src/gorillaproxy/gorillaproxy.clj with the following content:
(ns gorillaproxy.gorillaproxy
(:use [gorilla-repl.core :only [run-gorilla-server]])
(:gen-class))
(defn -main
[& args]
(run-gorilla-server {:port 8990}))
Then I added [gorilla-repl "0.4.0"] to the dependency list (in project.clj), and the line
:main gorillaproxy.gorillaproxy
In that way the uberjar started the Gorilla REPL, and when I put the worksheets (and data, resources, .. if needed) into the same directory, everything worked fine.
Gorilla is typically run via the lein-gorilla plugin and thus isn't a part of an uberjar.
If you really want to create a bundle containing gorilla repl dependencies, then you need to add it this capability manually to your project.
The question is why would you want to do that.
Do you want to distribute these samples to somebody else? If that's the case, you'll still need to have all those worksheets in the current directory from where your uberjar is run because that's how gorilla repl discovers worksheets.
You can look at lein-gorilla source code to see how gorilla repl can be started.
I'd then at the same code to your project (create new src/core.clj file or whatever) and configure it in your project.clj as :main.
You'll also need to add gorilla-repl as a dependency to your project.clj
Notice however, that you'll need to run that uberjar from a directory where your anglican worksheets are (or a parent directory of such a directory).

clojure classpath(?) troubles loading code and dependencies with leiningen

I'm a Clojure and JVM beginner, and the build tools are a bit confusing to me, so forgive the stupidity I'm about to carry out. I don't actually know what a "classpath" is... I just want a Lisp with lots of libraries!
But I'm trying to do a write a simple little bit of text-mining code in Clojure, and I don't seem to be able to get things working.
There are two problems, and I think they're related, both probably to this classpath thing from javaland that none of the introductory Clojure books seem to explain.
I started the project with lein new and my project.clj has the following dependencies:
:dependencies [[org.clojure/clojure "1.6.0"]
[clj-fuzzy "0.1.8"]
[snowball-stemmer "0.1.0"]
[net.mikera/core.matrix "0.50.0"]])
and the relevant ns call from my core.clj is as follows:
(ns blabber.core
(:require
[clojure.string :refer [lower-case split]]
[clojure.walk :refer [keywordize-keys]]))
My directory structure is the default setup leiningen uses for a new library.
First problem:
I gather from this prior SO that if things are set up right, I shouldn't need to call (load-file "src/blabber/core.clj") in the repl before calling (use 'blabber.core) in order to get at code I just wrote. (I'm firing up the lein repl from the src/blabber directory in bash). But when I fire up the repl, use fails unless I call load-file first. So does that mean my classpath is set up wrong somehow? And how does one fix such a thing?
Second problem:
Ok, so now I'm trying to actually use core.matrix. First line of my core.clj changed to:
(ns blabber.core
(:require
[clojure.string :refer [lower-case split]]
[clojure.walk :refer [keywordize-keys]]
[clojure.core.matrix :refer [dataset]]))
When I first started the repl with this project.clj but without the :require to core.matrix, leiningen happily fetched core.matrix: Retrieving net/mikera/core.matrix/0.50.0/core.matrix-0.50.0.jar from clojars
However, after including the :require to core.matrix in the actual code and starting up the repl again, at the repl I call (load-file "src/blabber/core.clj"), and get the following error:
CompilerException java.io.FileNotFoundException: Could not locate clojure/core/matrix__init.class or clojure/core/matrix.clj on classpath: , compiling:(/Users/pauliglot/github/blabber/src/blabber/core.clj:1:1)
I'm using Leiningen 2.5.1 on Java 1.8.0_45. And I'm just following the getting started guide for core.matrix.
So what happened? Why can't the JVM find any of my stuff? Save me? Thanks!
If it helps, here's a dump of the output of lein classpath
/Users/pauliglot/github/blabber/test:/Users/pauliglot/github/blabber/src:/Users/pauliglot/github/blabber/dev-resources:/Users/pauliglot/github/blabber/resources:/Users/pauliglot/github/blabber/target/classes:/Users/pauliglot/.m2/repository/clj-fuzzy/clj-fuzzy/0.1.8/clj-fuzzy-0.1.8.jar:/Users/pauliglot/.m2/repository/snowball-stemmer/snowball-stemmer/0.1.0/snowball-stemmer-0.1.0.jar:/Users/pauliglot/.m2/repository/org/clojure/tools.macro/0.1.5/tools.macro-0.1.5.jar:/Users/pauliglot/.m2/repository/clojure-complete/clojure-complete/0.2.3/clojure-complete-0.2.3.jar:/Users/pauliglot/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20140226-71326067/google-closure-library-third-party-0.0-20140226-71326067.jar:/Users/pauliglot/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar:/Users/pauliglot/.m2/repository/org/clojure/google-closure-library/0.0-20140226-71326067/google-closure-library-0.0-20140226-71326067.jar:/Users/pauliglot/.m2/repository/org/mozilla/rhino/1.7R4/rhino-1.7R4.jar:/Users/pauliglot/.m2/repository/org/json/json/20090211/json-20090211.jar:/Users/pauliglot/.m2/repository/net/mikera/core.matrix/0.50.0/core.matrix-0.50.0.jar:/Users/pauliglot/.m2/repository/com/google/javascript/closure-compiler/v20131014/closure-compiler-v20131014.jar:/Users/pauliglot/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/pauliglot/.m2/repository/org/clojure/clojurescript/0.0-2202/clojurescript-0.0-2202.jar:/Users/pauliglot/.m2/repository/args4j/args4j/2.0.16/args4j-2.0.16.jar:/Users/pauliglot/.m2/repository/org/clojure/tools.nrepl/0.2.6/tools.nrepl-0.2.6.jar:/Users/pauliglot/.m2/repository/com/google/guava/guava/15.0/guava-15.0.jar:/Users/pauliglot/.m2/repository/org/clojure/clojure/1.6.0/clojure-1.6.0.jar:/Users/pauliglot/.m2/repository/org/clojure/tools.reader/0.8.3/tools.reader-0.8.3.jar:/Users/pauliglot/.m2/repository/org/clojure/data.json/0.2.3/data.json-0.2.3.jar
Edit
Here's the directory tree:
| blabber
|-doc
|-resources
|-src
|---blabber
|-target
|---classes
|-----META-INF
|-------maven
|---------blabber
|-----------blabber
|---stale
|-test
and all my code is in blabber/src/blabber/core.clj
A classpath is a Java concept; Leiningen should set up the classpath correctly. What I suspect is that there's a mismatch between the directories, filenames, or namespace declarations. The file path should be <name passed to 'lein new'>/src/blabber/core.clj. It wasn't clear from what you wrote whether that's the case. Try running lein deps first, too, but you shouldn't need to do that. This and lein repl are usually run from the directory, i.e. the project root directory. You might need :source-paths or :main in project.clj, but I don't think so.
(use 'clojure.core.matrix) should work. Works for me, e.g. with
(defproject blabber "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.8.0"]
[net.mikera/core.matrix "0.50.0"]]).

access library functions in leiningen REPL

For the development of a library I started from a lein project, invoked like so:
lein new mylib
if I call lein install now, I can access my library in other projects. But trying to immidiately test the functions I wrote failed:
lein repl
...
(dir mylib.core)
Exception No namespace: mylib.core found clojure.core/the-ns (core.clj:4008)
Do I have to add something to the project.clj file maybe?
In order to use a library you must cause the code to be loaded - that it be on the classpath is not sufficient.
You can do this easily in an ns declaration in a file of course, but in the repl it can be easier to use (require '[my-lib.whatever :as w]) after which one can call (w/foo) (w/bar) etc. as expected. You can also use (in-ns 'my-lib.whatever) in order to switch to the namespace, but this will not give you a good result unless you have previously used require or use or load-file etc. to get the definitions first.
Let's say you created a new library named clj-foo.
% lein new clj-foo
Start your repl.
% cd clj-foo
% lein repl
In the repl, load the main entry point to your library and switch to its namespace.
(load-file "src/clj_foo/core.clj")
(ns clj-foo.core)
Now you're in the clj-foo.core namespace, make sure to add back in the repl ns to get things like doc available.
(use 'clojure.repl)
That's it. You're all set to start calling functions in your library. Note that other library files will be available from the clj-foo.core namespace if they were loaded by namespace declaration at the top of clj_foo/core.clj. If not, then you'll need to invoke load-file with their path as well.
If you make changes in core.clj. You can invoke load-file again to pick up the new code. As you progress, you can use cider to facilitate loading of individual functions and files. But that's for another question. :)
You need to add a dependency to use your library from another project. To do this add a vector (a tuple-2) to the vector that is the value of the :dependencies key in the project.clj file. Here's an example:
:dependencies [[org.clojure/clojure "1.7.0"]
[org.clojure/clojurescript "1.7.170"]
[org.clojure/core.async "0.2.371"]
[default-db-format "0.1.0-SNAPSHOT"]
[com.andrewmcveigh/cljs-time "0.3.14"]]
My own local library is called default-db-format. Its really no different to adding a dependency for com.andrewmcveigh/cljs-time.
As you say you can already do this, but are having trouble getting a REPL connection to the project of the library itself. When you go (in-ns 'some-path), you need the single quote in front of some-path. Note that some-path is a different thing to the name of your library.
Rather than use lein repl you can use the figwheel repl - if your project is setup with figwheel. My library has only one entry point and that is lein figwheel devcards. After that I had no problem going to a namespace and trying out a function:
cljs.user=> (in-ns 'default-db-format.core)
nil
default-db-format.core=> (check 1 2)
As noisesmith mentioned having a REPL in your IDE is the best setup. No fiddly typing just bring up pre-configured REPLs (per namespace) with the click of a button (or keystroke). Figwheel/Cursive setup instructions here.
I was also facing the same issue with the following configuration:
Leiningen 2.9.0 on Java 1.8.0_201 Java HotSpot(TM) 64-Bit Server VM
My file looks like this, and from the repl I desired to invoke the foo function
(ns cljtest.test
(:gen-class))
(defn foo [input]
(assoc {} "a" 123))
Both these approaches worked fine for me on the repl.
1)Switch to the appropriate name space:
cljtest.core=> (in-ns 'cljtest.test)
#object[clojure.lang.Namespace 0x90175dd "cljtest.test"]
cljtest.test=> (foo nil)
{"a" 123}
cljtest.test=>
2)Require the appropriate name space:
cljtest.core=> (require '[cljtest.test :as test])
nil
cljtest.core=> (test/foo nil)
{"a" 123}
cljtest.core=>

exclude certain clj namespaces from compilation in leiningen

I have a project that works fine using lein run. Now I want to compile it into a standalone jar using lein uberjar. However, there are a couple of source files in my src/projectname/ directory called e.g. playground.clj and stats.clj that I use for experimenting with emacs & the repl, but that I don't want to compile for the final project.
With something like make, I would specify all files that should be compiled. With clojure/leiningen, it seems, all files are compiled by default - how can I exclude files? I haven't found anything in the leiningen docs.
I am currently using :aot :all. Is this the place to change something? Again, I couldn't find detailed documentation on this.
UPDATE:
The suggestions so far haven't worked. What has worked, however, is to include all desired namespaces instead of excluding the ones that should not be compiled. E.g.:
(defproject myproject "version"
;; ...
:profiles {:uberjar {:aot [myproject.data
myproject.db
myproject.util]}})
Have a look at leiningen's sample project.clj, which describes how to use :jar-exclusions or :uberjar-exclusions to exclude arbitrary paths when creating jars (resp. uberjars).
;; Files with names matching any of these patterns will be excluded from jars.
:jar-exclusions [#"(?:^|/).svn/"]
;; Files with names matching any of these patterns will included in the jar
;; even if they'd be skipped otherwise.
:jar-inclusions [#"^\.ebextensions"]
;; Same as :jar-exclusions, but for uberjars.
:uberjar-exclusions [#"META-INF/DUMMY.SF"]
Old question, but I think I found the answer for those coming after me.
I found the answer in the link to the sample leiningen project from #amalloy's answer, except instead of :jar-exclusions I use source-paths, here.
The idea is to create two separate source directories, one for stuff you don't care to spread around and one for stuff you do:
dev-src/<your-project>/playground.clj
dev-src/<your-project>/stats.clj
src/<your-project>/<everything-else>
Then, in your project.clj, include src in source-paths normally, and include emacs-src in a special profile where your want it visible, say the usual :dev profile:
{
;; ...
:source-paths ["src"]
:profiles {
:dev {
:source-paths ["src" "dev-src"]
}
}
}
That way when you're messing around on your machine those files will be in the jar, and when you deploy to clojars or compile with uberjar they will not be included in the jar, nor compiled.
Try this (ns ^:skip-aot my-ns)
You can also do
(ns ^{:skip-aot true} my-ns
(require [...]))
Source

lein - how to use a downloaded library

Let's say I find a cool clojure library like https://github.com/clojurewerkz/buffy
Now I want to use it. And it only lives on github.
How do I do this? I would love a full start to finish hello world example.
I've read about compiling it as a jar and using that, or using :dependencies in my project.clj but so far no examples have been complete, and I'm new.
For example in python I'd git clone the repo into the root of my working tree and any file could just say import buffy
I just learned this five minutes ago. I wanted to use the clojure-csv library, here's what I did using Leiningen
Make a new leiningen project
lein new app csvtest
Now I have a folder called csvtest/. In the csvtest/project.clj file, edit the dependencies section to add the clojure-csv github path and a version. I'm not quite sure how the versioning works to be honest:
:dependencies [[org.clojure/clojure "1.5.1"]
[clojure-csv/clojure-csv "2.0.1"]]
Now run lein deps to automagically download any unresolved dependencies for your project
$ lein deps
...
Retrieving clojure-csv/clojure-csv/2.0.1/clojure-csv-2.0.1.pom from clojars
...
Now edit csvtest/src/csvtest/core.clj. Make it look like this:
Edit: (Based on sveri's comment I changed :use closjure-csv.core to the :require line)
(ns csvtest.core
(:gen-class)
(:require [clojure-csv.core :refer [parse-csv]]))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println (parse-csv "1,2,3,hello,world")))
You have to add the (:require [clojure-csv.core :refer [parse-csv]]) line to your ns, and in main it calls parse-csv, one of the functions provided by the library. From reading, it looks like :use is depreciated in favor of :require.
Note: For anyone coming from python (like me), it looks like doing :use clojure-csv.core is akin to from closure_csv import *, which is bad practice in python as well. And (:require [clojure-csv.core :refer [parse-csv]]) is like from clojure_csv import parse_csv
Run the project with lein run. My output was
$ lein run
([1 2 3 hello world])
I hope that helps!
You can download/use it from Clojars.
Clojars link:
https://clojars.org/clojurewerkz/buffy
To use it with Leiningen, add this to your dependencies(on project.clj file):
[clojurewerkz/buffy "1.0.0-beta4"]
After that, you can run lein deps from your project root folder in order to download dependencies.
To use dependencies directly from Github repos, you can check out this: https://github.com/tobyhede/lein-git-deps
Using lein (which is Clojure's build tool) you have a project.clj file. if you do lein new project-name you get a new project named "project-name", with a project.clj file in it. look at that file, you will see a :dependencies entry in it as in this example:
:dependencies [[org.clojure/clojure "1.5.1"]]
all you must do to include buffy, is look at their github site, and find out which version to use, then you add it to the :dependecies list:
:dependencies [[org.clojure/clojure "1.5.1"]
[clojurewerkz/buffy "1.0.0-beta3"]]
now you can simply do lein deps which will download the dependencies and once you written your code lein run to actually run it.
read a little about lein and check out its sample project.clj file to get a better understanding of how you should use clojure.