How to load/remove namespaces in clojure during tests - clojure

I'm working on a library that finds dependencies within the source tree during application startup and I'm trying to write an integration test to ensure it works. I've got fixture files in my test namespaces, and the test starts and succeeds just fine.
To be sure that the tests don't affect future runs, I added an "after" handler (in midje) that uses remove-ns to remove the test fixture namespaces.
On the next load, the tests fail because the namespaces are missing.
It seems as if remove-ns not only removes the namespace, it makes it impossible to use require to load it into the same running VM afterwards. I note that there is a "use with caution" note on remove-ns with no explanation.
I've verified that manually running require does not, indeed, seem to be able to re-load a namespace that has been removed:
user=>(test.util.fixtures.A/f)
{:item 1}
user=> (remove-ns 'test.util.fixtures.A)
#<Namespace test.util.fixtures.A>
user=> (test.util.fixtures.A/f)
ClassNotFoundException test.util.fixtures.A
user=> (require 'test.util.fixtures.A)
nil
user=> (test.util.fixtures.A/f)
ClassNotFoundException test.util.fixtures.A
Anyone understand why this is happening?

I traced through the source, and it ends up that require calls load-libs, which in turn calls load-lib, which in turn checks a global atom (the line is loaded (contains? #*loaded-libs* lib)).
Reading further, it seems that once something is loaded, you can specify the :reload option to the library loader. Now I remember seeing the :reload, so the solution was to put :reload in the require:
user=> (require 'test.util.fixtures.A :reload)
nil
user=> (test.util.fixtures.A/f)
{:item 1}

Related

clojure.tools/namespace refresh fails with "No namespace: foo"

I'm using tools.namespace to provide smart reloading of namespaces on the REPL. However, when calling refresh or refresh-all, it throws an error.
user=> (require '[clojure.tools.namespace.repl :as tn])
user=> (tn/refresh)
:reloading (ep31.common ep31.routes ep31.config ep31.application user ep31.common-test ep31.example-test)
:error-while-loading user
java.lang.Exception: No namespace: ep31.config, compiling:(user.clj:1:1)
And it seems to end up in this weird state where (require ep31.config) works without an error, but afterwards the namespace isn't actually defined.
I kind of figured this out, this seems to be a combination of circumstances
there were AOT compiled classes left in target/classes from doing lein uberjar previously
tools.namespace doesn't function correctly when loaded namespaces are AOT compiled
target/classes is by default on the classpath
So long story short, if you did a jar/uberjar build before, then remove target/ and things should start working again.
The question I haven't been able to solve yet is why target/classes is on the classpath to begin with. I'm suspecting it's being added by Leiningen, but haven't found yet where or why it's happening.
I learned this the hard way, documentation for :target-path says (https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L309-L313):
;; All generated files will be placed in :target-path. In order to avoid
;; cross-profile contamination (for instance, uberjar classes interfering
;; with development), it's recommended to include %s in in your custom
;; :target-path, which will splice in names of the currently active profiles.
:target-path "target/%s/"
I guess there has to be legacy reasons that :target-path "target/%s/" isn't the default.

Leiningen, repl, uberjar: Unable to resolve symbol, No such var

While my lein new app project runs merrily inside the Light Table, lein uberjar won't work. Curiously, it behaves exactly like a classic Pascal compiler: it can't resolve references ahead of definitions. Another curiosity: yesterday it worked. I am not aware of fooling with anything sensitive.
Google says that the subj symptoms are quite commonplace; I tried whatever helped other people in the same (?) plight, but to no avail. By the way, usually they blame it on software bugs: "get the latest version of Leiningen and Clojure". I've got 2.5.0 and 1.6.
The project (main file) is here: https://github.com/Tyrn/pcc/blob/master/src/pcc/core.clj
As it is, parsed-args can't be resolved inside build-album; if I move the -main function to the top of the file, 'No such var' happens to cli-options inside -main. No amount of fiddling with explicit use of namespaces makes any difference.
Again, inside the Light Table everything runs fine.
Using def inside of a function is not idiomatic, especially if there is no reason to have it as a global variable. Just pass it as a function parameter:
(let [parsed-args (parse-opts ...)]
...
(build-album parsed-args))
If you really need global state, you can use e.g. a promise (alternatively, an atom):
(defonce parsed-args (promise))
...
(deliver parsed-args (parse-opts ...))
However, Clojure files are read from top to bottom, and yes, functions not having access to bindings introduced later in the file is by design. You can use declare to tell the parser what to expect:
(declare ^:dynamic *parsed-args*)
(defn build-album ...)
(def ^:dynamic *parsed-args* ...)
...
(binding [*parsed-args* (parse-opts ...)]
(build-album))
TL;DR: If not necessary, avoid global state; if necessary, minimize it.

Is there a way to inject a method into clojure.core at lein repl startup?

the :injections keyword is really useful. However, I am hoping to dynamically install a couple of functions in core for debugging purposes. How can this be done?
:injections [(require 'spyscope.core)
(use '[cemerick.pomegranate :only (add-dependencies)])
(use '[clojure.tools.namespace.repl :only (refresh)])]
Ideally, I would want refresh to stay around to that I can use it everywhere
You could use intern for this purpose, although I suspect there might be a better way to have debugging functions available all the time. I've used intern with clojure.core the times that I wanted to mess around with existing functions to learn stuff, but injecting functions in other namespaces feels too hackish.
(intern 'clojure.core 'refresh (fn [] (println "refreshed!"))
(ns another-ns)
(refresh)
;=> refreshed!
And in your project.clj you can use the :repl-options key, specifically :init. This, though, depends on the workflow you have in mind, since the function will not be available in all the namespaces that already exist when the REPL fires up, because they all have already refered the public vars in clojure.core.
You could however, call (clojure.tools.namespace.repl/refresh) once, when the REPL starts, to get all namespaces reloaded and then the function should be available from then on. I just tried the following and it seems to work:
:repl-options {:init (do (require 'clojure.tools.namespace.repl)
(intern 'clojure.core 'refresh clojure.tools.namespace.repl/refresh)
(clojure.tools.namespace.repl/refresh))}

fn call doesnt execute on compile time

I have defined the following in one ns -
There is a services atom, and another function to add to that atom -
(ns ex.first)
(def services (atom []))
(defn add-service
[fns]
(swap! services conj fns))
In my code in another ns I do this -
(ns ex.second ..)
(add-service [fn1 fn2 fn3])
1) I am assuming that when I run my repl, which compiles the code, it should add the vector to the atom. However when I eval the #services it returns [].
2) The above works if I eval the (add-service [fn1 fn2 fn3]) in the repl.
3) I have also tried using converting the add-service fn to a macro. However still I find the #services to be empty.
So will appreciate if someone can help on these -
1) Why doesnt the add-service call add a vector to the atom on code compilation when -
add-service is defined as a fn.
add-service is defined as a macro.
2) How do I make it work :)
It depends on if you have set :aot to :all in project.clj or not.
If :aot is :all then the function call will execute as soon as repl is started, otherwise you will need to load the ex.second namespace (ex: using use). Loading will cause the ns to be compiled and the corresponding class to be loaded in jvm and the function call will get executed.
Also, the function call doesn't happen at the compile time, it happens when the compiled class (representing the namespace) is loaded by jvm.
UPDATED (based on comment):
If you make it a macro, then also you need to consider the aot thing.
If aot is set to compile the namespace then lein will create a jvm, load your code in it, call clojure compiler, which will read the code, execute the macro and compiles the code, at this time this jvm (which is used for compiling your code) will have services atom filled coz of macro execution, but this jvm was only for compilation. lein will then create another jvm for the run command and load the compiled class in that jvm but this jvm won't have the services filled in coz the classes doesn't have any code that fill it in.
If aot is not set then macro will work because the read,macroexpand,compile process will happen in the lein run jvm but only when you cause loading of the ns.
As far as "without loading the ns" is concern, what you can do is put the ex.second in the :aot of project.clj

How do I depend on every clojure contrib library?

I used to like to include all of clojure.contrib, and require all the libraries. This makes find-doc useful as a discovery tool.
Nowadays (clojure 1.4) clojure.contrib is split into many sub-libraries. And that rather spoils my scheme, and it also means that I am constantly having to restart the JVM every time I need a new library.
So I'm busy constructing a project.clj file with many lines:
[org.clojure/algo.generic "0.0.6"]
....
[org.clojure/data.xml "0.0.4"]
....
So that I can get leiningen to put every clojure contrib library on the classpath, whether I need them or not.
And I reckon that this is going to be a spectacular pain in the neck, what with the version numbers, and all.
And I wonder if anyone has a better way to do the same thing?
EDIT: Thinking about it, if there's a web page somewhere that has a list of library names and current versions, I can turn that into a project file fairly easily.
You could use pomegranate if you just want to run it in the REPL (which seems like it would be the only appropriate use case, right?). You can have it look up the latest versions using the Maven Central API. I think this is better than maintaining some sort of dependencies project, generated or otherwise.
(require '[cemerick.pomegranate :refer [add-dependencies]])
(add-dependencies
:coordinates '[[clj-http "0.5.8"]]
:repositories {"clojars" "http://clojars.org/repo"})
(require '[clj-http.client :as client])
;; contrib project names from https://github.com/clojure
(def contrib ["tools.nrepl" "tools.trace" "tools.namespace" "tools.macro"
"test.generative" "math.numeric-tower" "core.match" "core.logic"
"data.priority-map" "core.contracts" "tools.cli" "java.jmx"
"java.jdbc" "java.classpath" "data.xml" "data.json" "core.unify"
"core.incubator" "core.cache" "algo.monads" "data.generators"
"core.memoize" "math.combinatorics" "java.data" "tools.logging"
"data.zip" "data.csv" "algo.generic" "data.codec"
"data.finger-tree"])
(defn add-contrib-dependencies
"look up the latest version of every contrib project in maven central,
and add them as dependencies using pomegranate."
[project-names]
(add-dependencies
:coordinates
(map (juxt
(comp symbol (partial format "org.clojure/%s"))
(fn [proj]
(Thread/sleep 100)
(-> "http://search.maven.org/solrsearch/select?q=%s&rows=1&wt=json"
(format proj)
(client/get {:as :json})
:body :response :docs first :latestVersion)))
project-names)))
Now you can just invoke this function on the list of project names:
user=> (add-contrib-dependencies contrib)
{[org.clojure/data.zip "0.1.1"] nil,
[org.clojure/java.classpath "0.2.0"] nil,
[org.clojure/core.cache "0.6.2"] nil, ...}
UPDATE: as suggested earlier, I had made this answer into a library. It can be used either as nREPL middleware or invoked manually from a running REPL session. The code can be found at https://github.com/rplevy/contrib-repl, where usage instructions can also be found.
I feel your pain. It should be helpful http://dev.clojure.org/display/community/Where+Did+Clojure.Contrib+Go