Now, I need some kind of code reloading in the clojure/java mixing system.
I know tools.namespace lib can be used to refresh clj code. But when I try to reload code which is used in a thread, it is not updated.
This is the example clojure code I need to update: (just switch between upper case and lower case.)
(ns com.test.test-util)
(defn get-word [sentence]
(let [a-word (rand-nth (clojure.string/split sentence #"\s"))
;;a-word (clojure.string/upper-case a-word)
]
(println a-word)
a-word))
I used a future to start a thread and call the code above.
In main method, in my first test, I just create a future and refresh the code after a while. But it is not updated. So, I also tested to create another thread after refresh. But the code is not updated.
(defn start-job []
(future (
(println "start worker job.")
(while #job-cond
(do (get-word (rand-nth sentences))
(Thread/sleep 2000)) )
(println "return!")
)))
(defn -main []
(let [work-job (start-job)]
(println "modify clj file...")
(Thread/sleep 10000)
(swap! job-cond not)
(future-cancel work-job)
(swap! job-cond not)
(refresh)
(Thread/sleep 3000)
(let [new-work-job (start-job)]
(Thread/sleep 10000)
(swap! job-cond not)
(future-cancel new-work-job))))
Thanks.
In general you don't want code-reloading to affect runtime. Rather you should try to seek for clean ways to boot and shutdown the running application code using e. g. Stuart Sierras library component or juxt/jig by Malcolm Sparks. Then reload namespaces only when your application is not up and running (or at least not the parts of running code that depend on namespaces being reloaded).
That being said, here is why your example does not work:
start-job can not automatically refer to a reloaded version of get-word in its lifetime because the version of get-work it uses is resolved when it is compiled. So it will still use the old version.
Explicitly resolve the var of get-word using
(resolve `get-word)
and invoke the returned var as a function instead of using get-word directly in the future.
What will happen then?
After you made the changes and refresh has been invoked, -main will still refer to the old-version of start-job (so if you change that, it will not show effect in runtime). However within that get-word will be resolved as described above and the new version of get-word will be used.
Short explanation: You can't expect a running compiled function to replace itself with reloaded code on the fly (no matter whether it runs on a separate thread or not). That would (unless you are using resolve in it) be a necessity though for it to invoke reloaded functions.
Related
I am getting NullPointerExceptions in backtype.storm.utils.DisruptorQueue.consumeBatchToCursor method when running my topology, specifically in a bolt. Spouts are duly executed.
Storm's troubleshooting page says that it might be due to multiple threads issuing methods on the OutputCollector. However, i cannot see where does it relate to my case.
Here's the code for the spout:
(defspout stub-spout ["stub-spout"]
[conf context collector]
(spout
(nextTuple []
(let [channel-value (<!! storm-async-channel)]
(emit-spout! collector [channel-value])))
(ack [id]
))))
and for the bolt:
(defbolt stub-bolt ["stub-bolt"] [tuple collector]
(println "Invocation!")
(let [obj (get tuple "object")
do-some-calculations (resolve 'calclib/do-some-calculations)
new-obj (do-some-calculations obj)]
(emit-bolt! collector new-obj)))
After some investigation it turned out that the call to resolve returns null (i need to resolve during runtime as some calculation occurs in a macro located in calclib).
The code runs properly in local cluster though. Why is this happening?
Will be grateful for any suggestions.
Thanks!
I think i've found a solution. The bolt definition is changed to a prepared bolt:
(defbolt stub-bolt ["stub-bolt"]
{:prepare true}
[conf context collector]
(let [f (load "/calclib/core")
do-some-calculations (resolve 'calclib/do-some-calculations)]
(bolt
(execute [tuple]
(let [obj (get tuple "object")
new-obj (do-some-calculations obj)]
(emit-bolt! collector new-obj))))))
Key is the call to load. I wonder if there's a more elegant approach though.
Is it ok to use binding with core.async? I'm using ClojureScript so core.async is very different.
(def ^:dynamic token "no-token")
(defn call
[path body]
(http-post (str host path) (merge {:headers {"X-token" token}} body)))) ; returns a core.async channel
(defn user-settings
[req]
(call "/api/user/settings" req))
; elsewhere after I've logged in
(let [token (async/<! (api/login {:user "me" :pass "pass"}))]
(binding
[token token]
(user-settings {:all-settings true})))
In ClojureScript1, binding is basically with-redefs plus an extra check that the Vars involved are marked :dynamic. On the other hand, gos get scheduled for execution1 in chunks (that is, they may be "parked" and later resumed, and interleaving between go blocks is arbitrary). These models don't mesh very well at all.
In short, no, please use explicitly-passed arguments instead.
1 The details are different in Clojure, but the conclusion remains the same.
2 Using the fastest mechanism possible, setTimeout with a time of 0 if nothing better is available.
I have a closure in which a future takes a do block. Each function inside the do block is provided by the arguments of the closure:
(defn accept-order
[persist record track notify log]
(fn [sponsor order]
(let [datetime (to-timestamp (local-now))
order (merge order {:network_reviewed_at datetime
:workflow_state "unconfirmed"
:sponsor_id (:id sponsor)})]
(future
(do
(persist order
(select-keys order [:network_reviewed_at
:workflow_state
:sponsor_id]))
(record sponsor order true)
(track)
(notify sponsor order)
(log sponsor order)))
order)))
No function in the do block is fired. If I deref the future, it works. If I remove the future it works. If I run from a REPL, it works. But if I run lein test, it won't work.
Any ideas? Thank you!
Adding a (Thread/sleep 2000) to a test invoking your function causes the future to run, so I'd venture a guess that Leiningen is killing the VM before your future gets to run (or at least before it manages to cause its side effects). Leiningen does kill the VM immediately after running tests.
As a side note, you don't need the do. future takes a body, not a single expression.
Based on command-line input, I need to set some run-time constants which a number of downstream functions are going to use. The code in those functions may execute in other threads so I am not considering the "declare var and use binding macro" combination. What are the pros/cons of using a var (with alter-var-root) for this versus using an atom? That is,
(declare *dry-run*) ; one of my constants
(defn -main [& args]
; fetch command line option
;(cli args ...)
(alter-var-root #'*dry-run* (constantly ...))
(do-stuff-in-thread-pool))
versus
(def *dry-run* (atom true))
(defn -main [& args]
; fetch command line option
;(cli args ...)
(reset! *dry-run* ...)
(do-stuff-in-thread-pool))
If there is another option besides these two which I should consider, would love to know.
Also, ideally I would've preferred not to provide an initial val to the atom because I want to set defaults elsewhere (with the cli invocation), but I can live with it, especially if using the atom offers advantages compared to the alternative(s).
Write-once values are exactly the use case promises are designed for:
(def dry-run (promise))
(defn -main []
(deliver dry-run true))
(defn whatever [f]
(if #dry-run
...))
AFAIK alter-var-root only guarantees synchronized variable value changing and doesn't guarantee safe reading during this change. On other hand the atom really provides atomically change the state of identity.
If you don't want provide an initial value you can just set it to nil:
(def *dry-run* (atom nil))
What is wrong with just using a var and alter-var-root? You set up the new value in your startup function, before you really kick-off the workers. So there is no race in reading. And you can save the # everywhere you need the value.
I have a Clojure program that I build as a JAR file using Maven. Embedded in the JAR Manifest is a build-version number, including the build timestamp.
I can easily read this at runtime from the JAR Manifest using the following code:
(defn set-version
"Set the version variable to the build number."
[]
(def version
(-> (str "jar:" (-> my.ns.name (.getProtectionDomain)
(.getCodeSource)
(.getLocation))
"!/META-INF/MANIFEST.MF")
(URL.)
(.openStream)
(Manifest.)
(.. getMainAttributes)
(.getValue "Build-number"))))
but I've been told that it is bad karma to use def inside defn.
What is the Clojure-idiomatic way to set a constant at runtime? I obviously do not have the build-version information to embed in my code as a def, but I would like it set once (and for all) from the main function when the program starts. It should then be available as a def to the rest of the running code.
UPDATE: BTW, Clojure has to be one of the coolest languages I have come across in quite a while. Kudos to Rich Hickey!
I still think the cleanest way is to use alter-var-root in the main method of your application.
(declare version)
(defn -main
[& args]
(alter-var-root #'version (constantly (-> ...)))
(do-stuff))
It declares the Var at compile time, sets its root value at runtime once, doesn't require deref and is not bound to the main thread. You didn't respond to this suggestion in your previous question. Did you try this approach?
You could use dynamic binding.
(declare *version*)
(defn start-my-program []
(binding [*version* (read-version-from-file)]
(main))
Now main and every function it calls will see the value of *version*.
While kotarak's solution works very well, here is an alternative approach: turn your code into a memoized function that returns the version. Like so:
(def get-version
(memoize
(fn []
(-> (str "jar:" (-> my.ns.name (.getProtectionDomain)
(.getCodeSource)
(.getLocation))
"!/META-INF/MANIFEST.MF")
(URL.)
(.openStream)
(Manifest.)
(.. getMainAttributes)
(.getValue "Build-number")))))
I hope i dont miss something this time.
If version is a constant, it's going to be defined one time and is not going to be changed you can simple remove the defn and keep the (def version ... ) alone. I suppose you dont want this for some reason.
If you want to change global variables in a fn i think the more idiomatic way is to use some of concurrency constructions to store the data and access and change it in a secure way
For example:
(def *version* (atom ""))
(defn set-version! [] (swap! *version* ...))