I have a Ring handler which uses several functions to build the response. If any of these functions throw an exception, it should be caught so a custom response body can be written before returning a 500.
I'm writing the unit test for the handler, and I want to ensure that an exception thrown by any of these functions will be handled as described above. My instinct was to use with-redefs inside a doseq:
(doseq [f [ns1/fn1 ns1/fn2 ns2/fn1]]
(with-redefs [f (fn [& args] (throw (RuntimeException. "fail!"))]
(let [resp (app (request :get "/foo")))))]
(is (= (:status resp) 500))
(is (= (:body resp) "Something went wrong")))))
Of course, given that with-redefs wants to change the root bindings of vars, it is treating f as a symbol. Is there any way to get it to rebind the var referred to by f? I suspect that I'm going to need a macro to accomplish this, but I'm hoping that someone can come up with a clever solution.
with-redefs is just sugar over repeated calls to alter-var-root, so you can just write the desugared form yourself, something like:
(doseq [v [#'ns1/fn1 #'ns1/fn2 #'ns2/fn1]]
(let [old #v]
(try
(alter-var-root v (constantly (fn [& args]
(throw (RuntimeException. "fail!")))))
(let [resp (app (request :get "/foo")))
(is (= (:status resp) 500))
(is (= (:body resp) "Something went wrong")))
(finally (alter-var-root v (constantly old))))))
Following on from amalloy's great answer, here's a utility function that I wrote to use in my tests:
(defn with-mocks
"Iterates through a list of functions to-mock, rebinding each to mock-fn, and calls
test-fn with the optional test-args.
Example:
(defn foobar [a b]
(try (+ (foo a) (bar b))
(catch Exception _ 1)))
(deftest test-foobar
(testing \"Exceptions are handled\"
(with-mocks
[#'foo #'bar]
(fn [& _] (throw (RuntimeException. \"\")))
(fn [a b] (is (= 1 (foobar a b)))) 1 2)))"
[to-mock mock-fn test-fn & test-args]
(doseq [f to-mock]
(let [real-fn #f]
(try
(alter-var-root f (constantly mock-fn))
(apply test-fn test-args)
(finally
(alter-var-root f (constantly real-fn)))))))
Related
I've been struggling to create a macro that'll allow me to dynamically bind whatever is in the &env into a binding form and then delegate to a pry like function to open a REPL that can see those bound &env symbols.
My simplistic pry func, which works as expected
(defn pry []
(print (str "pry(" *ns* ")> "))
(flush)
(let [expr (read)]
(when-not (= expr :exit)
(println (eval expr))
(recur))))
Using the pry func:
clojure-noob.core=> (def a 1)
#'clojure-noob.core/a
clojure-noob.core=> (pry)
pry(clojure-noob.core)> (+ a 1)
2
pry(clojure-noob.core)> :exit
nil
clojure-noob.core=>
My attempt at creating a dynamic invocation of binding:
(defmacro binding-pry []
(let [ks (keys &env)]
`(let [ks# '~ks
vs# [~#ks]
bs# (vec (interleave ks# vs#))]
(binding bs# (pry)))))
However, this fails because the inner symbol bs# is not expanded to an actual vector but instead is the generated symbol and binding tosses a clojure.core/binding requires a vector for its binding exception.
clojure-noob.core=> (let [a 1 b 2] (binding-pry))
Syntax error macroexpanding clojure.core/binding at (/tmp/form-init14332359378145135257.clj:1:16).
clojure.core/binding requires a vector for its binding in clojure-noob.core:
clojure-noob.core=>
The code quoted form with a debug print, the bs# symbol is resolved when printing but I don't know how to make it resolve to a vector when constructing the binding form.
(defmacro binding-pry []
(let [ks (keys &env)]
`(let [ks# '~ks
vs# [~#ks]
bs# (vec (interleave ks# vs#))]
(println bs#)
`(binding bs# (pry)))))
clojure-noob.core=> (let [a 1 b 2] (binding-pry))
[a 1 b 2]
(clojure.core/binding clojure-noob.core/bs__2464__auto__ (clojure-noob.core/pry))
clojure-noob.core=>
I'm very confident I'm tackling this incorrectly but I don't see another approach.
The Joy of Clojure demonstrates a break macro that does this already. I can't reproduce its source here, because it's EPL and not CC. But you can see its source at https://github.com/joyofclojure/book-source/blob/b76ef15/first-edition/src/joy/breakpoint.clj. It refers to a contextual-eval function as well: https://github.com/joyofclojure/book-source/blob/b76ef15/first-edition/src/joy/macros.clj#L4-L7.
A first step towards improving your attempt could be writing:
(defmacro binding-pry []
(let [ks (keys &env)]
`(binding [~#(interleave ks ks)] (pry))))
This still doesn't work because binding expects that the symbols can be resolved to existing dynamic vars. To tackle this problem you could make binding-pry introduce such vars as shown below:
(defmacro binding-pry []
(let [ks (keys &env)]
`(do
~#(map (fn [k] `(def ~(with-meta k {:dynamic true}))) ks)
(binding [~#(interleave ks ks)] (pry)))))
But this can have undesirable side-effects, like polluting the namespace with new var-names or making existing vars dynamic. So I would prefer an approach like the one mentioned in amalloy's answer but with a better implementation of eval-in-context (see my comment there).
To write a self-contained answer based on your pry function, let's first define eval-in which evaluates a form in an environment:
(defn eval-in [env form]
(apply
(eval `(fn* [~#(keys env)] ~form))
(vals env)))
Then let's modify pry to take an environment as an argument and use eval-in instead of eval:
(defn pry [env]
(let [prompt (str "pry(" *ns* ")> ")]
(loop []
(print prompt)
(flush)
(let [expr (read)]
(when-not (= expr :exit)
(println (eval-in env expr))
(recur))))))
An equivalent, less primitive version could be:
(defn pry [env]
(->> (repeatedly (let [prompt (str "pry(" *ns* ")> ")]
#(do (print prompt) (flush) (read))))
(take-while (partial not= :exit))
(run! (comp println (partial eval-in env)))))
Now we can define binding-pry as follows:
(defmacro binding-pry []
`(pry ~(into {}
(map (juxt (partial list 'quote) identity))
(keys &env))))
Finally, here is a direct/"spaghetti" implementation of binding-pry:
(defmacro binding-pry []
(let [ks (keys &env)]
`(->> (repeatedly (let* [prompt# (str "pry(" *ns* ")> ")]
#(do (print prompt#) (flush) (read))))
(take-while (partial not= :exit))
(run! (comp println
#((eval `(fn* [~~#(map (partial list 'quote) ks)] ~%))
~#ks))))))
Does Clojure have an equivalent of Java's try-with-resources construct?
If not, what is the normal way to handle this idiom in Clojure code?
The pre-Java-7 idiom for safely opening and closing resources is verbose enough that they actually added support for try-with-resources to the language. It seems strange to me that I can't find a macro for this use case in the standard Clojure library.
An example to a mainstream Clojure-based project repository—showing how this issue is handled in practice—would be very helpful.
You can use with-open to bind a resource to a symbol and make sure the resource is closed once the control flow left the block.
The following example is from clojuredocs.
(with-open [r (clojure.java.io/input-stream "myfile.txt")]
(loop [c (.read r)]
(when (not= c -1)
(print (char c))
(recur (.read r)))))
This will be expanded to the following:
(let [r (clojure.java.io/input-stream "myfile.txt")]
(try
(loop [c (.read r)]
(when (not= c -1)
(print (char c))
(recur (.read r))))
(finally (.close r))))
You can see that a let block is created with a try-finally to the call .close() method.
you can do something more close to java, making up some macro on top of with-open. It could look like this:
(defmacro with-open+ [[var-name resource & clauses] & body]
(if (seq clauses)
`(try (with-open [~var-name ~resource] ~#body)
~#clauses)
`(with-open [~var-name ~resource] ~#body)))
so you can pass additional clauses alongside the binding.
(with-open+ [x 111]
(println "body"))
expands to simple with-open:
(let*
[x 111]
(try (do (println "body")) (finally (. x clojure.core/close))))
while additional clauses lead to wrapping the it in try-catch:
(with-open+ [x 111
(catch RuntimeException ex (println ex))
(finally (println "finally!"))]
(println "body"))
expands to
(try
(let*
[x 111]
(try (do (println "body")) (finally (. x clojure.core/close))))
(catch RuntimeException ex (println ex))
(finally (println "finally!")))
But still it is quite an opinionated solution.
I'm writing a Clojure wrapper for an object-oriented API that heavily involves resource handling. For instance, for the Foo object, I've written three basic functions: foo?, which returns true iff something is a Foo; create-foo, which attempts to obtain the resources to create a Foo, then returns a map containing a return code and (if the construction succeeded) the newly created Foo; and destroy-foo, which takes a Foo and releases its resources. Here are some stubs for those three functions:
(def foo? (comp boolean #{:placeholder}))
(defn create-foo []
(let [result (rand-nth [::success ::bar-too-full ::baz-not-available])]
(merge {::result result}
(when (= ::success result)
{::foo :placeholder}))))
(defn destroy-foo [foo] {:pre [(foo? foo)]} nil)
Obviously, every time create-foo is called and succeeds, destroy-foo must be called with the returned Foo. Here's a simple example that doesn't use any custom macros:
(let [{:keys [::result ::foo]} (create-foo)]
(if (= ::success result)
(try
(println "Got a Foo:")
(prn foo)
(finally
(destroy-foo foo)))
(do
(println "Got an error:")
(prn result))))
There's a lot of boilerplate here: the try-finally-destroy-foo construct must be present to ensure that all Foo resources are released, and the (= ::success result) test must be present to ensure that nothing gets run assuming a Foo when there is no Foo.
Some of that boilerplate can be eliminated by a with-foo macro, similar to the with-open macro in clojure.core:
(defmacro with-foo [bindings & body]
{:pre [(vector? bindings)
(= 2 (count bindings))
(symbol? (bindings 0))]}
`(let ~bindings
(try
~#body
(finally
(destroy-foo ~(bindings 0))))))
While this does help somewhat, it doesn't do anything about the (= ::success result) boilerplate, and now two separate binding forms are required to achieve the desired result:
(let [{:keys [::result] :as m} (create-foo)]
(if (= ::success result)
(with-foo [foo (::foo m)]
(println "Got a Foo:")
(prn foo))
(do
(println "Got an error:")
(prn result))))
I simply can't figure out a good way to handle this. I mean, I could complect the behaviors of if-let and with-foo into some sort of if-with-foo macro:
(defmacro if-with-foo [bindings then else]
{:pre [(vector? bindings)
(= 2 (count bindings))]}
`(let [{result# ::result foo# ::foo :as m#} ~(bindings 1)
~(bindings 0) m#]
(if (= ::success result#)
(try
~then
(finally
(destroy-foo foo#)))
~else)))
This does eliminate even more boilerplate:
(if-with-foo [{:keys [::result ::foo]} (create-foo)]
(do
(println "Got a Foo:")
(prn foo))
(do
(println "Got a result:")
(prn result)))
However, I don't like this if-with-foo macro for several reasons:
it's very tightly coupled to the specific structure of the map returned by create-foo
unlike if-let, it causes all bindings to be in scope in both branches
its ugly name reflects its ugly complexity
Are these macros the best I can do here? Or is there a more elegant way to handle resource handling with possible resource obtainment failure? Perhaps this is a job for monads; I don't have enough experience with monads to know whether they would be useful tool here.
I'd add an error-handler to with-foo. This way the macro has a focus on what should be done. However, this simplifies the code only when all error-cases are treated by a handful of error handlers. If you have to define a custom error-handler every time you call with-foo this solution makes readability worse than an if-else construct.
I added copy-to-map. copy-to-map should copy all relevant information from the object to a map. This way the user of the macro doesn't by accident return the foo-object, since it gets destroyed inside the macro
(defn foo? [foo]
(= ::success (:result foo)))
(defn create-foo [param-one param-two]
(rand-nth (map #(merge {:obj :foo-obj :result %} {:params [param-one param-two]})
[::success ::bar-too-full ::baz-not-available])))
(defn destroy-foo [foo]
nil)
(defn err-handler [foo]
[:error foo])
(defn copy-to-map [foo]
;; pseudo code here
(into {} foo))
(defmacro with-foo [[f-sym foo-params & {:keys [on-error]}] & body]
`(let [foo# (apply ~create-foo [~#foo-params])
~f-sym (copy-to-map foo#)]
(if (foo? foo#)
(try ~#body
(finally (destroy-foo foo#)))
(when ~on-error
(apply ~on-error [~f-sym])))))
Now you call it
(with-foo [f [:param-one :param-two] :on-error err-handler]
[:success (str "i made it: " f)])
Building from #murphy's excellent idea to put the error handler into with-foo's bindings to keep the focus on the normal case, I've ended up with a solution that I like quite a lot:
(defmacro with-foo [bindings & body]
{:pre [(vector? bindings)
(even? (count bindings))]}
(if-let [[sym init temp error] (not-empty bindings)]
(let [error? (= :error temp)]
`(let [{result# ::result foo# ::foo :as m#} ~init]
(if (contains? m# ::foo)
(try
(let [~sym foo#]
(with-foo ~(subvec bindings (if error? 4 2))
~#body))
(finally
(destroy-foo foo#)))
(let [f# ~(if error? error `(constantly nil))]
(f# result#)))))
`(do
~#body)))
like my if-with-foo macro in the question, this with-foo macro is still tied to the structure returned by create-foo; unlike my if-with-foo macro and #murphy's with-foo macro, it eliminates the need for the user to manually take apart that structure
all names are properly scoped; the user's sym is only bound in the main body, not in the :error handler, and conversely, the ::result is only bound in the :error handler, not in the main body
like #murphy's solution, this macro has a nice, fitting name, instead of something ugly like if-with-foo
unlike #murphy's with-foo macro, this with-foo macro allows the user to provide any init value, rather than forcing a call to create-foo, and doesn't transform the returned value
The most basic use case simply binds a symbol to a Foo returned by create-foo in some body, returning nil if the construction fails:
(with-foo [foo (create-foo)]
["Got a Foo!" foo])
To handle the exceptional case, an :error handler can be added to the binding:
(with-foo [foo (create-foo)
:error (partial vector "Got an error!")]
["Got a Foo!" foo])
Any number of Foo bindings can be used:
(with-foo [foo1 (create-foo)
foo2 (create-foo)]
["Got some Foos!" foo1 foo2])
Each binding can have its own :error handler; any missing error handlers are replaced with (constantly nil):
(with-foo [foo1 (create-foo)
:error (partial vector "Got an error!")
foo2 (create-foo)]
["Got some Foos!" foo1 foo2])
I'd like to use memoize for a function that uses core.async and <! e.g
(defn foo [x]
(go
(<! (timeout 2000))
(* 2 x)))
(In the real-life, it could be useful in order to cache the results of server calls)
I was able to achieve that by writing a core.async version of memoize (almost the same code as memoize):
(defn memoize-async [f]
(let [mem (atom {})]
(fn [& args]
(go
(if-let [e (find #mem args)]
(val e)
(let [ret (<! (apply f args))]; this line differs from memoize [ret (apply f args)]
(swap! mem assoc args ret)
ret))))))
Example of usage:
(def foo-memo (memoize-async foo))
(go (println (<! (foo-memo 3)))); delay because of (<! (timeout 2000))
(go (println (<! (foo-memo 3)))); subsequent calls are memoized => no delay
I am wondering if there are simpler ways to achieve the same result.
**Remark: I need a solution that works with <!. For <!!, see this question: How to memoize a function that uses core.async and blocking channel read? **
You can use the built in memoize function for this. Start by defining a method that reads from a channel and returns the value:
(defn wait-for [ch]
(<!! ch))
Note that we'll use <!! and not <! because we want this function block until there is data on the channel in all cases. <! only exhibits this behavior when used in a form inside of a go block.
You can then construct your memoized function by composing this function with foo, like such:
(def foo-memo (memoize (comp wait-for foo)))
foo returns a channel, so wait-for will block until that channel has a value (i.e. until the operation inside foo finished).
foo-memo can be used similar to your example above, except you do not need the call to <! because wait-for will block for you:
(go (println (foo-memo 3))
You can also call this outside of a go block, and it will behave like you expect (i.e. block the calling thread until foo returns).
This was a little trickier than I expected. Your solution isn't correct, because when you call your memoized function again with the same arguments, sooner than the first run finishes running its go block, you will trigger it again and get a miss. This is often the case when you process lists with core.async.
The one below uses core.async's pub/sub to solve this (tested in CLJS only):
(def lookup-sentinel #?(:clj ::not-found :cljs (js-obj))
(def pending-sentinel #?(:clj ::pending :cljs (js-obj))
(defn memoize-async
[f]
(let [>in (chan)
pending (pub >in :args)
mem (atom {})]
(letfn
[(memoized [& args]
(go
(let [v (get #mem args lookup-sentinel)]
(condp identical? v
lookup-sentinel
(do
(swap! mem assoc args pending-sentinel)
(go
(let [ret (<! (apply f args))]
(swap! mem assoc args ret)
(put! >in {:args args :ret ret})))
(<! (apply memoized args)))
pending-sentinel
(let [<out (chan 1)]
(sub pending args <out)
(:ret (<! <out)))
v))))]
memoized)))
NOTE: it probably leaks memory, subscriptions and <out channels are not closed
I have used this function in one of my projects to cache HTTP calls. The function caches results for a given amount of time and uses a barrier to prevent executing the function multiple times when the cache is "cold" (due to the context switch inside the go block).
(defn memoize-af-until
[af ms clock]
(let [barrier (async/chan 1)
last-return (volatile! nil)
last-return-ms (volatile! nil)]
(fn [& args]
(async/go
(>! barrier :token)
(let [now-ms (.now clock)]
(when (or (not #last-return-ms) (< #last-return-ms (- now-ms ms)))
(vreset! last-return (<! (apply af args)))
(vreset! last-return-ms now-ms))
(<! barrier)
#last-return)))))
You can test that it works properly by setting the cache time to 0 and observe that the two function calls take approximately 10 seconds. Without the barrier the two calls would finish at the same time:
(def memo (memoize-af-until #(async/timeout 5000) 0 js/Date))
(async/take! (memo) #(println "[:a] Finished"))
(async/take! (memo) #(println "[:b] Finished"))
I have a situation where I am creating and destroying objects in one clojure namespace, and want another namespace to co-ordinate. However I do not want the first namespace to have to call the second explicitly on object destruction.
In Java, I could use a listener. Unfortunately the underlying java libraries do not signal events on object destruction. If I were in Emacs-Lisp, then I'd use hooks which do the trick.
Now, in clojure I am not so sure. I have found the Robert Hooke library https://github.com/technomancy/robert-hooke. But this is more like defadvice in elisp terms -- I am composing functions. More over the documentation says:
"Hooks are meant to extend functions you don't control; if you own the target function there are obviously better ways to change its behaviour."
Sadly, I am not finding it so obvious.
Another possibility would be to use add-watch, but this is marked as alpha.
Am I missing another obvious solution?
Example Added:
So First namespace....
(ns scratch-clj.first
(:require [scratch-clj.another]))
(def listf (ref ()))
(defn add-object []
(dosync
(ref-set listf (conj
#listf (Object.))))
(println listf))
(defn remove-object []
(scratch-clj.another/do-something-useful (first #listf))
(dosync
(ref-set listf (rest #listf)))
(println listf))
(add-object)
(remove-object)
Second namespace
(ns scratch-clj.another)
(defn do-something-useful [object]
(println "object removed is:" object))
The problem here is that scratch-clj.first has to require another and explicitly push removal events across. This is a bit clunky, but also doesn't work if I had "yet-another" namespace, which also wanted to listen.
Hence I thought of hooking the first function.
Is this solution suitable to your requirements?
scratch-clj.first:
(ns scratch-clj.first)
(def listf (atom []))
(def destroy-listeners (atom []))
(def add-listeners (atom []))
(defn add-destroy-listener [f]
(swap! destroy-listeners conj f))
(defn add-add-listener [f]
(swap! add-listeners conj f))
(defn add-object []
(let [o (Object.)]
(doseq [f #add-listeners] (f o))
(swap! listf conj o)
(println #listf)))
(defn remove-object []
(doseq [f #destroy-listeners] (f (first #listf)))
(swap! listf rest)
(println #listf))
Some listeners:
(ns scratch-clj.another
(:require [scratch-clj.first :as fst]))
(defn do-something-useful-on-remove [object]
(println "object removed is:" object))
(defn do-something-useful-on-add [object]
(println "object added is:" object))
Init binds:
(ns scratch-clj.testit
(require [scratch-clj.another :as another]
[scratch-clj.first :as fst]))
(defn add-listeners []
(fst/add-destroy-listener another/do-something-useful-on-remove)
(fst/add-add-listener another/do-something-useful-on-add))
(defn test-it []
(add-listeners)
(fst/add-object)
(fst/remove-object))
test:
(test-it)
=> object added is: #<Object java.lang.Object#c7aaef>
[#<Object java.lang.Object#c7aaef>]
object removed is: #<Object java.lang.Object#c7aaef>
()
It sounds a lot like what you're describing is callbacks.
Something like:
(defn make-object
[destructor-fn]
{:destructor destructor-fn :other-data "data"})
(defn destroy-object
[obj]
((:destructor obj) obj))
; somewhere at the calling code...
user> (defn my-callback [o] (pr [:destroying o]))
#'user/my-callback
user> (destroy-object (make-object my-callback))
[:destroying {:destructor #<user$my_callback user$my_callback#73b8cdd5>, :other-data "data"}]
nil
user>
So, here is my final solution following mobytes suggestion. A bit more work, but
I suspect that I will want this in future.
Thanks for all the help
;; hook system
(defn make-hook []
(atom []))
(defn add-hook [hook func]
(do
(when-not
(some #{func} #hook)
(swap! hook conj func))
#hook))
(defn remove-hook [hook func]
(swap! hook
(partial
remove #{func})))
(defn clear-hook [hook]
(reset! hook []))
(defn run-hook
([hook]
(doseq [func #hook] (func)))
([hook & rest]
(doseq [func #hook] (apply func rest))))
(defn phils-hook []
(println "Phils hook"))
(defn phils-hook2 []
(println "Phils hook2"))
(def test-hook (make-hook))
(add-hook test-hook phils-hook)
(add-hook test-hook phils-hook2)
(run-hook test-hook)
(remove-hook test-hook phils-hook)
(run-hook test-hook)