(def ^:dynamic *d* 1)
(binding [*d* 2]
(println *d*)
(repeatedly 1 #(println *d*)))
Output:
2
1
Why? Why does the function inside the repeatedly see the value of the dynamic var from outside the binding?
By the way, I checked (.getId (java.lang.Thread/currentThread)) inside and outside the anonymous function: it's the same.
The lazy sequence created by repeatedly is returned from the form, and then realized only when printed via the REPL, after the binding has been "unwound," and it is at this point that the anonymous function is being called. To see that this is the case, try these two variations:
(binding [*d* 2]
(println *d*)
(let [x (repeatedly 1 #(println *d*))]
(println (realized? x))
x))
and
(binding [*d* 2]
(println *d*)
(doall (repeatedly 1 #(println *d*))))
The second variation forces the sequence to be fully realized while still within the scope of the binding.
Note that another way to force the issue is to "capture" the binding by using bound-fn:
(binding [*d* 2]
(println *d*)
(repeatedly 1 (bound-fn [] (println *d*))))
Related
I'm learning core.async and have written a simple producer consumer code:
(ns webcrawler.parallel
(:require [clojure.core.async :as async
:refer [>! <! >!! <!! go chan buffer close! thread alts! alts!! timeout]]))
(defn consumer
[in out f]
(go (loop [request (<! in)]
(if (nil? request)
(close! out)
(do (print f)
(let [result (f request)]
(>! out result))
(recur (<! in)))))))
(defn make-consumer [in f]
(let [out (chan)]
(consumer in out f)
out))
(defn process
[f s no-of-consumers]
(let [in (chan (count s))
consumers (repeatedly no-of-consumers #(make-consumer in f))
out (async/merge consumers)]
(map #(>!! in %1) s)
(close! in)
(loop [result (<!! out)
results '()]
(if (nil? result)
results
(recur (<!! out)
(conj results result))))))
This code works fine when I step in through the process function in debugger supplied with Emacs' cider.
(process (partial + 1) '(1 2 3 4) 1)
(5 4 3 2)
However, if I run it by itself (or hit continue in the debugger) I get an empty result.
(process (partial + 1) '(1 2 3 4) 1)
()
My guess is that in the second case for some reason producer doesn't wait for consumers before exiting, but I'm not sure why. Thanks for help!
The problem is that your call to map is lazy, and will not run until something asks for the results. Nothing does this in your code.
There are 2 solutions:
(1) Use the eager function mapv:
(mapv #(>!! in %1) items)
(2) Use the doseq, which is intended for side-effecting operations (like putting values on a channel):
(doseq [item items]
(>!! in item))
Both will work and produce output:
(process (partial + 1) [1 2 3 4] 1) => (5 4 3 2)
P.S. You have a debug statement in (defn consumer ...)
(print f)
that produces a lot of noise in the output:
<#clojure.core$partial$fn__5561 #object[clojure.core$partial$fn__5561 0x31cced7
"clojure.core$partial$fn__5561#31cced7"]>
That is repeated 5 times back to back. You probably want to avoid that, as printing function "refs" is pretty useless to a human reader.
Also, debug printouts in general should normally use println so you can see where each one begins and ends.
I'm going to take a safe stab that this is being caused by the lazy behavior of map, and this line that's carrying out side effects:
(map #(>!! in %1) s)
Because you never explicitly use the results, it never runs. Change it to use mapv, which is strict, or more correctly, use doseq. Never use map to run side effects. It's meant to lazily transform a list, and abuse of it leads to behaviour like this.
So why is it working while debugging? I'm going to guess because the debugger forces evaluation as part of its operation, which is masking the problem.
As you can read from docstring map returns a lazy sequence. And I think the best way is to use dorun. Here is an example from clojuredocs:
;;map a function which makes database calls over a vector of values
user=> (map #(db/insert :person {:name %}) ["Fred" "Ethel" "Lucy" "Ricardo"])
JdbcSQLException The object is already closed [90007-170] org.h2.message.DbE
xception.getJdbcSQLException (DbException.java:329)
;;database connection was closed before we got a chance to do our transactions
;;lets wrap it in dorun
user=> (dorun (map #(db/insert :person {:name %}) ["Fred" "Ethel" "Lucy" "Ricardo"]))
DEBUG :db insert into person values name = 'Fred'
DEBUG :db insert into person values name = 'Ethel'
DEBUG :db insert into person values name = 'Lucy'
DEBUG :db insert into person values name = 'Ricardo'
nil
This code is from Clojure in action listing 3.5. When I try to run it i get the following error:
Can't dynamically bind non-dynamic var: joy.q/twice
Am I doing something wrong or has Clojure changed its binding rules since this book was printed?
(defn twice [x]
(println "original function")
(* 2 x))
(defn call-twice [y]
(twice y))
(defn with-log [function-to-call log-statement]
(fn [& args]
(println log-statement)
(apply function-to-call args)))
(call-twice 10)
(binding [twice (with-log twice "Calling the twice function")]
(call-twice 20))
From the binding documentation:
As of Clojure 1.3, vars need to be explicitly marked as ^:dynamic in order for
them to be dynamically rebindable
So you need:
(defn ^:dynamic twice [x]
(println "original function")
(* 2 x))
clojure.core has the macros bindings and with-redefs. Looking at the docstrings and the examples on clojuredocs.org, they seem to do something very similar. What is the difference and which one should I use in which situations?
Clojure Vars can have thread-local bindings. binding uses these, while with-redefs actually alters the root binding (which is someting like the default value) of the var.
Another difference is that binding only works for :dynamic vars while with-redefs works for all vars.
Examples:
user=> (def ^:dynamic *a* 1)
#'user/*a*
user=> (binding [*a* 2] *a*)
2
user=> (with-redefs [*a* 2] *a*)
2
user=> (binding [*a* 2] (doto (Thread. (fn [] (println "*a* is " *a*))) (.start) (.join)))
*a* is 1
#<Thread Thread[Thread-2,5,]>
user=> (with-redefs [*a* 2] (doto (Thread. (fn [] (println "*a* is " *a*))) (.start) (.join)))
*a* is 2
#<Thread Thread[Thread-3,5,]>
You can use the (undocumented) binding-conveyor-fn to convey thread-local bindings into new threads:
user=> (binding [*a* 2] (doto (Thread. (#'clojure.core/binding-conveyor-fn (fn [] (println "*a* is " *a*)))) (.start) (.join)))
*a* is 2
#<Thread Thread[Thread-5,5,]>
I'm new to Clojure, and am trying to redirect output to a file by rebinding *out*. In a simple case, it works nicely:
(binding [*out* (new java.io.FileWriter "test.txt")]
(println "Hi"))
This does what I expect, printing "Hi" to the file test.txt. However, if I introduce a for loop, things go awry:
(binding [*out* (new java.io.FileWriter "test.txt")]
(for [x [1 2]]
(println "Hi" x)))
This time, all the output goes to stdout, and the file is empty. What's going on here?
I'm using Leiningen, if that makes any difference:
Leiningen 2.0.0 on Java 1.7.0_13 Java HotSpot(TM) 64-Bit Server VM
you have been bitten by the lazy bug.
put a doall or dorun around the for and within the binding
(binding [*out* (new java.io.FileWriter "test.txt")]
(doall (for [x [1 2]]
(println "Hi" x))))
In your example the printing is happening then the result is printed by the repl after it is returned from the binding. So at the time of printing the binding is no longer in place.
Nothing is printed because result is a lazy sequence that will later be evaluated when used
user> (def result (binding [*out* (new java.io.FileWriter "test.txt")]
(for [x [1 2]]
(println "Hi" x))))
#'user/result
When the repl prints the resuls the printlns are evaluated:
user> result
(Hi 1
Hi 2
nil nil)
If we force the evaluation of the lazy sequence returned by for within the binding
nothing is printed to the repl,
user> (def result (binding [*out* (new java.io.FileWriter "test.txt")]
(doall (for [x [1 2]]
(println "Hi" x)))))
#'user/result
user> result
(nil nil)
and instead the output ends up in the file:
arthur#a:~/hello$ cat test.txt
Hi 1
Hi 2
I have a function that is supposed to take a lazy seq and return an unrealized lazy seq. Now I want to write a unit test (in test-is btw) to make sure that the result is an unrealized lazy sequence.
user=> (instance? clojure.lang.LazySeq (map + [1 2 3 4] [1 2 3 4]))
true
If you have a lot of things to test, maybe this would simplify it:
(defmacro is-lazy? [x] `(is (instance? clojure.lang.LazySeq ~x)))
user=> (is-lazy? 1)
FAIL in clojure.lang.PersistentList$EmptyList#1 (NO_SOURCE_FILE:7)
expected: (clojure.core/instance? clojure.lang.LazySeq 1)
actual: (not (clojure.core/instance? clojure.lang.LazySeq 1))
false
user=> (is-lazy? (map + [1 2 3 4] [1 2 3 4]))
true
As of Clojure 1.3 there is also the realized? function: "Returns true if a value has been produced for a promise, delay, future or lazy sequence."
Use a function with a side effect (say, writing to a ref) as the sequence generator function in your test case. If the side effect never happens, it means the sequence remains unrealized... as soon as the sequence is realized, the function will be called.
First, set it up like this:
(def effect-count (ref 0))
(defn test-fn [x]
(do
(dosync (alter effect-count inc))
x))
Then, run your function. I'll just use map, here:
(def result (map test-fn (range 1 10)))
Test if test-fn ever ran:
(if (= 0 #effect-count)
(println "Test passed!")
(println "Test failed!"))
Since we know map is lazy, it should always work at this point. Now, force evaluation of the sequence:
(dorun result)
And check the value of effect-count again. This time, we DO expect the side effect to have triggered. And, it is so...
user=>#effect-count
9