I'm trying to understand what ^:const does in clojure. This is what the dev docs say. http://dev.clojure.org/display/doc/1.3
(def constants
{:pi 3.14
:e 2.71})
(def ^:const pi (:pi constants))
(def ^:const e (:e constants))
The overhead of looking up :e and :pi in the map happens at compile time, as (:pi constants) and (:e constants) are evaluated when their parent def forms are evaluated.
This is confusing since the metadata is for the var bound to symbol pi, and the var bound to symbol e, yet the sentence below says it helps speed up the map lookups, not the var lookups.
Can someone explain the what ^:const is doing and the rationale behind using it? How does this compare to using a giant let block or using a macro like (pi) and (e)?
That looks like a bad example to me, since the stuff about map-lookup just confuses the issue.
A more realistic example would be:
(def pi 3.14)
(defn circ [r] (* 2 pi r))
In this case, the body of circumference is compiled into code that dereferences pi at runtime (by calling Var.getRawRoot), each time circumference is called.
(def ^:const pi 3.14)
(defn circ2 [r] (* 2 pi r))
In this case, circ2 is compiled into exactly the same code as if it had been written like this:
(defn circ2 [r] (* 2 3.14 r))
That is, the call to Var.getRawRoot is skipped, which saves a little bit of time. Here is a quick measurement, where circ is the first version above, and circ2 is the second:
user> (time (dotimes [_ 1e5] (circ 1)))
"Elapsed time: 16.864154 msecs"
user> (time (dotimes [_ 1e5] (circ2 1)))
"Elapsed time: 6.854782 msecs"
Besides the efficiency aspect described above, there is a safety aspect that is also useful. Consider the following code:
(def two 2)
(defn times2 [x] (* two x))
(assert (= 4 (times2 2))) ; Expected result
(def two 3) ; Ooops! The value of the "constant" changed
(assert (= 6 (times2 2))) ; Used the new (incorrect) value
(def ^:const const-two 2)
(defn times2 [x] (* const-two x))
(assert (= 4 (times2 2))) ; Still works
(def const-two 3) ; No effect!
(assert (= 3 const-two )) ; It did change...
(assert (= 4 (times2 2))) ; ...but the function did not.
So, by using the ^:const metadata when defining vars, the vars are effectively "inlined" into every place they are used. Any subsequent changes to the var, therefore, do not affect any code where the "old" value has already been inlined.
The use of ^:const also serves a documentation function. When one reads (def ^:const pi 3.14159) is tells the reader that the var pi is not ever intended to change, that it is simply a convenient (& hopefully descriptive) name for the value 3.14159.
Having said all the above, note that I never use ^:const in my code, since it is deceptive and provides "false assurance" that a var will never change. The problem is that ^:const implies one cannot redefine a var, but as we saw with const-two it does not prevent the var from being changed. Instead, ^:const hides the fact that the var has a new value, since const-two has been copied/inlined (at compile-time) to each place of use before the var is changed (at run-time).
A much better solution would be to throw an Exception upon attempting to change a ^:const var.
In the example docs they are trying to show that in most cases if you def a var to be the result of looking something up in a map without using const, then the lookup will happen when the class loads. so you pay the cost once each time you run the program (not at every lookup, just when the class loads). And pay the cost of looking up the value in the var each time it is read.
If you instead make it const then the compiler will preform the lookup at compile time and then emit a simple java final variable and you will pay the lookup cost only once total at the time you compile the program.
This is a contrived example because one map lookup at class load time and some var lookups at runtime are basically nothing, though it illustrates the point that some work happens at compile time, some at load time, and the rest well ... the rest of the time
Related
I've just started learning Clojure and trying to understand the difference between 2 approaches which at first sight seem very identical.
(def func0 (delay (do
(println "did some work")
100)))
so.core=> (force my-delay2)
did some work
100
so.core=> (force my-delay2)
100
(defn vanilla-func [] (println "did some work") 100)
(def func1 (memoize vanilla-func))
so.core=> (func1)
did some work
100
so.core=> (func1)
100
Both approaches do some sort of function memoization. What am I missing?
I've tried to find the explanation on https://clojuredocs.org/clojure.core/delay & https://clojuredocs.org/clojure.core/memoize but couldn't.
delay holds one result and you have to deref to get the result.
memoize is an unbound cache, that caches the result depending on the
input arguments. E.g.
user=> (def myinc (memoize (fn [x] (println x) (inc x))))
#'user/myinc
user=> (myinc 1)
1
2
user=> (myinc 1)
2
In your (argument-less) example the only difference is that you can use
the result directly (no deref needed)
Classic use-cases for delay are things needed later, that would block
or delay startup. Or if you want to "hide" top-level defs from
the compiler (e.g. they do side-effects).
memoize is a classic cache and is best used if the calculation is
expensive and the set of input arguments is not excessive. There are
other caching options in the clojure-verse, that allow better
configurations (e.g. they are not unbound).
Suppose I've got a function (remove-bad-nodes g) that returns a sequence like this:
[updated-g bad-nodes]
where updated-g is a graph with its bad nodes removed, and bad-nodes is a collection containing the removed nodes.
As an argument to a function or inside a let, I could destructure it like this:
(let [[g bads] (remove-bad-nodes g)]
...)
but that only defines local variables. How could I do that in the REPL, so that in future commands I can refer to the updated graph as g and the removed nodes as bads? The first thing that comes to mind is this:
(def [g bads] (remove-bad-nodes g)
but that doesn't work, because def needs its first argument to be a Symbol.
Note that I'm not asking why def doesn't have syntax like let; there's already a question about that. I'm wondering what is a convenient, practical way to work in the REPL with functions that return "multiple values". If there's some reason why in normal Clojure practice there's no need to destructure in the REPL, because you do something else instead, explaining that might make a useful answer. I've been running into this a lot lately, which is why I'm asking. Usually, but not always, these functions return an updated version of something along with some other information. In side-effecting code, the function would modify the object and return only one value (the removed nodes, in the example), but obviously that's not the Clojurely way to do it.
I think the way to work with such functions in the repl is just to not def your intermediate results unless they are particularly interesting; for interesting-enough intermediate results it's not a big hassle to either def them to a single name, or to write multiple defs inside a destructuring form.
For example, instead of
(def [x y] (foo))
(def [a b] (bar x y))
you could write
(let [[x y] (foo),
[x' y'] (bar x y)])
(def a x') ; or maybe leave this out if `a` isn't that interesting
(def b y'))
A nice side effect of this is that the code you write while playing around in the repl will look much more similar to the code you will one day add to your source file, where you will surely not be defing things over and over, but rather destructuring them, passing them to functions, and so on. It will be easier to adapt the information you learned at the repl into a real program.
There's nothing unique about destructuring w/r/t the REPL. The answer to your question is essentially the same as this question. I think your options are:
let:
(let [[light burnt just-right] (classify-toasts (make-lots-of-toast))]
(prn light burnt just-right))
def the individual values:
(def result (classify-toasts (make-lots-of-toast)))
(def light (nth result 0))
(def burnt (nth result 1))
(def just-right (nth result 2))
Or write a macro to do that def work for you.
You could also consider a different representation if your function is always returning a 3-tuple/vector e.g. you could alternatively return a map from classify-toasts:
{:light 1, :burnt 2, :just-right 3}
And then when you need one of those values, destructure the map using the keywords wherever you need:
(:light the-map) => 1
Observe:
user=> (def results [1 2 3])
#'user/results
user=> (let [[light burnt just-right] results] (def light light) (def burnt burnt) (def just-right just-right))
#'user/just-right
user=> light
1
user=> burnt
2
user=> just-right
3
I find I very rarely use let in Clojure. For some reason I took a dislike to it when I started learning and have avoided using it ever since. It feels like the flow has stopped when let comes along. I was wondering, do you think we could do without it altogether ?
You can replace any occurrence of (let [a1 b1 a2 b2...] ...) by ((fn [a1 a2 ...] ...) b1 b2 ...) so yes, we could. I am using let a lot though, and I'd rather not do without it.
Let offers a few benefits. First, it allows value binding in a functional context. Second, it confers readability benefits. So while technically, one could do away with it (in the sense that you could still program without it), the language would be impoverished without a valuable tool.
One of the nice things about let is that it helps formalize a common (mathematical) way of specifying a computation, in which you introduce convenient bindings and then a simplified formula as a result. It's clear the bindings only apply to that "scope" and it's tie in with a more mathematical formulation is useful, especially for more functional programmers.
It's not a coincidence that let blocks occur in other languages like Haskell.
Let is indispensable to me in preventing multiple execution in macros:
(defmacro print-and-run [s-exp]
`(do (println "running " (quote ~s-exp) "produced " ~s-exp)
s-exp))
would run s-exp twice, which is not what we want:
(defmacro print-and-run [s-exp]
`(let [result# s-exp]
(do (println "running " (quote ~s-exp) "produced " result#)
result#))
fixes this by binding the result of the expression to a name and referring to that result twice.
because the macro is returning an expression that will become part of another expression (macros are function that produce s-expressions) they need to produce local bindings to prevent multiple execution and avoid symbol capture.
I think I understand your question. Correct me if it's wrong. Some times "let" is used for imperative programming style. For example,
... (let [x (...)
y (...x...)
z (...x...y...)
....x...y...z...] ...
This pattern comes from imperative languages:
... { x = ...;
y = ...x...;
...x...y...;} ...
You avoid this style and that's why you also avoid "let", don't you?
In some problems imperative style reduces amount of code. Furthermore, some times It's more efficient to write in java or c.
Also in some cases "let" just holds values of subexpressions regardless of evaluation order. For example,
(... (let [a (...)
b (...)...]
(...a...b...a...b...) ;; still fp style
There are at least two important use cases for let-bindings:
First, using let properly can make your code clearer and shorter. If you have an expression that you use more than once, binding it in a let is very nice. Here's a portion of the standard function map that uses let:
...
(let [s1 (seq c1) s2 (seq c2)]
(when (and s1 s2)
(cons (f (first s1) (first s2))
(map f (rest s1) (rest s2)))))))
...
Even if you use an expression only once, it can still be helpful (to future readers of the code) to give it a semantically meaningful name.
Second, as Arthur mentioned, if you want to use the value of an expression more than once, but only want it evaluated once, you can't simply type out the entire expression twice: you need some kind of binding. This would be merely wasteful if you have a pure expression:
user=> (* (+ 3 2) (+ 3 2))
25
but actually changes the meaning of the program if the expression has side-effects:
user=> (* (+ 3 (do (println "hi") 2))
(+ 3 (do (println "hi") 2)))
hi
hi
25
user=> (let [x (+ 3 (do (println "hi") 2))]
(* x x))
hi
25
Stumbled upon this recently so ran some timings:
(testing "Repeat vs Let vs Fn"
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
(* (+ 3 2) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)
n (+ 3 2)]
(dotimes [x 1000000]
(* n n))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
((fn [x] (* x x)) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))))
Output
Testing Repeat vs Let vs Fn
116
18
60
'let' wins over 'pure' functional.
How do I programmatically figure out which Vars may affect the results of a function defined in Clojure?
Consider this definition of a Clojure function:
(def ^:dynamic *increment* 3)
(defn f [x]
(+ x *increment*))
This is a function of x, but also of *increment* (and also of clojure.core/+(1); but I'm less concerned with that). When writing tests for this function, I want to make sure that I control all relevant inputs, so I do something like this:
(assert (= (binding [*increment* 3] (f 1)) 4))
(assert (= (binding [*increment* -1] (f 1)) 0))
(Imagine that *increment* is a configuration value that someone might reasonably change; I don't want this function's tests to need changing when this happens.)
My question is: how do I write an assertion that the value of (f 1) can depend on *increment* but not on any other Var? Because I expect that one day someone will refactor some code and cause the function to be
(defn f [x]
(+ x *increment* *additional-increment*))
and neglect to update the test, and I would like to have the test fail even if *additional-increment* is zero.
This is of course a simplified example – in a large system, there can be lots of dynamic Vars, and they can get referenced through a long chain of function calls. The solution needs to work even if f calls g which calls h which references a Var. It would be great if it didn't claim that (with-out-str (prn "foo")) depends on *out*, but this is less important. If the code being analyzed calls eval or uses Java interop, of course all bets are off.
I can think of three categories of solutions:
Get the information from the compiler
I imagine the compiler does scan function definitions for the necessary information, because if I try to refer to a nonexistent Var, it throws:
user=> (defn g [x] (if true x (+ *foobar* x)))
CompilerException java.lang.RuntimeException: Unable to resolve symbol: *foobar* in this context, compiling:(NO_SOURCE_PATH:24)
Note that this happens at compile time, and regardless of whether the offending code will ever be executed. Thus the compiler should know what Vars are potentially referenced by the function, and I would like to have access to that information.
Parse the source code and walk the syntax tree, and record when a Var is referenced
Because code is data and all that. I suppose this means calling macroexpand and handling each Clojure primitive and every kind of syntax they take. This looks so much like a compilation phase that it would be great to be able to call parts of the compiler, or somehow add my own hooks to the compiler.
Instrument the Var mechanism, execute the test and see which Vars get accessed
Not as complete as the other methods (what if a Var is used in a branch of the code that my test fails to exercise?) but this would suffice. I imagine I would need to redefine def to produce something that acts like a Var but records its accesses somehow.
(1) Actually that particular function doesn't change if you rebind +; but in Clojure 1.2 you can bypass that optimization by making it (defn f [x] (+ x 0 *increment*)) and then you can have fun with (binding [+ -] (f 3)). In Clojure 1.3 attempting to rebind + throws an error.
Regarding your first point you could consider using the analyze library. With it you can quite easily figure out which dynamic vars are used in an expression:
user> (def ^:dynamic *increment* 3)
user> (def src '(defn f [x]
(+ x *increment*)))
user> (def env {:ns {:name 'user} :context :eval})
user> (->> (analyze-one env src)
expr-seq
(filter (op= :var))
(map :var)
(filter (comp :dynamic meta))
set)
#{#'user/*increment*}
I know that this doesn't answer your question, but wouldn't it be a lot less work to just provide two versions of a function where one version has no free variables, and the other version calls the first one with the appropriate top-level defines?
For example:
(def ^:dynamic *increment* 3)
(defn f
([x]
(f x *increment*))
([x y]
(+ x y)))
This way you can write all your tests against (f x y), which doesn't rely on any global state.
OK. I've been tinkering with Clojure and I continually run into the same problem. Let's take this little fragment of code:
(let [x 128]
(while (> x 1)
(do
(println x)
(def x (/ x 2)))))
Now I expect this to print out a sequence starting with 128 as so:
128
64
32
16
8
4
2
Instead, it's an infinite loop, printing 128 over and over. Clearly my intended side effect isn't working.
So how am I supposed to redefine the value of x in a loop like this? I realize this may not be Lisp like (I could use an anonymous function that recurses on it's self, perhaps), but if I don't figure out how to set variable like this, I'm going to go mad.
My other guess would be to use set!, but that gives "Invalid assignment target", since I'm not in a binding form.
Please, enlighten me on how this is supposed to work.
def defines a toplevel var, even if you use it in a function or inner loop of some code. What you get in let are not vars. Per the documentation for let:
Locals created with let are not variables. Once created their values never change!
(Emphasis not mine.) You don't need mutable state for your example here; you could use loop and recur.
(loop [x 128]
(when (> x 1)
(println x)
(recur (/ x 2))))
If you wanted to be fancy you could avoid the explicit loop entirely.
(let [xs (take-while #(> % 1) (iterate #(/ % 2) 128))]
(doseq [x xs] (println x)))
If you really wanted to use mutable state, an atom might work.
(let [x (atom 128)]
(while (> #x 1)
(println #x)
(swap! x #(/ %1 2))))
(You don't need a do; while wraps its body in an explicit one for you.) If you really, really wanted to do this with vars you'd have to do something horrible like this.
(with-local-vars [x 128]
(while (> (var-get x) 1)
(println (var-get x))
(var-set x (/ (var-get x) 2))))
But this is very ugly and it's not idiomatic Clojure at all. To use Clojure effectively you should try to stop thinking in terms of mutable state. It will definitely drive you crazy trying to write Clojure code in a non-functional style. After a while you may find it to be a pleasant surprise how seldom you actually need mutable variables.
Vars (that's what you get when you "def" something) aren't meant to be reassigned (but can be):
user=> (def k 1)
#'user/k
user=> k
1
There's nothing stopping you from doing:
user=> (def k 2)
#'user/k
user=> k
2
If you want a thread-local settable "place" you can use "binding" and "set!":
user=> (def j) ; this var is still unbound (no value)
#'user/j
user=> j
java.lang.IllegalStateException: Var user/j is unbound. (NO_SOURCE_FILE:0)
user=> (binding [j 0] j)
0
So then you can write a loop like this:
user=> (binding [j 0]
(while (< j 10)
(println j)
(set! j (inc j))))
0
1
2
3
4
5
6
7
8
9
nil
But I think this is pretty unidiomatic.
If you think that having mutable local variables in pure functions would be a nice convenient feature that does no harm, because the function still remains pure, you might be interested in this mailing list discussion where Rich Hickey explains his reasons for removing them from the language. Why not mutable locals?
Relevant part:
If locals were variables, i.e. mutable, then closures could close over
mutable state, and, given that closures can escape (without some extra
prohibition on same), the result would be thread-unsafe. And people
would certainly do so, e.g. closure-based pseudo-objects. The result
would be a huge hole in Clojure's approach.
Without mutable locals, people are forced to use recur, a functional
looping construct. While this may seem strange at first, it is just as
succinct as loops with mutation, and the resulting patterns can be
reused elsewhere in Clojure, i.e. recur, reduce, alter, commute etc
are all (logically) very similar. Even though I could detect and
prevent mutating closures from escaping, I decided to keep it this way
for consistency. Even in the smallest context, non-mutating loops are
easier to understand and debug than mutating ones. In any case, Vars
are available for use when appropriate.
The majority of the subsequent posts concerns implementing a with-local-vars macro ;)
You could more idiomatically use iterate and take-while instead,
user> (->> 128
(iterate #(/ % 2))
(take-while (partial < 1)))
(128 64 32 16 8 4 2)
user>