I'm having a little trouble with a threading macro I'm trying to write. I've posted a stripped down version to show what problem I'm having.
(defmacro <->
[v & fs]
`(do
(-> ~v ~#fs)
~v))
The macro is equivalent to the thread first -> macro, but instead of returning the result of the threading operation, it returns the original value it was passed.
The trouble, I'm having is that when I do something like:
(<-> 1
(<-> println)
(<-> println))
I would expect the output to be
1
1
=> 1
but because the macro evaluates outside in, the macroexpand looks like:
(do
(do
(println
(do
(println 1)
1))
(do
(println 1)
1)) 1)
and the result is
1
1
1
=> 1
I can see why this is happening since the macro is evaluated from outside in, but I'm not sure how to fix it so the macro actually works as expected (i.e. evaluating the value v before threading it to the next form).
Your macro expands to a form in which its v argument is evaluated twice. You need to evaluate v only once, and let-bind the result so you can refer to that value later.
(defmacro <-> [v & fs]
(let [$v (gensym "$v_")]
`(let [~$v ~v]
(-> ~$v ~#fs)
~$v)))
Note the use of gensym to generate a fresh symbol.
Related
user=> (def v-1 "this is v1")
user=> (def v-2 "this is v2")
user=> (defmacro m [v] (symbol (str "v-" v)))
user=> (m 1)
"this is v1"
user=> (m 2)
"this is v2"
user=> (let [i 2] (m i))
CompilerException java.lang.RuntimeException: Unable to resolve symbol: v-i in this context, compiling:(NO_SOURCE_PATH:73:12)
Can I write a macro let both
(m 2)
and
(let [i 2] (m i))
get "this is v2" ?
This is possible without a macro:
(defn m [v] (var-get (resolve (symbol (str "v-" v)))))
(m 1) ;; => "This is v1"
(let [i 2] (m i)) ;; => "This is v2"
You can use a macro too if you want:
(defmacro m [v] `#(resolve (symbol (str "v-" ~v))))
A plain function seems much more likely to be what you want.
First, though, to address the original question, if you wanted to insist on using a macro, macros are regular functions that happen to be called at compile time, so you can look up a Var using its symbolic name and obtain its value using deref just like you could at (your application's, as opposed to your macro's) runtime:
(defmacro var-value [vsym] #(resolve vsym))
(def foo 1)
(var-value foo)
;= 1
(macroexpand-1 '(var-value foo))
;= 1
Note that the above 1 is the actual macroexpansion here. This is different to
(defmacro var-value [vsym] `#(resolve ~vsym))
in that the latter expands to a call to resolve, and so the lookup given that implementation is postponed to your app's runtime.
(macroexpand-1 '(var-value foo))
;= (clojure.core/deref (clojure.core/resolve foo))
So this code will just be inlined wherever you call the macro.
Of course the macro could also expand to a symbol – e.g.
(defmacro prefixed-var [suffix]
`(symbol (str "v-" ssuffix)))
will produce expansions like v-1 (for (prefixed-var 1)) etc.
Going back to the subject of the suitability of macros here, however, if you use a macro, all the information that you need to produce your expansion must be available at compile time, and so in general you cannot use the values of let / loop locals or function arguments in your expansion for the fundamental reason that they don't have any fixed value at compile time.1
Thus the cleanest approach would probably be to wrap a resolve call in defn and call the resulting function – although of course to know for sure, we'd need to know what problem you were trying to solve by introducing a macro that performs a Var lookup.
1 Except if statically assigned constant values, as in the example given in the question text; I'm assuming you're thinking of using runtime values of locals in general, not just those that whose initialization expressions are constant literals.
Here's my failed attempt:
(defmacro until
[condition body setup increment]
`(let [c ~#condition]
(loop [i setup]
(when (not c)
(do
~#body
(recur ~#increment))))))
(def i 1)
(until (> i 5)
(println "Number " i)
0
(inc i))
I get: CompilerException java.lang.RuntimeException: Can't let qualified name: clojure-noob.core/c
I am expecting this output:
Number 1
Number 2
Number 3
Number 4
Number 5
What's wrong?
There are a few issues with the macro:
You need to generate symbols for bindings inside macros. A convenient way to do this is suffix the names with #. Otherwise the bindings in your macros could overshadow bindings elsewhere in your code.
Some of the macro inputs were unnecessarily spliced when unquoted i.e. ~# instead of ~
Here's a version of the macro that will compile/expand:
(defmacro until [condition body setup increment]
`(let [c# ~condition]
(loop [i# ~setup]
(when-not c#
~body
(recur ~increment)))))
But this will loop forever in your example because condition is only evaluated once and i's value would never change anyway. We could fix that:
(defmacro until [condition body increment]
`(loop []
(when-not ~condition
~body
~increment
(recur))))
And we need to make i mutable if we want to change its value:
(def i (atom 1))
(until (> #i 5)
(println "Number " #i)
(swap! i inc))
;; Number 1
;; Number 2
;; Number 3
;; Number 4
;; Number 5
But now until is starting to look a lot like the complement of while, and its extra complexity doesn't seem beneficial.
(defmacro until [test & body]
`(loop []
(when-not ~test
~#body
(recur))))
This version of until is identical to while except the test is inverted, and the sample code above with the atom still behaves correctly. We can further simplify until by using while directly, and it'll ultimately expand to the same code:
(defmacro until [test & body]
`(while (not ~test) ~#body))
Change the let line too:
...
`(let [c# ~#condition]
...
Then rename all references of c to c#. The postfix # generates a unique, non-namespaced-qualified identifier to ensure that the symbol created by the macro doesn't clash with any existing symbols in the context that the macro expands into. Whenever you bind a symbol in a quoted form, you should be using # to prevent collisions, unless you have a good reason to not use it.
Why is this necessary in this case? I can't remember exactly the reason, but if I recall correctly, any symbols bound in a syntax quoted form (`()) are namespace qualified, and you can't use a let to create namespace qualified symbols.
You can recreate the error by typing:
(let [a/a 1]
a/a)
I want to write a macro (my-dotimes [x init end] & body) that computes the value of body for x going from init to end-1 in increments of 1. Here you again have to make sure to avoid the "variable capture problem". It should work like this:
user=> (my-dotimes [x 0 4] (print x))
0123nil
my code is :
(defmacro my-dotimes [[x initial end] & body]
`(loop [i# ~initial]
(when (< i# ~end)
~#body
(recur (inc i#))))))
but when I use macroexpand to check it and find:
user=> (macroexpand '(my-dotimes [x 0 4] (println x)))
(loop* [i__4548__auto__ 0] (clojure.core/when (clojure.core/<i__4548__auto__ 4)
(println x)
(recur (clojure.core/inc i__4548__auto__))))
I am wondering how to change
(println x) => (clojure.core/println i__4548__auto__)
Here, you supply the symbol that should be bound to the counter (here x), so you don't need to use gensyms.
Instead of using i#, just introduce the symbol given to you by the user of the macro.
You need gensyms when you introduce new symbols and don't want them to collide with existing symbols.
In Common Lisp, it would make sense to wrap the body with a binding from the user-supplied symbol to the current value of i, using (let ((,x ,i)) ,#body), because the user's code could change the value of the counter during iteration (which could be bad). But here I think you cannot mutate the variable directly, so you don't need to worry about that.
Your second example is:
(defmacro for-loop [[symb ini t change] & body]
`(loop [symb# ~ini]
(if ~t
~#body
(recur ~change))))
First problem: when you expand the body, which might be one or more form, you'll end-up with an if form with many branches instead of 2. You would have for example (if test x1 x2 x3 (recur ...)) if your body contains x1, x2 and x3. You need to wrap bodies in do expressions, with (do ~#body).
Now, the situation is not very different than before: you still have a symbol, given by the user, and you are responsible for establishing the bindings in the macro. Instead of using symb#, which creates a new symbol, completely distinct from symb, just use symb directly.
You could do this for example (untested):
(defmacro for-loop [[symb init test change] &body]
`(loop [~symb ~init]
(if ~test (do ~#body) (recur ~change))))
As long as you use the symbol provided by the caller of your macro, gensyms are not necessary. You need gensyms when you have to create a new variable in the generated code, which requires to have a fresh symbol. For example, you evaluate an expression only once and need a variable to hold its value:
(defmacro dup [expr]
`(let [var# ~expr]
[var# var#]))
I'm examining the Listing 11.9 of the named book (p.269 of pdf).
Could anyone explain me how tests value is being set (line [tests all-tests :as results])?
thanks
To set the context of the question for people without The Joy of Clojure (a book I do enjoy btw), the macro in question is:
(defmacro with-promises [[n tasks _ as] & body]
(when as
`(let [tasks# ~tasks
n# (count tasks#)
promises# (take n# (repeatedly promise))]
(dotimes [i# n#]
(dothreads!
(fn []
(deliver (nth promises# i#)
((nth tasks# i#))))))
(let [~n tasks#
~as promises#]
~#body))))
And is used thusly:
(defn run-tests [& all-tests]
(with-promises
[tests all-tests :as results]
(into (TestRun. 0 0 0)
(reduce #(merge-with + %1 %2) {}
(for [r results]
(if #r
{:run 1 :passed 1}
{:run 1 :failed 1}))))))
and the final call to run-tests is like:
(run-tests pass fail fail fail pass)
=> #user.TestRun{:run 5, :passed 2, :failed 3}
Ultimately, the latter part of the macro is doing a let assignment and running the body, so you end up with
(let [tests tasks#
results promises#]
(into (TestRun. 0 0 0)
;; rest of body
In the macro, ~n is unquoting the starting back-tick around the '(let so you can just read it as n which is the first parameter to the macro (well, the first parameter of the vector that is the first parameter to the macro).
This all happens after the macro has setup the promises using the custom dothreads! function that uses a thread pool - non of which is important to understand the macro.
You can determine more about the macro by wrapping it in a (pprint (macroexpand-1 '(with-promises ... which generates something like (I've replaced the auto generated names with something simpler, v1, n1, p1 and i1):
(clojure.core/let
[v1 all-tests
n1 (clojure.core/count v1)
p1 (clojure.core/take n1 (clojure.core/repeatedly clojure.core/promise))]
(clojure.core/dotimes
[i1 n1]
(user/dothreads!
(clojure.core/fn
[]
(clojure.core/deliver
(clojure.core/nth p1 i1)
((clojure.core/nth v1 i1))))))
(clojure.core/let
[tests v1
results p1]
(into
(TestRun. 0 0 0)
;; ... rest of main body
which clearly shows the parameters passed in are used as variables in the final let bindings.
However, in this example usage (i.e. run-tests function), the tests variable isn't actually used in the body of the with-promises call, only results is, so you're right to question it, it simply isn't needed.
Looking at the macro definition, there may be further optimizations for this case, as the tasks# binding doesn't seem to give anything extra than wrapping tasks. At first I wondered if this was about immutability in the dothreads! call, or macro-niceness for providing a closure around the usage, rather than directly using the parameter to the macro.
I tried changing the macro to remove tasks# completely and directly use ~tasks, which seems to still work, and as "tests" isn't a required binding variable in the body of run-tests, you can drop both the n parameter from the macro, and the ~n tasks# part of the final let binding without issue.
Actually after reading it several times it finally dawned on me it's to make the whole vector read like a standard destructuring binding.
EDIT: some more explanation on "tests".
This is just a name, it could be "foo-tests", "foo-bar", because ultimately it's used to define something in a let binding.
Had the run-tests body been something like:
(defn run-tests [& all-tests]
(with-promises
[foo all-tests :as results]
(println "foo was set to" foo)
(into (TestRun. 0 0 0)
;; rest of body
you can see how foo (and results) are just used to ultimately define variables (eck - you know what i mean) that can be used in the body part of the call to the macro. The body being everything after the initial vector [foo all-tests :as results] but in the original code, tests is declared but unused.
tests appears to be a function, so :as puts the result (output) of running tests on all-tests.
(edit:)
Upon careful inspection, the with-promises macro appears to be setting the tests to the count of tests.
From what I'm reading (don't know much about macros), the arguments appear to map ("tests" "all-tests" ":as" "results") -> ("n" "tasks" "_" "as") but what I can't quite get is that would imply when requires a value for results ("as") when we're supposed to be creating it. Anyway, the value of tests is set in the final let of the macro.
This code is far too clever, in my humble opinion. Fogus is a master, but this is not his best work.
(If I'm wrong, hopefully someone will be inspired.)
My puzzle is the following example:
(defmacro macro1 [x]
(println x))
(defn func1 [x]
(println x))
(defmacro macro2 [x]
`(macro1 ~x)
(func1 x))
(defmacro macro3 [x]
(func1 x)
`(macro1 ~x))
(println "macro2")
(macro2 hello)
(println "macro3")
(macro3 hello)
Surprisingly, the output is:
macro2
hello
macro3
hello
hello
Why the output of macro2 and macro3 are different? In my understanding, all the calling of macro inside macro could be substituted with function (except for the reason of reuse). Anything wrong in my understanding?
Thanks Michael for clarifying. My general question is how to choose between using function or macro inside macro for the purpose of manipulating the s-expression. I wonder whether they can be used exchangeably except that they're evaled at different phases. Another example:
(defn manipulate-func [x]
(list + x 1))
(defmacro manipulate-macro [x]
(list + x 1))
(defmacro macro1 [x y]
[(manipulate-func x) `(manipulate-macro ~y)])
(println (clojure.walk/macroexpand-all '(macro1 (+ 1 2) (+ 3 4))))
;; [(#<core$_PLUS_ clojure.core$_PLUS_#332b9f79> (+ 1 2) 1) (#<core$_PLUS_ clojure.core$_PLUS_#332b9f79> (+ 3 4) 1)]
macro2 doesn't call macro1. Look at its body:
`(macro1 ~x)
(func1 x)
The first line is syntax-quoted; its value is list structure of the form (user/macro1 x-value) (assuming macro1 is defined in the user namespace; x-value here is the literal argument provided to macro2) and it has no side effects. Because there are no side effects and the value is discarded, this line has no effect.
Responding to the edit:
Firstly, it is important to distinguish calling another macro inside a macros body from emitting a call to another macro:
(defmacro some-macro []
...)
;; calls some-macro:
(defmacro example-1 []
(some-macro))
;; emits a call to some-macro:
(defmacro example-2 []
`(some-macro))
Secondly, in the case of calling functions and macros inside a macro's body, one must keep in mind what the relevant notions of runtime and compile time are:
functions called by a macro will be called at the macro expander's runtime, which is compile time from the point of view of user code;
macros called by a macro will be expanded when the macro body is compiled.
If a macro emits a call to another macro, the notions of runtime and compile time relevant to the emitted macro call will be the same as those relevant to the original macro call. If a macro calls another macro, they are shifted one step back, as it were.
To illustrate, let's consider a macro that delegates all its work to a helper function:
(defn emit-abc [abc-name [a b c]]
`(def ~abc-name {:a ~a :b ~b :c ~c}))
(defmacro defabc [abc-name abc-vals]
(emit-abc abc-name abc-vals))
From the REPL:
user> (defabc foo [1 2 3])
#'user/foo
user> foo
{:a 1, :c 3, :b 2}
If emit-abc were itself a macro, the above definition of defabc wouldn't even compile, because emit-abc would attempt to destructure the literal symbol abc-vals, throwing an UnsupportedOperationException.
Here's another example that makes it easier to explain what's happening:
(let [[a b c] [1 2 3]]
(defabc foo [a b c]))
defabc receives the vector of the three literal symbols a, b and c as the second argument; it has no access to the runtime values 1, 2 and 3. It passes this exact vector of symbols to the function emit-abc, which is then able to reach into this vector and extract the symbols to produce the map {:a a :b b :c c}. This map becomes the expansion of the defabc call. At runtime a, b and c turn out to be bound to the values 1, 2 and three, and so the map {:a 1 :b 2 :c 3} is produced.
Suppose we tried to write emit-abc as a macro with the same body (just changing defn to defmacro in its definition). Then we couldn't usefully call it from defabc, because we wouldn't have any way of conveying to it the actual values of the arguments to defabc. We could write
(emit-abc abc-name [(abc-vals 0) (abc-vals 1) (abc-vals 2)])
to make defabc compile, but this would end up emitting abc-name as the name of the Var being defined and include code for the vector literal [a b c] three times in the generated code. We could however emit a call to it:
`(emit-abc ~abc-name ~abc-vals)
This works as expected.
I think you're confused about the difference between macros and functions.
Macros are evaluated at compile time, and they take code as input and give code as output.
Functions evaluate their results at run time, taking run-time values as input and returning run-time values as output.
The result of a macro should pretty much always be an s-expression representing the source code resulting form applying the macro. This is why macros usually use the syntax quote functionality, since it makes it easy to generate source code with inserted parameterized values via the ~ and ~# escapes.
Defining a couple of functions might help you see how this works. Let's run the following code:
(defn testing-macro-2 [my-arg]
(macro2 my-arg))
(testing-macro-2 "macro 2 test")
(defn testing-macro-3 [my-arg]
(macro3 my-arg))
(testing-macro-3 "macro 3 test")
Here's what I get in my REPL:
user=>
(defn testing-macro-2 [my-arg]
(macro2 my-arg))
my-arg
#'user/testing-macro-2
user=> (testing-macro-2 "macro 2 test")
nil
user=>
(defn testing-macro-3 [my-arg]
(macro3 my-arg))
my-arg
my-arg
#'user/testing-macro-3
user=> (testing-macro-3 "macro 3 test")
nil
As you can see, my-arg is printed when defining the functions where the macros are invoked, not when I call the functions. This is because the macros are evaluated when the Clojure compiler is generating code for the function, so that's when the call to println happens.
However, if you use the syntax-quote in macro1 to make it return code instead of calling println, which returns nil, then it all changes:
user=>
(defmacro macro1 [x]
`(println ~x))
#'user/macro1
user=>
(defn func1 [x]
(println x))
#'user/func1
user=>
(defmacro macro2 [x]
`(macro1 ~x)
(func1 x))
#'user/macro2
user=>
(defmacro macro3 [x]
(func1 x)
`(macro1 ~x))
#'user/macro3
user=>
(defn testing-macro-2 [my-arg]
(macro2 my-arg))
my-arg
#'user/testing-macro-2
user=> (testing-macro-2 "macro 2 test")
nil
(defn testing-macro-3 [my-arg]
(macro3 my-arg))
my-arg
#'user/testing-macro-3
user=> (testing-macro-3 "macro 3 test")
macro 3 test
nil
user=> (macro2 hello)
hello
nil
user=> (macro3 hello)
hello
CompilerException java.lang.RuntimeException: Unable to resolve symbol: hello in this context, compiling:(NO_SOURCE_PATH:107)
Each of the macros still prints the argument due to a println being called when the macro is evaluated, but since macro3 now actually returns source code it actually works like println.
Note that testing-macro-2 prints nothing because macro2 throws away the result of the intermediate calculation `(macro1 ~x) and simply returns nil (the result of println). In other words, using (macro2 foo) is the same as just putting a nil literal in your code, except that the compiler will print foo as a side-effect when it evaluates the macro.
Invoking (macro3 hello) results in a CompilerException because the macro substitution results in the code (println hello), but hello is not defined. If you do something like (def hello "Hello there!") then you won't get an error since it will find a binding for hello.
I'm not satisfied with the answers so far, so let me take a stab...
The issue is that defmacro returns data that is then used as code. Only the last expression in defmacro is returned and, in your example, is then evaluated as code.
So... in your call to macro2 the following steps occur
`(macro1 ~x) is syntax quoted so it evaluates to (macro1 hello) and is not evaluated further because of the syntax quote. This line effectively does nothing as a result.
(func1 x) executes inside the macro, prints the string, and the result nil is returned.
The result of calling (macro2 hello) is nil so nothing further occurs.
Later, you call macro3 and the following steps occur
(func1 x) executes, prints hello, returns nil. But since the macro is not finished this nil does nothing and the next expression is evaluated.
`(macro1 ~x) evaluates (just as before) to (macro1 hello) and is returned as code (it is not evaluated further, yet, because it is syntax quoted). This is the return value of calling macro3 since it is the last expression in the implicit do of defmacro.
The return value of calling macro3 from the previous step is now evaluated. It evaluates to (println hello), but this is not evaluated yet.
Finally the previous steps result is evaluated and a second hello is printed.
I believe the point to understand is that a macro returns code to be executed and only the last expression is returned. Perhaps also, remember that the syntax quote ` prevents the expression following it from being evaluated and instead creates a list in place that is not evaluated further (unless some other action is taken to evaluate the expression).
Yes there is a runtime/compile time difference between when code is evaluated, but for this question I don't think that is the key thing to notice. The key thing to notice is that with macro2 the result of calling macro1 is not returned, but in macro3 it is returned and is therefore evaluated further.