In Clojure, you need to use gensym to create symbols for internal use in your macros to keep them hygienic. However, sometimes you need to use the same symbol in nested syntax-quotes. For example, if I want to bind a value to a symbol with let and print it three times in an unrolled loop, I'd do
`(let [x# 1]
~#(repeat 3
`(println x#)))
But that would produce
(clojure.core/let [x__2__auto__ 1]
(clojure.core/println x__1__auto__)
(clojure.core/println x__1__auto__)
(clojure.core/println x__1__auto__))
x# generates a different symbol in the let form than in the println forms nested within it - because they were created from different syntax-quotes.
To solve it, I can generate the symbol beforehand and inject it to the syntax-quotes:
(let [x (gensym)]
`(let [~x 1]
~#(repeat 3
`(println ~x)))
)
This will produce the correct result, with the same symbol everywhere needed:
(clojure.core/let [G__7 1]
(clojure.core/println G__7)
(clojure.core/println G__7)
(clojure.core/println G__7))
Now, while it does produce the right result, the code itself looks ugly and verbose. I don't like having to "declare" a symbol, and the injection syntax makes it look like it came from outside the macro, or calculated somewhere within in it. I want to be able to use the auto-gensym syntax, which makes it clear that those are macro-internal symbols.
So, is there any way to use auto-gensym with nested syntax-quotes and make them produce the same symbol?
Auto-gensym'd symbols are only valid within the syntax-quote that defines them and they don't work in unquoted code because that is not part of the syntax quote.
Here the symbol x# gets replaced by it's gensym because it is within the scope of the syntax quote:
core> `(let [x# 1] x#)
(clojure.core/let [x__1942__auto__ 1] x__1942__auto__)
And if you unquote it it no longer gets translated into it's syntax quote:
core> `(let [x# 1] ~#x#)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: x# in this context, compiling:(NO_SOURCE_PATH:1)
Auto-gensyms are a very convenient shortcut within the syntax-quote, everywhere else you apparently need to use gensym directly as is the case with your later example.
there are other ways to structure this macro so autogensyms will work though declaring gensymed symbols in a let at the top of a macro is very normal in Clojure and other lisps as well.
Your method (calling gensym) is the right one.
However in some cases you can get by with a clever use of doto, -> or ->>. See:
`(let [x# 1]
(doto x#
~#(repeat 3 `println)))
More generally you can do the following whenever faced with this situation:
(let [x `x#]
`(let [~x 1]
~#(repeat 3
`(println ~x))))
To be clear, you create the auto-gensym and bind it outside of the syntax-quoted form, and then inside any nested forms which require it you can just use syntax-unquote.
Related
Recently I came across a use for eval within a macro, which I understand is a bit of a faux pas but let's ignore that for now. What I found surprising, was that eval was able to resolve global vars at macroexpansion time. Below is a contrived example, just to illustrate the situation I'm referring to:
(def list-of-things (range 10))
(defmacro force-eval [args]
(apply + (eval args)))
(macroexpand-1 '(force-eval list-of-things))
; => 45
I would have expected args to resolve to the symbol list-of-things inside force-eval, and then list-of-things to be evaluated resulting in an error due to it being unbound:
"unable to resolve symbol list-of-things in this context"
However, instead list-of-things is resolved to (range 10) and no error is thrown - the macroexpansion succeeds.
Contrast this with attempting to perform the same macroexpansion, but within a local binding context:
(defmacro force-eval [args]
(apply + (eval args)))
(let [list-of-things (range 10)]
(macroexpand-1 '(force-eval list-of-things)))
; => Unable to resolve symbol: list-of-thingss in this context
Note in the above examples I'm assuming list-of-things is not previously bound, e.g. a fresh REPL. One final example illustrates why this is important:
(defmacro force-eval [args]
(apply + (eval args)))
(def list-of-things (range 10 20))
(let [list-of-thing (range 10)]
(macroexpand-1 '(force-eval list-of-things)))
; => 145
The above example shows that the locals are ignored, which is expected behavior for eval, but is a bit confusing when you are expecting the global to not be available at macroexpansion time either.
I seem to have a misunderstanding about what exactly is available at macroexpansion time. I had previously thought that essentially any binding, be it global or local, would not be available until runtime. Apparently this is an incorrect assumption. Is the answer to my confusion simply that global vars are available at macroexpansion time? Or am I missing some further nuance here?
Note: this related post closely describes a similar problem, but the focus there is more on how to avoid inappropriate use of eval. I'm mainly interested in understanding why eval works in the first example and by extension what's available to eval at macroexpansion time.
Of course, vars must be visible at compile time. That's where functions like first and + are stored. Without them, you couldn't do anything.
But keep in mind that you have to make sure to refer to them correctly. In the repl, *ns* will be bound, and so a reference to a symbol will look in the current namespace. If you are running a program through -main instead of the repl, *ns* will not be bound, and only properly qualified vars will be found. You can ensure that you qualify them correctly by using
`(force-eval list-of-things)
instead of
'(force-eval list-of-things)
Note I do not distinguish between global vars and non-global vars. All vars in Clojure are global. Local bindings are not called vars. They're called locals, or bindings, or variables, or some combination of those words.
Clojure is designed with an incremental compilation model. This is poorly documented.
In C and other traditional languages, source code must be compiled, then linked with pre-compiled libraries before the final result can be executed. Once execution begins, no changes to the code can occur until the program is terminated, when new source code can be compiled, linked, then executed. Java is normally used in this manner just like C.
With the Clojure REPL, you can start with zero source code in a live executing environment. You can call existing functions like (+ 2 3), or you can define new functions and variables on the fly (both global & local), and redefine existing functions. This is only possible because core Clojure is already available (i.e. clojure.core/+ etc is already "installed"), so you can combine these functions to define your own new functions.
The Clojure "compiler" works just like a giant REPL session. It reads and evaluates forms from your source code files one at a time, incrementally adding them the the global environment. Indeed, it is a design goal/requirement that the result of compiling and executing source code is identical to what would occur if you just pasted each entire source code file into the REPL (in proper dependency order).
Indeed, the simplest mental model for code execution in Clojure is to pretend it is an interpreter instead of a traditional compiler.
And eval in a macro makes no sense.
Because:
a macro already implicitely contains an eval
at the very final step.
If you use macroexpand-1, you make visible how the code was manipulated in the macro before the evocation of the implicite eval inside the macro.
An eval in a macro is an anti-pattern which might indicate that you should use a function instead of a macro - and in your examle this is exactly the case.
So your aim is to dynamically (in run-time) evoke sth in a macro. This you can only do through an eval applied over a macro call OR you should rather use a function.
(defmacro force-eval [args]
(apply + (eval args)))
;; What you actually mean is:
(defn force-eval [args]
(apply + args))
;; because a function in lisp evaluates its arguments
;; - before applying the function body.
;; That means: args in the function body is exactly
;; `(eval args)`!
(def list-of-things (range 10))
(let [lit-of-things (range 10 13)]
(force-eval list-of-things))
;; => 45
;; so this is exactly the behavior you wanted!
The point is, your construct is a "bad" example for a macro.
Because apply is a special function which allows you to
dynamically rearrange function call structures - so it has
some magic of macros inside it - but in run-time.
With apply you can do quite some meta programming in some cases when you just quote some of your input arguments.
(Try (force-eval '(1 2 3)) it returns 6. Because the (1 2 3) is put together with + at its front by apply and then evaluated.)
The second point - I am thinking of this answer I once gave and this to a dynamic macro call problem in Common Lisp.
In short: When you have to control two levels of evaluations inside a macro (often when you want a macro inject some code in runtime into some code), you need too use eval when calling the macro and evaluate those parts in the macro call which then should be processed in the macro.
In Clojure,
(def x 3)
(eval '(prn x))
prints 3, whereas
(let [y 3]
(eval '(prn y)))
and
(binding [z 3] (eval '(prn z)))
generate an 'Unable to resolve var' exception.
According to http://clojure.org/evaluation, eval, load-string, etc generate temporary namespaces to evaluate their contents. Therefore, I'd expect neither of the above code samples to work, since (def x 3) is done in my current namespace, not the one created by eval.
Why does the first code sample work and not the last two?
How can I eval a form with bound variables without using def?
Thanks!
1.:
The reason this doesn't work is (more or less) given on the page you linked:
It is an error if there is no global var named by the symbol […]
And:
[…]
A lookup is done in the current namespace to see if there is a mapping
from the symbol to a var. If so, the
value is the value of the binding of
the var referred-to by the symbol.
It is an error.
eval evaluates forms in an empty (null in CL-lingo) lexical environment. This means, that you cannot access lexical variable bindings from the caller's scope. Also, binding creates new bindings for existing vars, which is why you cannot use it "by itself", without having declared or defed the variables you try to bind. Besides, lexical variables (at least in CL, but I would be surprised if this wasn't the case for Clojure) already ceased to exist at runtime – They are translated to addresses or values.
See also my older post about this topic.
2.:
So, you have to use dynamic variables. You can avoid the explicit def, but you still at least need to declare them (which defs var names without bindings):
user=> (declare ^:dynamic x)
#'user/x
user=> (binding [x 10] (eval '(prn x)))
10
nil
By the way: I suppose you know why you need eval, and that its use is considered evil when other solutions would be appropriate.
I'm trying to find a way to thread a value through a list of functions.
Firstly, I had a usual ring-based code:
(defn make-handler [routes]
(-> routes
(wrap-json-body)
(wrap-cors)
;; and so on
))
But this was not optimal as I wanted to write a test to check the routes are actually wrapped with wrap-cors. I decided to extract the wrappers into a def. So the code became as follows:
(def middleware
(list ('wrap-json-body)
('wrap-cors)
;; and so on
))
(defn make-handler [routes]
(-> routes middleware))
This apparently doesn't work and is not supposed to as the -> macro doesn't take a list as the second argument. So I tried to use the apply function to resolve that:
(defn make-handler [routes]
(apply -> routes middleware))
Which eventually bailed out with:
CompilerException java.lang.RuntimeException: Can't take value of a
macro: #'clojure.core/->
So the question arises: How does one pass a list of values to the -> macro (or, say, any other macro) as one would do with apply for a function?
This is an XY Problem.
The main point of -> is to make code easier to read. But if one writes a new macro solely in order to use -> (in code nobody will ever see because it exists only at macro-expansion), it seems to me that this is doing a lot of work for no benefit. Moreover, I believe it obscures, rather than clarifies, the code.
So, in the spirit of never using a macro where functions will do, I suggest the following two equivalent solutions:
Solution 1
(reduce #(%2 %) routes middleware)
Solution 2
((apply comp middleware) routes)
A Better Way
The second solution is easily simplified by changing the definition of middleware from being a list of the functions to being the composition of the functions:
(def middleware
(comp wrap-json-body
wrap-cors
;; and so on
))
(middleware routes)
When I began learning Clojure, I ran across this pattern often enough that many of my early projects have an freduce defined in core:
(defn freduce
"Given an initial input and a collection of functions (f1,..,fn),
This is logically equivalent to ((comp fn ... f1) input)."
[in fs]
(reduce #(%2 %) in fs))
This is totally unnecessary, and some might prefer the direct use of reduce as being more clear. However, if you don't like staring at #(%2 %) in your application code, adding another utility word to your language is fine.
you can make a macro for that:
;; notice that it is better to use a back quote, to qoute function names for macro, as it fully qualifies them.
(def middleware
`((wrap-json-body)
(wrap-cors))
;; and so on
)
(defmacro with-middleware [routes]
`(-> ~routes ~#middleware))
for example this:
(with-middleware [1 2 3])
would expand to this:
(-> [1 2 3] (wrap-json-body) (wrap-cors))
Occasionally when looking at other people's Clojure code, I see a function defined via defn and then called using the var-quote syntax, e.g.:
user> (defn a [] 1)
#'user/a
user> (a) ; This is how you normally call a function
1
user> (#'a) ; This uses the var-quote syntax and produces the same result
1
For the life of me I can't figure out the difference between these two ways of calling a function. I can't find anything in the evaluation documentation to say what happens when the operator of a call is a var that might suggest why the second form would be preferred. They both seem to respond in the same to binding assignments and syntax-quoting.
So, can somebody please provide a code sample that will illustrate the difference between (a) and (#'a) above?
Edit: I know that var-quote can be used to get to a var that's shadowed by a let lexical binding, but that doesn't seem to be the case in the code that I'm looking at.
(#'a) always refers to the var a, while (a) can be shadowed by local bindings:
user> (defn a [] 1)
#'user/a
user> (let [a (fn [] "booh")] [(a) (#'a)])
["booh" 1]
But most actual uses of var-quote / function call are not calling the var-quote expression directly, but instead cache its value so that higher-order constructs refer to the current value of var a instead of its value when passed in:
(defn a [] 1)
(defn my-call [f] (fn [] (+ 1 (f))))
(def one (my-call a))
(def two (my-call #'a))
(defn a [] 2)
user> (one)
2
user> (two)
3
This is mostly useful for interactive development, where you're changing some function that gets wrapped in a bunch of other functions in other packages.
The second form allows you to circumvent the privacy restrictions that clojure puts in place.
So, for instance, if you develop a library with private functions, but want to test them from a separate namespace, you cannot refer to them directly. But you can get to them using the var quote syntax. It's very useful for this.
Privacy is clojure is, in essence, a form of automatic documentation, as opposed to the privacy you see in Java. You can get around it.
user> (defn- a [] 1)
#'user/a
user> (ns user2)
nil
user2> (user/a)
CompilerException java.lang.IllegalStateException: var: #'user/a is not public, compiling:(NO_SOURCE_PATH:1)
user2> (#'user/a)
1
user2>
This is similar to the problem discussed in Treat Clojure macro as a function but when trying the approach in the top answer, I got an error. Hopefully too much information about my specific application is not necessary, because it is quite complicated, but here is a distilled version of what I tried to do:
(defmacro make-fn [m arg1]
`(fn [& args#]
(eval `(~'~m ~'~arg1 ~#args#))))
I used the macro in this context:
(let [columns (make-columns table-width)
table-name (keyword (str "table_" n))]
(apply (make-fn helpers/tbl table-name) columns))
"helpers/tbl" is a macro that expects a table name keyword and a variable number of lists containing column specifications (like [:varchar 100] or something). I am trying to create random database table specifications on the fly to facilitate some testing. Anyway, when trying to execute the above code, I get the following error:
CompilerException java.lang.RuntimeException: Unable to resolve symbol: table-name in this context, compiling:(NO_SOURCE_PATH:1)
I sort of grasp the problem: macro expansion is done at compile-time, and I am trying to include a runtime value in the macro expansion, hence the odd use of quoting and unquoting to get everything set up just right. I basically want a partial for macros, and I need to be able to reuse this mechanism for different macros in different namespaces, and have all of the variable resolution come out right. Is this even possible?
The problem is caused by the way Clojure resolves symbols within a syntax-quote (backtick) expression. To avoid unintentional variable capture, Clojure always interprets symbols within a syntax-quote expression as referring to Vars (not locals).
You can get around this by "rolling your own" form-building code, equivalent to that generated by syntax-quote. It's as ugly as sin, but it works... just don't say I didn't warn you:
(defmacro make-fn [m arg1]
(let [g (gensym)]
(list 'fn ['& g]
(list 'eval (list 'concat (list 'list m arg1) g)))))
Wow, this is like a flashback to my Common Lisp days...