This is similar to the problem discussed in Treat Clojure macro as a function but when trying the approach in the top answer, I got an error. Hopefully too much information about my specific application is not necessary, because it is quite complicated, but here is a distilled version of what I tried to do:
(defmacro make-fn [m arg1]
`(fn [& args#]
(eval `(~'~m ~'~arg1 ~#args#))))
I used the macro in this context:
(let [columns (make-columns table-width)
table-name (keyword (str "table_" n))]
(apply (make-fn helpers/tbl table-name) columns))
"helpers/tbl" is a macro that expects a table name keyword and a variable number of lists containing column specifications (like [:varchar 100] or something). I am trying to create random database table specifications on the fly to facilitate some testing. Anyway, when trying to execute the above code, I get the following error:
CompilerException java.lang.RuntimeException: Unable to resolve symbol: table-name in this context, compiling:(NO_SOURCE_PATH:1)
I sort of grasp the problem: macro expansion is done at compile-time, and I am trying to include a runtime value in the macro expansion, hence the odd use of quoting and unquoting to get everything set up just right. I basically want a partial for macros, and I need to be able to reuse this mechanism for different macros in different namespaces, and have all of the variable resolution come out right. Is this even possible?
The problem is caused by the way Clojure resolves symbols within a syntax-quote (backtick) expression. To avoid unintentional variable capture, Clojure always interprets symbols within a syntax-quote expression as referring to Vars (not locals).
You can get around this by "rolling your own" form-building code, equivalent to that generated by syntax-quote. It's as ugly as sin, but it works... just don't say I didn't warn you:
(defmacro make-fn [m arg1]
(let [g (gensym)]
(list 'fn ['& g]
(list 'eval (list 'concat (list 'list m arg1) g)))))
Wow, this is like a flashback to my Common Lisp days...
Related
Recently I came across a use for eval within a macro, which I understand is a bit of a faux pas but let's ignore that for now. What I found surprising, was that eval was able to resolve global vars at macroexpansion time. Below is a contrived example, just to illustrate the situation I'm referring to:
(def list-of-things (range 10))
(defmacro force-eval [args]
(apply + (eval args)))
(macroexpand-1 '(force-eval list-of-things))
; => 45
I would have expected args to resolve to the symbol list-of-things inside force-eval, and then list-of-things to be evaluated resulting in an error due to it being unbound:
"unable to resolve symbol list-of-things in this context"
However, instead list-of-things is resolved to (range 10) and no error is thrown - the macroexpansion succeeds.
Contrast this with attempting to perform the same macroexpansion, but within a local binding context:
(defmacro force-eval [args]
(apply + (eval args)))
(let [list-of-things (range 10)]
(macroexpand-1 '(force-eval list-of-things)))
; => Unable to resolve symbol: list-of-thingss in this context
Note in the above examples I'm assuming list-of-things is not previously bound, e.g. a fresh REPL. One final example illustrates why this is important:
(defmacro force-eval [args]
(apply + (eval args)))
(def list-of-things (range 10 20))
(let [list-of-thing (range 10)]
(macroexpand-1 '(force-eval list-of-things)))
; => 145
The above example shows that the locals are ignored, which is expected behavior for eval, but is a bit confusing when you are expecting the global to not be available at macroexpansion time either.
I seem to have a misunderstanding about what exactly is available at macroexpansion time. I had previously thought that essentially any binding, be it global or local, would not be available until runtime. Apparently this is an incorrect assumption. Is the answer to my confusion simply that global vars are available at macroexpansion time? Or am I missing some further nuance here?
Note: this related post closely describes a similar problem, but the focus there is more on how to avoid inappropriate use of eval. I'm mainly interested in understanding why eval works in the first example and by extension what's available to eval at macroexpansion time.
Of course, vars must be visible at compile time. That's where functions like first and + are stored. Without them, you couldn't do anything.
But keep in mind that you have to make sure to refer to them correctly. In the repl, *ns* will be bound, and so a reference to a symbol will look in the current namespace. If you are running a program through -main instead of the repl, *ns* will not be bound, and only properly qualified vars will be found. You can ensure that you qualify them correctly by using
`(force-eval list-of-things)
instead of
'(force-eval list-of-things)
Note I do not distinguish between global vars and non-global vars. All vars in Clojure are global. Local bindings are not called vars. They're called locals, or bindings, or variables, or some combination of those words.
Clojure is designed with an incremental compilation model. This is poorly documented.
In C and other traditional languages, source code must be compiled, then linked with pre-compiled libraries before the final result can be executed. Once execution begins, no changes to the code can occur until the program is terminated, when new source code can be compiled, linked, then executed. Java is normally used in this manner just like C.
With the Clojure REPL, you can start with zero source code in a live executing environment. You can call existing functions like (+ 2 3), or you can define new functions and variables on the fly (both global & local), and redefine existing functions. This is only possible because core Clojure is already available (i.e. clojure.core/+ etc is already "installed"), so you can combine these functions to define your own new functions.
The Clojure "compiler" works just like a giant REPL session. It reads and evaluates forms from your source code files one at a time, incrementally adding them the the global environment. Indeed, it is a design goal/requirement that the result of compiling and executing source code is identical to what would occur if you just pasted each entire source code file into the REPL (in proper dependency order).
Indeed, the simplest mental model for code execution in Clojure is to pretend it is an interpreter instead of a traditional compiler.
And eval in a macro makes no sense.
Because:
a macro already implicitely contains an eval
at the very final step.
If you use macroexpand-1, you make visible how the code was manipulated in the macro before the evocation of the implicite eval inside the macro.
An eval in a macro is an anti-pattern which might indicate that you should use a function instead of a macro - and in your examle this is exactly the case.
So your aim is to dynamically (in run-time) evoke sth in a macro. This you can only do through an eval applied over a macro call OR you should rather use a function.
(defmacro force-eval [args]
(apply + (eval args)))
;; What you actually mean is:
(defn force-eval [args]
(apply + args))
;; because a function in lisp evaluates its arguments
;; - before applying the function body.
;; That means: args in the function body is exactly
;; `(eval args)`!
(def list-of-things (range 10))
(let [lit-of-things (range 10 13)]
(force-eval list-of-things))
;; => 45
;; so this is exactly the behavior you wanted!
The point is, your construct is a "bad" example for a macro.
Because apply is a special function which allows you to
dynamically rearrange function call structures - so it has
some magic of macros inside it - but in run-time.
With apply you can do quite some meta programming in some cases when you just quote some of your input arguments.
(Try (force-eval '(1 2 3)) it returns 6. Because the (1 2 3) is put together with + at its front by apply and then evaluated.)
The second point - I am thinking of this answer I once gave and this to a dynamic macro call problem in Common Lisp.
In short: When you have to control two levels of evaluations inside a macro (often when you want a macro inject some code in runtime into some code), you need too use eval when calling the macro and evaluate those parts in the macro call which then should be processed in the macro.
I'm starting to learn clojure and I've stumbled upon the following, when I found myself declaring a "sum" function (for learning purposes) I wrote the following code
(def sum (fn [& args] (apply + args)))
I have understood that I defined the symbol sum as containing that fn, but why do I have to enclose the Fn in parenthesis, isn't the compiler calling that function upon definition instead of when someone is actually invoking it? Maybe it's just my imperative brain talking.
Also, what are the use cases of let? Sometimes I stumble on code that use it and other code that don't, for example on the Clojure site there's an exercise to use the OpenStream function from the Java Interop, I wrote the following code:
(defn http-get
[url]
(let [url-obj (java.net.URL. url)]
(slurp (.openStream url-obj))))
(http-get "https://www.google.com")
whilst they wrote the following on the clojure site as an answer
(defn http-get [url]
(slurp
(.openStream
(java.net.URL. url))))
Again maybe it's just my imperative brain talking, the need of having a "variable" or an "object" to store something before using it, but I quite don't understand when I should use let or when I shouldn't.
To answer both of your questions:
1.
(def sum (fn [& args] (apply + args)))
Using def here is very unorthodox. When you define a function you usually want to use defn. But since you used def you should know that def binds a name to a value. fn's return value is a function. Effectively you bound the name sum to the function returned by applying (using parenthesis which are used for application) fn.
You could have used the more traditional (defn sum [& args] (apply + args))
2.
While using let sometimes makes sense for readability (separating steps outside their nested use) it is sometimes required when you want to do something once and use it multiple times. It binds the result to a name within a specified context.
We can look at the following example and see that without let it becomes harder to write (function is for demonstration purposes):
(let [db-results (query "select * from table")] ;; note: query is not a pure function
;; do stuff with db-results
(f db-results)
;; return db-results
db-results)))
This simply re-uses a return value (db-results) from a function that you usually only want to run once - in multiple locations. So let can be used for style like the example you've given, but its also very useful for value reuse within some context.
Both def and defn define a global symbol, sort of like a global variable in Java, etc. Also, (defn xxx ...) is a (very common) shortcut for (def xxx (fn ...)). So, both versions will work exactly the same way when you run the program. Since the defn version is shorter and more explicit, that is what you will do 99% of the time.
Typing (let [xxx ...] ...) defines a local symbol, which cannot be seen by code outside of the let form, just like a local variable (block-scope) in Java, etc.
Just like Java, it is optional when to have a local variable like url-obj. It will make no difference to the running program. You must answer the question, "Which version makes my code easier to read and understand?" This part is no different than Java.
I'm trying to find a way to thread a value through a list of functions.
Firstly, I had a usual ring-based code:
(defn make-handler [routes]
(-> routes
(wrap-json-body)
(wrap-cors)
;; and so on
))
But this was not optimal as I wanted to write a test to check the routes are actually wrapped with wrap-cors. I decided to extract the wrappers into a def. So the code became as follows:
(def middleware
(list ('wrap-json-body)
('wrap-cors)
;; and so on
))
(defn make-handler [routes]
(-> routes middleware))
This apparently doesn't work and is not supposed to as the -> macro doesn't take a list as the second argument. So I tried to use the apply function to resolve that:
(defn make-handler [routes]
(apply -> routes middleware))
Which eventually bailed out with:
CompilerException java.lang.RuntimeException: Can't take value of a
macro: #'clojure.core/->
So the question arises: How does one pass a list of values to the -> macro (or, say, any other macro) as one would do with apply for a function?
This is an XY Problem.
The main point of -> is to make code easier to read. But if one writes a new macro solely in order to use -> (in code nobody will ever see because it exists only at macro-expansion), it seems to me that this is doing a lot of work for no benefit. Moreover, I believe it obscures, rather than clarifies, the code.
So, in the spirit of never using a macro where functions will do, I suggest the following two equivalent solutions:
Solution 1
(reduce #(%2 %) routes middleware)
Solution 2
((apply comp middleware) routes)
A Better Way
The second solution is easily simplified by changing the definition of middleware from being a list of the functions to being the composition of the functions:
(def middleware
(comp wrap-json-body
wrap-cors
;; and so on
))
(middleware routes)
When I began learning Clojure, I ran across this pattern often enough that many of my early projects have an freduce defined in core:
(defn freduce
"Given an initial input and a collection of functions (f1,..,fn),
This is logically equivalent to ((comp fn ... f1) input)."
[in fs]
(reduce #(%2 %) in fs))
This is totally unnecessary, and some might prefer the direct use of reduce as being more clear. However, if you don't like staring at #(%2 %) in your application code, adding another utility word to your language is fine.
you can make a macro for that:
;; notice that it is better to use a back quote, to qoute function names for macro, as it fully qualifies them.
(def middleware
`((wrap-json-body)
(wrap-cors))
;; and so on
)
(defmacro with-middleware [routes]
`(-> ~routes ~#middleware))
for example this:
(with-middleware [1 2 3])
would expand to this:
(-> [1 2 3] (wrap-json-body) (wrap-cors))
In Clojure, you need to use gensym to create symbols for internal use in your macros to keep them hygienic. However, sometimes you need to use the same symbol in nested syntax-quotes. For example, if I want to bind a value to a symbol with let and print it three times in an unrolled loop, I'd do
`(let [x# 1]
~#(repeat 3
`(println x#)))
But that would produce
(clojure.core/let [x__2__auto__ 1]
(clojure.core/println x__1__auto__)
(clojure.core/println x__1__auto__)
(clojure.core/println x__1__auto__))
x# generates a different symbol in the let form than in the println forms nested within it - because they were created from different syntax-quotes.
To solve it, I can generate the symbol beforehand and inject it to the syntax-quotes:
(let [x (gensym)]
`(let [~x 1]
~#(repeat 3
`(println ~x)))
)
This will produce the correct result, with the same symbol everywhere needed:
(clojure.core/let [G__7 1]
(clojure.core/println G__7)
(clojure.core/println G__7)
(clojure.core/println G__7))
Now, while it does produce the right result, the code itself looks ugly and verbose. I don't like having to "declare" a symbol, and the injection syntax makes it look like it came from outside the macro, or calculated somewhere within in it. I want to be able to use the auto-gensym syntax, which makes it clear that those are macro-internal symbols.
So, is there any way to use auto-gensym with nested syntax-quotes and make them produce the same symbol?
Auto-gensym'd symbols are only valid within the syntax-quote that defines them and they don't work in unquoted code because that is not part of the syntax quote.
Here the symbol x# gets replaced by it's gensym because it is within the scope of the syntax quote:
core> `(let [x# 1] x#)
(clojure.core/let [x__1942__auto__ 1] x__1942__auto__)
And if you unquote it it no longer gets translated into it's syntax quote:
core> `(let [x# 1] ~#x#)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: x# in this context, compiling:(NO_SOURCE_PATH:1)
Auto-gensyms are a very convenient shortcut within the syntax-quote, everywhere else you apparently need to use gensym directly as is the case with your later example.
there are other ways to structure this macro so autogensyms will work though declaring gensymed symbols in a let at the top of a macro is very normal in Clojure and other lisps as well.
Your method (calling gensym) is the right one.
However in some cases you can get by with a clever use of doto, -> or ->>. See:
`(let [x# 1]
(doto x#
~#(repeat 3 `println)))
More generally you can do the following whenever faced with this situation:
(let [x `x#]
`(let [~x 1]
~#(repeat 3
`(println ~x))))
To be clear, you create the auto-gensym and bind it outside of the syntax-quoted form, and then inside any nested forms which require it you can just use syntax-unquote.
At a conceptual level a macro in LISP (and dialects) take a piece of code (as list) and returns another piece of code (again as list).
Based on above principle a simple macro could be:
(defmacro zz [a] (list print a))
;macroexpand says : (#<core$print clojure.core$print#749436> "Hello")
But in clojure this can also be written as:
(defmacro zz [a] `(print ~a))
;macroexpand says : (clojure.core/print "Hello")
I am not exactly sure about the difference here and which should be the preferred way. The first one look simple as we are supposed to return list and avoid using weird characters like back tick.
No one has pointed this out yet...the difference between your 2 macros is this: your second form (using backtick)
(defmacro zz [a] `(print ~a))
is equivalent to:
(defmacro zz [a] (list 'print a))
Which is different from your first example:
(defmacro zz [a] (list print a))
Note the missing single quote -- that is why your macroexpand is different. I agree with the other people posting: using backquote is more conventional if your macro has a fairly simple 'shape'. If you have to do code walking or dynamic construction (i.e. a complex macro), then using lists and building it up is often what's done.
I hope this explanation makes sense.
Constructing lists explicitly is "simplest", in a way, because there are few core concepts you need to know: just accept a list and change it around till you have a new list. Backtick is a convenient shortcut for "templating" chunks of code; it is possible to write any macro without it, but for any large macro it quickly becomes very unpleasant. For example, consider two ways of writing let as a macro over fn:
(defmacro let [bindings & body]
(let [names (take-nth 2 bindings)
vals (take-nth 2 (rest bindings))]
`((fn [~#names]
(do ~#body))
~#vals)))
(defmacro let [bindings & body]
(let [names (take-nth 2 bindings)
vals (take-nth 2 (rest bindings))]
(cons (list `fn (vec names) (cons `do body))
vals)))
In the first case, using backtick makes it fairly clear that you're writing a function of the names containing the body, and then calling it with the values - the macro code is "shaped" the same as the expansion code, so you can imagine what it will look like.
In the second case, with just cons and list all over the place, it is a real headache to work out what the expansion will look like. This isn't always the case, of course: sometimes it can be clearer to write something without a backtick.
Another very important point was made by Kyle Burton: print is not the same as 'print! Your macro expansion should contain the symbol print, not its value (which is a function). Embedding objects (such as functions) in code is very fragile and only works by accident. So make sure your macros expand to code you could actually have written yourself, and let the evaluation system do the hard work - you could type in the symbol print, but you couldn't type in a pointer to the current value of the function print.
There's a style difference between them. Your example is very simple but in more complex macros the difference will be bigger.
For example the unless macro as defined in "The Joy of Clojure" book:
(defmacro unless [condition & body]
`(if (not ~condition)
(do ~#body)))
From the book:
Syntax-quote allows the following if-form to act as a sort of template for the expression
that any use of the macro become when it is expanded.
When creating a macro always choose the most readable and idiomatic style.
To contrast, the above code can equivalently be written like so:
(defmacro unless [condition & body]
(list 'if (list 'not condition)
(list* 'do body)))
In my experience they are equivalent. Though there may be some edge cases I'm not aware of.
#islon 's example can equivalently be written as:
To contrast, the above code can equivalently be written like so:
(defmacro unless [condition & body]
(list 'if (list 'not condition)
(list* 'do body)))