This is not my 'production code' but a simplication of the problem for illustration purposes. Also, the title of this question is misleading because it brings to mind the ~# expansion, which I understand, and which may not necessarily be the problem. Please suggest a better question title if you can.
Given a macro with the following form:
(defmacro my-add [x & ys] `(+ ~x ~#ys))
Now let's say we have a list:
(def my-lst '(2 3))
Now I want a function that uses my-add that I can pass my-lst to as an arg, i.e.
(call-my-add 1 my-lst)
I define the function in what would seem to be the obvious way:
(defn call-my-add [x ys]
(apply my-add (cons x ys)))
But:
java.lang.Exception: Can't take value of a macro: #'user/call-my-add (repl-1:60)
I've tried all sorts of wild tricks to get the call-my-add function to work using evals, applies, and even defining call-my-add as a macro, but they all give similar ClassCastExceptions.
Is there any way out of this?
No. Macros do not, cannot, will never, have access to the actual run-time values contained in their arguments, so cannot splice them into the expansion. All they get is the symbol(s) you pass to them, in this case my-list. The "way around this" is to define my-add as a function, and then (optionally) have a macro that calls that function in order to generate its code.
I wrote a blog post about this semi-recently that you might find enlightening.
You could do it with evals if you wanted to, but that is a terrible idea in almost every case:
(let [my-list '(1 2)]
(eval `(my-add 5 ~#my-list)))
What a great example showing that macros are not first class citizens in Clojure (or any Lisp that I know of). They cannot be applied to functions, stored in containers, or passed to functions etc. In exchange for this they get to control when and if their arguments are evaluated.
What happens at macro expansion time must stay in macro expansion time. So if my-add is being evaluated at macro expansion time and you want to use apply then you need... another macro; to do the applying.
(defmacro call-my-add [x ys]
`(my-add ~#(cons x ys)))
Macros are somewhat contagious in this way.
PS: I'm not at my repl so please edit if you see a bug in this example (or I'll fix it when I get back)
Related
I'm playing around with clojure macros and I'm finding that a lot of macro behavior I can just replicate with function composition.
A good example of this is the threading macro:
(defn add1 [n] (+ n 1))
(defn mult10 [n] (* n 10))
(defn threadline [arg]
(-> arg
add1
mult10))
I can replicate this easily with a higher order function like pipe:
(defn pipe [& fns]
(reduce (fn [f g] (fn [arg] (g(f arg)))) fns))
(def pipeline
(pipe
#(+ % 1)
#(* % 10)))
There must be instances where a macro can not be replaced by a function. I was wondering if someone had some good examples of these sorts of situations, and the reoccurring themes involved.
One important advantage of macros is their ability to transform code at compile-time without evaluating any of it. Macros receive code as data during compilation, but functions receive values at run-time. Macros allow you to extend the compiler in a sense.
For example, Clojure's and and or are implemented as recursive macros that expand into nested if forms. This allows lazy evaluation of the and/or's inner forms i.e. if the first or form is truthy, its value will be returned and none of the others will be evaluated. If you wrote and/or as a function, all its arguments would be evaluated before they could be examined.
Short-circuiting control flow isn't an issue in your pipe function example, but pipe adds considerable run-time complexity compared to -> which simply unrolls to nested forms. A more interesting macro to try to implement as a function might be some->.
I'm finding that a lot of macro behavior I can just replicate with function composition
If your functions are amenable to it, you can certainly replace a simple threading macro with function composition with comp, similar to "point free" style in other functional languages: #(-> % inc str) is functionally equivalent to (comp str inc) and #(str (inc %)).
It's generally advised to prefer functions when possible, and even when writing a macro you can usually farm out most of the "work" to function(s).
The first macro I ever learned is a good example of a macro that can't be written as a plain function:
(defmacro infix [[arg1 f arg2]]
(list f arg1 arg2))
(infix (1 + 2))
=> 3
Of course this exact macro would never be used in the wild, but it sets the stage for more useful macros that act as readability helpers. It should also be noted that while you can replicate a lot of basic macro's behavior with plain functions, should you? It would be hard to argue that your pipe example leads to easier to read/write code than, say, as->.
The "reoccurring themes" you're looking for are cases where you're manipulating data at compile-time ("data" being the code itself), not run-time. Any case that requires the function to take its argument unevaluated must be a macro. You can partially "cheat" in some cases and just wrap the code in a function to delay evaluation, but that doesn't work for all cases (like the infix example).
Macros are not interchangeable with functions, but you examples are:
(macroexpand '#(+ % 1))
; ==> (fn* [p1__1#] (+ p1__1# 1))
The reason why it works is because the argument expect a function and you use a macro that becomes a function. However I know that cond is a macro. It cannot be replaced with a function implementation since the arguments of a function gets evaluated and the whole point of cond is to only evaluate some parts of the arguments in a specific order based on evaluation of their predicates. eg. making a recursive function with that would never terminate since the default case will also always be called before the body of the function cond is evaluated.
The whole point of macros is to expand the lamguage and since the evaluation is controlled by the result you can make all sorts of new features that would be impossible with function except if one passed all arguments as functions to delay evaluation.
In any language, macros -- compile-time functions from code to code -- let you do three things:
Define new binding forms (e.g. Clojure's destructuring let).
Change the order of evaluation (e.g. or, and).
Present a domain-specific language (e.g. Instaparse).
You can debate 3 -- whether implementing DSLs truly requires macros. Sure you can do code generators that are functions from text files to text files. Or, sure you can do Ruby style runtime DSLs. But if you want a DSL that's integrated into the compiler at compile-time, then macros are effectively your "compiler API".
Having said that, it makes sense to use macros only for these special purposes. Use functions and/or data-driven code as much as possible. Including to do work behind the "facade" provide by a macro.
The two big things macros do is control evaluation of their arguments and transform code at compile time. You can do both with functions by requiring calling code to quote their arguments.
For instance, you could write a version of defn that is called this way:
(defn 'name '[arg1 arg2]
'(expression1)
'(expression2)
'etc)
Then you could eval arguments at will, evaluating them or not, changing the order of execution, or altering forms before you evaluate them, exactly the things macros are good for.
What macros can do that functions can't is gain this ability without any cooperation from the calling code. Users can call macros as if they were plain functions and don't have to treat their arguments any differently.
That's how macros allow you to extend the language: you don't have to treat macro code any differently than regular code, unlike, say, JavaScript, Ruby, or Python, where the language can only be extended with new control flow constructs by doing what you've done in your example, wrapping code in a block, lambda, or function.
I've been into Clojure lately and have avoided macros up until now, so this is my first exposure to them. I've been reading "Mastering Clojure Macros", and on Chapter 3, page 28, I encountered the following example:
user=> (defmacro square [x] `(* ~x ~x))
;=> #'user/square
user=> (map (fn [n] (square n)) (range 10))
;=> (0 1 4 9 16 25 36 49 64 81)
The context is the author is explaining that while simply passing the square macro to map results in an error (can't take value of a macro), wrapping it in a function works because:
when the anonymous function (fn [n] (square n)) gets compiled, the
square expression gets macroexpanded, to (fn [n] (clojure.core/* n
n)). And this is a perfectly reasonable function, so we don’t have any
problems with the compiler
This makes sense to me if we assume the body of the function is evaluated before runtime (at compile, or "definition" time) thus expanding the macro ahead of runtime. However, I always thought that function bodys were not evaluated until runtime, and at compile time you would basically just have a function object with some knowledge of it's lexical scope (but no knowledge of its body).
I'm clearly mixed up on the compile/runtime semantics here, but when I look at this sample I keep thinking that square won't be expanded until map forces it's call, since it's in the body of the anonymous function, which I thought would be unevaluated until runtime. I know my thinking is wrong, because if that was the case, then n would be bound to each number in (range 10), and there wouldn't be an issue.
I know it's a pretty basic question, but macros are proving to be pretty tricky for me to fully wrap my head around at first exposure!
Generally speaking function bodies aren't evaluated at compile time, but macros are always evaluated at compile time because they're always expanded at compile time whether inside a function or not.
You can write a macro that expands to a function, but you still can't refer/pass the macro around as if it were a function:
(defmacro inc-macro [] `(fn [x#] (inc x#)))
=> #'user/inc-macro
(map (inc-macro) [1 2 3])
=> (2 3 4)
defmacro is expanded at compile time, so you can think of it as a function executed during compilation. This will replace every occurrence of the macro "call" with the code it "returns".
You may consider a macros as a syntax rule. For example, in Scheme, a macros is declared with define-syntax form. There is a special step in compiler that substitutes all the macros calls into their content before compiling the code. As a result, there won't be any square calls in your final code. Say, if you wrote something like
(def value (square 3))
the final version after expansion would be
(def value (clojure.core/* 3 3))
There is a special way to check what will be the body of your macros after being expanded:
user=> (defmacro square [x] `(* ~x ~x))
#'user/square
user=> (macroexpand '(square 3))
(clojure.core/* 3 3)
That's why a macros is an ephemeral thing that lives only in source code but not in the compiled version of it. That's why it cannot be passed as a value or referenced somehow.
The best rule regarding macroses is: try to avoid them until you really need them in your work.
I want to write functions without think of the order of function declarations, I don't want to use declare function because I need to declare all function names which I don't want to do it.
I want some macro or some function that does the magic for me. Long story short, I need to write functions like Java(method declaration order does not matter )
One of the best things I like most about functional programming is, that its flow of writing is the same as the flow of thinking. It's like peeling an onion, and at every moment, I only need to concentrate on working on this single layer, and take the inner part for granted. And don't worry about function names, foo and bar would be fine at first. In this style of writing, functions are defined and implemented from the end of the source file back to the top. In cases when one function calls multiple other functions, this linear structure becomes a tree-like structure, but there is always a single point in file to insert new functions. No choices and worries.
Yes, there are times when we need to work on some code snippets with no other dependencies. They can be put at the top of the source file of course.
To answer the question upfront, macros are not magic. If there exists such a macro, this macro will need to take the whole source file as input, analyze the dependency between each code blocks, and re-flow them in the correct order. The analysis of dependency is non-trivial because of lexical scoping. It's almost like writing a compiler. I don't believe such macro exists (has been written by anyone), and the goods it can do is, well to me, not so big.
It is kind of unneeded, but easily possible as an exercise (up to some point). You can write macro, that would forward declare all the top level functions wrapped in it:
(defmacro with-forward-declaration [& body]
(let [names (keep #(when (#{'defn 'defn-} (first %))
(if (map? (second %)) (nth % 2) (second %)))
body)]
`(do (declare ~#names)
~#body)))
that is how it is expanded:
(with-forward-declaration
(defn ^long ff1 [x] (* 2 (ff2)))
(println "just to be sure it doesn't eat other forms")
(defn ^{:name 'xx} ff2 [] (+ 10 (ff3)))
(defn ff3 [] 101)
(defn- ff4 [] :x))
would do the following:
(do
(declare ff1 ff2 ff3 ff4)
(defn ff1 [x] (* 2 (ff2)))
(println "just to be sure it doesn't eat other forms")
(defn ff2 [] (+ 10 (ff3)))
(defn ff3 [] 101)
(defn- ff4 [] :x))
so if you wrap all your namespace's code into this macro, it would predeclare all the functions for you. If you want to go deeper, like predeclaring all some possible defns that are not in the top level, you could update this toy macro by using clojure.walk to find these inner forms.. But i guess this one is just enough, to play with clojure quickly, without thinking about functions' order. I wouldn't do that in production though (at least not without the heavy testing).
I read that data > functions > macros
Say you want to evaluate code in a postfix fashion.
Which approach would be better?
;; Macro
(defmacro reverse-fn [expression]
(conj (butlast expression) (last expression)))
(reverse-fn ("hello world" println))
; => "hello world"
;; Function and data
(def data ["hello world" println])
(defn reverse-fn [data]
(apply (eval (last data)) (butlast data)))
(reverse-fn ["hello world" println])
; => "hello world"
Thanks!
If you require different evaluation behavior for any kind of data in your code macros are your best choice because they can transform unevaluated data at compile time to the code you'd like to be evaluated instead.
Clojure has a programmatic macro system which allows the compiler to be extended by user code. Macros can be used to define syntactic constructs which would require primitives or built-in support in other languages. (http://clojure.org/macros)
The example you provide has specifically that requirement ("evaluate code in a postfix fashion") which is why a macro is the correct choice.
Your macro is better than your function: a macro is better than a function employing eval.
However, the function need not employ eval. You could write it
(defn reverse-fn [data]
(apply (last data) (butlast data)))
Then, for example, as before,
(reverse-fn [3 inc])
=> 4
On a related topic, you might find this explanation of transducers interesting.
Edit:
Notice that functions are literals, in the sense that a function evaluates to itself:
((eval +) 1 1)
=> 2
In general:
Macros have their use however; macros expand at the point they are encountered so you will have one or more code blocks being inlined in the resulting byte code. They are ideal for encapsulating High Order DSL terminologies.
Functions, on the other hand, are wonderful for reuse where you localize the purity of the function and it can be called from multiple other functions without increasing the code footprint. They are ideal for localized or global usage.
REPL behavior consideration: A single function is easier to rework without worrying about fully evaluating the entire source file(s) to ensure all macro reference expansions get updated.
Hope this helps
Simple rules are the best: Use a function if you can, a macro if you must.
At a conceptual level a macro in LISP (and dialects) take a piece of code (as list) and returns another piece of code (again as list).
Based on above principle a simple macro could be:
(defmacro zz [a] (list print a))
;macroexpand says : (#<core$print clojure.core$print#749436> "Hello")
But in clojure this can also be written as:
(defmacro zz [a] `(print ~a))
;macroexpand says : (clojure.core/print "Hello")
I am not exactly sure about the difference here and which should be the preferred way. The first one look simple as we are supposed to return list and avoid using weird characters like back tick.
No one has pointed this out yet...the difference between your 2 macros is this: your second form (using backtick)
(defmacro zz [a] `(print ~a))
is equivalent to:
(defmacro zz [a] (list 'print a))
Which is different from your first example:
(defmacro zz [a] (list print a))
Note the missing single quote -- that is why your macroexpand is different. I agree with the other people posting: using backquote is more conventional if your macro has a fairly simple 'shape'. If you have to do code walking or dynamic construction (i.e. a complex macro), then using lists and building it up is often what's done.
I hope this explanation makes sense.
Constructing lists explicitly is "simplest", in a way, because there are few core concepts you need to know: just accept a list and change it around till you have a new list. Backtick is a convenient shortcut for "templating" chunks of code; it is possible to write any macro without it, but for any large macro it quickly becomes very unpleasant. For example, consider two ways of writing let as a macro over fn:
(defmacro let [bindings & body]
(let [names (take-nth 2 bindings)
vals (take-nth 2 (rest bindings))]
`((fn [~#names]
(do ~#body))
~#vals)))
(defmacro let [bindings & body]
(let [names (take-nth 2 bindings)
vals (take-nth 2 (rest bindings))]
(cons (list `fn (vec names) (cons `do body))
vals)))
In the first case, using backtick makes it fairly clear that you're writing a function of the names containing the body, and then calling it with the values - the macro code is "shaped" the same as the expansion code, so you can imagine what it will look like.
In the second case, with just cons and list all over the place, it is a real headache to work out what the expansion will look like. This isn't always the case, of course: sometimes it can be clearer to write something without a backtick.
Another very important point was made by Kyle Burton: print is not the same as 'print! Your macro expansion should contain the symbol print, not its value (which is a function). Embedding objects (such as functions) in code is very fragile and only works by accident. So make sure your macros expand to code you could actually have written yourself, and let the evaluation system do the hard work - you could type in the symbol print, but you couldn't type in a pointer to the current value of the function print.
There's a style difference between them. Your example is very simple but in more complex macros the difference will be bigger.
For example the unless macro as defined in "The Joy of Clojure" book:
(defmacro unless [condition & body]
`(if (not ~condition)
(do ~#body)))
From the book:
Syntax-quote allows the following if-form to act as a sort of template for the expression
that any use of the macro become when it is expanded.
When creating a macro always choose the most readable and idiomatic style.
To contrast, the above code can equivalently be written like so:
(defmacro unless [condition & body]
(list 'if (list 'not condition)
(list* 'do body)))
In my experience they are equivalent. Though there may be some edge cases I'm not aware of.
#islon 's example can equivalently be written as:
To contrast, the above code can equivalently be written like so:
(defmacro unless [condition & body]
(list 'if (list 'not condition)
(list* 'do body)))