When I was reading The Joy of Clojure I came across some code.
(fn [~'key ~'r old# new#]
(println old# " -> " new#)
What is the exact behaviour of this declaration ~'some-symbol.
Differences between some-symbol# and '~another-symbol or gensym?
The Joy Of Clojure: (Did not understand)
You’ll see the pattern ~'symbol at times in Clojure
macros for selectively capturing a symbolic name in the body of a
macro. The reason for this bit of awkwardness[11] is that Clojure’s
syntax-quote attempts to resolve symbols in the current context,
resulting in fully qualified symbols. Therefore, ~' avoids that
resolution by unquoting a quote.
You can see an example in the Tupelo library with the Literate Threading Macro. We want the user to type the symbol it and have it recognized by the macro. Here's the definition:
(defmacro it->
"A threading macro like as-> that always uses the symbol 'it'
as the placeholder for the next threaded value "
[expr & forms]
`(let [~'it ~expr
~#(interleave (repeat 'it) forms)
]
~'it))
This is also referred to as an "anaphoric" macro. The user then creates code like this:
(it-> 1
(inc it) ; thread-first or thread-last
(+ it 3) ; thread-first
(/ 10 it) ; thread-last
(str "We need to order " it " items." ) ; middle of 3 arguments
;=> "We need to order 2 items." )
The user includes the special symbol it in their code, which the macro is expecting (& is required in this case).
This is a bit of a special case. In most cases, you want the macro to work no matter what symbols the user picks. That is why most macros use (gensym...) or the reader version with "#" suffix like this example:
(defmacro with-exception-default
"Evaluates body & returns its result. In the event of an exception, default-val is returned
instead of the exception."
[default-val & body]
`(try
~#body
(catch Exception e# ~default-val)))
This is the "normal" case, where the macro creates a "local variable" e# that is guaranteed not to overlap with any user symbol. A similar example shows the spyx macro creating a "local variable" named spy-val# to temporarily hold the result of evaluating the expression expr:
(defmacro spyx
"An expression (println ...) for use in threading forms (& elsewhere). Evaluates the supplied
expression, printing both the expression and its value to stdout, then returns the value."
[expr]
`(let [spy-val# ~expr]
(println (str (spy-indent-spaces) '~expr " => " (pr-str spy-val#)))
spy-val#))
Note that for the (println...) statement, we see the opposite syntax of '~expr -- but that is a topic for another day.
Related
Here's my failed attempt:
(defmacro until
[condition body setup increment]
`(let [c ~#condition]
(loop [i setup]
(when (not c)
(do
~#body
(recur ~#increment))))))
(def i 1)
(until (> i 5)
(println "Number " i)
0
(inc i))
I get: CompilerException java.lang.RuntimeException: Can't let qualified name: clojure-noob.core/c
I am expecting this output:
Number 1
Number 2
Number 3
Number 4
Number 5
What's wrong?
There are a few issues with the macro:
You need to generate symbols for bindings inside macros. A convenient way to do this is suffix the names with #. Otherwise the bindings in your macros could overshadow bindings elsewhere in your code.
Some of the macro inputs were unnecessarily spliced when unquoted i.e. ~# instead of ~
Here's a version of the macro that will compile/expand:
(defmacro until [condition body setup increment]
`(let [c# ~condition]
(loop [i# ~setup]
(when-not c#
~body
(recur ~increment)))))
But this will loop forever in your example because condition is only evaluated once and i's value would never change anyway. We could fix that:
(defmacro until [condition body increment]
`(loop []
(when-not ~condition
~body
~increment
(recur))))
And we need to make i mutable if we want to change its value:
(def i (atom 1))
(until (> #i 5)
(println "Number " #i)
(swap! i inc))
;; Number 1
;; Number 2
;; Number 3
;; Number 4
;; Number 5
But now until is starting to look a lot like the complement of while, and its extra complexity doesn't seem beneficial.
(defmacro until [test & body]
`(loop []
(when-not ~test
~#body
(recur))))
This version of until is identical to while except the test is inverted, and the sample code above with the atom still behaves correctly. We can further simplify until by using while directly, and it'll ultimately expand to the same code:
(defmacro until [test & body]
`(while (not ~test) ~#body))
Change the let line too:
...
`(let [c# ~#condition]
...
Then rename all references of c to c#. The postfix # generates a unique, non-namespaced-qualified identifier to ensure that the symbol created by the macro doesn't clash with any existing symbols in the context that the macro expands into. Whenever you bind a symbol in a quoted form, you should be using # to prevent collisions, unless you have a good reason to not use it.
Why is this necessary in this case? I can't remember exactly the reason, but if I recall correctly, any symbols bound in a syntax quoted form (`()) are namespace qualified, and you can't use a let to create namespace qualified symbols.
You can recreate the error by typing:
(let [a/a 1]
a/a)
Why do I get the error:
IllegalArgumentException First argument to defn must be a symbol clojure.core/defn (core.clj:277)
When I try to define a function like this:
(defn (symbol "f[]") 1)
Or like this:
(defn (symbol "f") [] 1)
Why aren't those the equivalent of straight forward example below ?
(defn f [] 1)
This is esoteric I know: but it just occurred to me that I might want to name a function dynamically at some point. (No real use case here - just trying to understand Clojure's mind...)
When you pass arguments to a macro, they are not evaluated beforehand. Since defn is a macro, what you're passing it in those two cases are not equivalent.
You are mixing code and data. It is a very common mistake to do. Eg.
(+ 4 5) ; ==> 9
('+ 4 5) ; ==> Error
'+ evaluates to a symbol. It is not the same as the variable + that is code and evaluates for a function. It's easy to check by evaluating them:
+ ; ==> #<core$_PLUS_ clojure.core$_PLUS_#312aa7c>
'+ ; ==> +
defn is a macro that expands to def so your beef is with def. The reason (def (symbol "x") 5) doesn't work is because def happens at compile time. The first arguments is never evaluated, but used for all references to the same identifiers within the same namespace. An expression like (symbol "x") won't work pretty much because of the same reason + and '+ cannot be mixed. You can do this in compile time though:
(defmacro make-fun [name expression]
`(defn ~(symbol name) [] ~expression))
(macroexpand-1 '(make-fun "f" 1))
; ==> (clojure.core/defn f [] 1)
(make-fun "f" 1)
; ==> #'user/f
(f) ; ==> 1
So what is happening is that before the code runs (make-fun "f" 1) gets replaced with (clojure.core/defn f [] 1) and the runtime never ever sees where it came from. While this seems useful you still cannot use a binding or input to make your function:
(def fun-name "f")
(def fun-value 1)
(macroexpand-1 '(make-fun fun-name fun-value))
; ==> (clojure.core/defn fun-name [] fun-value)
Macros are just a way to simplify and abstract on syntax. If you always write a pattern that looks like (defn name [& args] (let ...) you can make the parts that differ bindings in a macro and shorten every place you use the abstraction with the new macro. It is a code translation service. In compile time the arguments are just the literal code that it is suppsoed to replace and you never have the luxury to see if a variable or expression has a certain value since you only knows about the code and never what they actually represent. Thus the errors usually arises in when the code in the end result runs.
In the end you can do anything in runtime with eval. I've seen eval being used in a sensible manner twice in my 19 year run as a professional programmer. You could do:
(defn make-fun [name value]
(eval `(defn ~(symbol name) [] ~value)))
(make-fun fun-name fun-value)
; #'user/f
(f)
; ==> 1
Now while this works you shouldn't do it unless this is some sort of tool to test or do something with code rather than it being a part of the code to be run as a service with the string coming in from a unsafe source. I would have opted for using dictionaries instead such that you do not update your own environment. Imagine if the input was make-fun or some other part of your code that would give the client control over your software.
The answer is what Josh said (defn is a macro; if it was a function then your code really would work in this way). You can define your own defn variation macro that would do what you want or just use eval:
(eval `(defn ~(symbol "f") [] 1))
; => #'user/f
(f)
; => 1
You really don't need to use eval.
You have hit the problem known as "turtles all the way down". Once you try to treat a macro like a function (perhaps passing it to map, for example), you find you cannot do it without writing another macro. The same applies to macro #2, etc.
Thus, you can't compose macros as well as you can compose functions. This is the genesis of the general advice, "Never use a macro when you can use a function."
In this case, defn is a macro, so you have no choice but to write another macro (def behaves the same way, even though it is a special form instead of a macro). Our new macro dyn-defn dynamically creates the function name from a list of strings:
(defn fun-1 [] 1)
(def fun-2 (fn [] 2))
; (def (symbol (str "fun" "-3")) (fn [] 3))
; => Exception: First argument to def must be a Symbol
(defmacro dyn-defn
"Construct a function named dynamically from the supplied strings"
[name-strs & forms]
(let [name-sym (symbol (str/join name-strs)) ]
(spyx name-sym)
`(defn ~name-sym ~#forms)))
(dyn-defn ["fun" "-3"]
[]
3)
with result:
*************** Running tests ***************
:reloading (tst.demo.core)
name-sym => fun-3 ; NOTE: this is evaluated at compile-time
Testing _bootstrap
-------------------------------------
Clojure 1.9.0 Java 1.8.0_161
-------------------------------------
Testing demo.core
Testing tst.demo.core
(fun-1) => 1 ; NOTE: these are all evaluated at run-time
(fun-2) => 2
(fun-3) => 3
Note that the function name is an argument to the defn macro, and must be a symbol, not a function call.
Note:
Correct, you can't tell by looking at it if a form is "calling" a function or a macro. In fact, many "build-in" features of Clojure are constructed from more fundamental parts of the language, whether macros like when (source code) or functions like into (source code).
I have a macro defprinter where I can define functions that: get a value in a dic based on destructuring + print that variable.
This looks like:
(defmacro defprinter [name pattern]
(list 'defn name [pattern] '(prn to-print)))
So I can do something like (defprinter print-lemons {to-print :lemons}) and then (print-lemons {:lemons "lemons"}) and it will print the correct thing (and I can define printers with any kind of destructuring on the first argument).
But now I want to give the option of that function maybe knowing how to print with a color as well, such as, if the symbol color is defined, it should do (prn color "-colored " to-print), but else just (prn color).
So, the function generated by (defprinter print-lemons {to-print :lemons}) should work the same as above, while (defprinter print-lemons {to-print :lemons, color :color}) would write a function that performs the colored version.
Moreover, I want the same effect if I do
(let [color "black"] (defprinter print-lemons {to-print :lemons})) or (def color "black") (defprinter print-lemons {to-print :lemons}).
Is that possible?
I've tried writing it the following way:
(defmacro defprinter [name pattern]
(list 'defn name [pattern]
'(try (eval '(prn (str color "-colored " to-print)))
(catch java.lang.RuntimeException _
(prn to-print)))))
In my understanding the function the macro will write will try to eval the expression at runtime, fail with a RuntimeException if color is not defined and then execute the (prn to-print). And even though it would check for the existence of color at runtime, to-print (which always need to exist for that function) would be checked at compile time when the macro expands.
But what happens here is that I always get a RuntimeException, even when color is defined (even if I leave only to-print on the eval statement, it can't find it but the clause in the catch works fine). Seems like the symbol isn't being resolved as I would expect during eval, but I can't think of any other way to achieve this.
Any suggestions?
First off, it probably makes sense to split apart the two concerns you're working with here: defining a function, and determining how to print based on what vars/locals are available. Making macros as simple as possible tends to make it easier to figure out what's going on.
When deciding what to print, you really have 3 cases:
color is a local
color is a var (or class, I guess?)
color is undefined
&env is a useful and rarely-used tool (only available within macros) that lets you take a look at what's available as a local.
And resolve lets you look at what vars there are with that name.
So in this example, there are 3 potential expressions that could make up the body of the function defined with def-color-printer:
(defn make-print-expression [env]
(if (contains? env 'color)
`(prn (str ~'color "-colored " ~'to-print))
(if-let [color (resolve 'color)]
`(prn (str #~color "-colored " ~'to-print))
`(prn ~'to-print))))
(defmacro def-color-printer [name pattern]
(list `defn name [pattern]
(make-print-expression &env)))
(def-color-printer print-limes {to-print :limes})
(let [color "green"]
(def-color-printer print-limes-color-with-local {to-print :limes}))
(def color "greyish")
(def-color-printer print-limes-color-with-var {to-print :limes})
(print-limes {:limes "limes!"})
;=> "limes!"
(print-limes-color-with-local {:limes "limes!"})
;=> "green-colored limes!"
(print-limes-color-with-var {:limes "limes!"})
;=> "greyish-colored limes!"
I also wrote a blog about Clojure's quoting in case the syntax-quoting syntax is unfamiliar.
the resolve function will likely help you out with this. It returns the thing a symbol represents in the current namespace, or it returns nil. You can feed it to an if expression:
user> (resolve 'asdf)
nil
user> (if (resolve 'asdf) :defined :not-defined)
:not-defined
Remember to quote the symbol you want to resolve in the test.
While I may incorrectly interpret the concept of homoiconicity, I've understood it as 'code being data'.
So, I can write code like this:
(def subject "world")
(def helo '(str "Hello " subject))
At this point, helo is only data, but can be executed as code like this:
(eval helo)
which returns "Hello world".
I can also continue to treat helo as data:
(first helo)
(count helo)
which returns respectively str and 3.
So far so good. However, as soon as I wrap the code in a function, I seem to lose the ability to treat code as data:
(defn helofn [subject]
(str "Hello " subject))
How do I decompose helofn? It seems that I can't treat it as data; if I do this:
(count helofn)
I get an exception:
java.lang.UnsupportedOperationException: count not supported on this type: user$helofn
Is there another way to decompose helofn, or am I just expecting too much from homoiconicity?
The helofn definition is data, but you're letting it be evaluated (just as you explicitly evaluated the helo list). If you treated the definition in the same way as helo, then it will remain data, and amenable to whatever transformations you want to apply:
(def helofndata '(defn helofn [subject]
(str "Hello " subject))
=> (second helofndata)
helofn
=> (eval helofndata)
#'user/helofn
defn is just a macro:
(macroexpand '(defn helofn [subject]
(str "Hello " subject)))
(def helofn (clojure.core/fn ([subject] (str "Hello " subject))))
If you define helofn the way you defined helo, you'll be able to treat it as data:
(def helofn '(fn [subject]
(str "Hello " subject)))
Now you can eval and call this function:
((eval helofn) "world")
and to treat it as a data:
(count helofn)
But, when you use defn macro you associates helofn variable with compiled function and not with it's code.
It's not just functions. Let's say you defined hello with the following code:
(def helo (str "Hello " subject))
Now hello is associated with "Hello world" string and not with (str "Hello " subject) code. So, now there is no way to get the code this string was built with.
N.B. If you want to treat clojure code as data you should look into its macros. Any code passed to a macro is treated as data and any data returned by a macro is treated as code.
Homoiconicity is a very powerful concept and I don't think you are expecting too much from it.
defn is actually a macro that uses the def special form to define a function, so:
(defn sq [x]
(* x x))
Is actually equivalent to:
(def sq (fn ([x] (* x x))))
So defn here is receiving the args sq [x] (* x x), then builds the list (def sq (fn ([x] (* x x)))), returns it as the result of the macro and is then eval'ed. This is all done through the manipulation of lists, maps, vectors, symbols, etc., by the defn macro.
The fact that in Clojure you can't get the original list of symbols from which you defined a function, has to do with the fact that in Clojure all code is compiled. This is why evaluating (fn [x] 1) in the REPL returns something like #<user$eval809$fn__810 user$eval809$fn__810#10287d>
. But still, as mentioned in a previous answer, the code that is evaluated is data.
Maybe I'm going too far with this, but if you wanted to have for each function you define, the data from which it was created, you could add it to its metadata by creating your own custom macro.
Here's a naive implementation for such a macro:
(defmacro defn* [x & body ]
(let [form `'~&form
x (vary-meta x assoc :form form)]
`(defn ~x ~#body)))
;=> #'user/defn*
(defn* sq [x]
(* x x))
;=> #'user/sq
(:form (meta #'sq))
;=> (defn* sq [x] (* x x))
&form is an implicit argument (together with &env) that contains the whole (unevaluated) form with which the macro was called (i.e. the data that is evaluated by the compiler).
Hope this helps and it doesn't bring more confusion.
It looks like no based on
get a clojure function's code
and
Can you get the "code as data" of a loaded function in Clojure?
Basically you can get the source from a function defined in a .clj file but there's no reliable way to retrieve the data structures that built a function from the function alone.
EDIT: Also I think you are expecting too much from homoiconicity. The code itself is data yes but it's fairly standard to not be able to retrieve the original source code based on the artifact emitted by that code. Like when I have 2 I have no way of knowing that it was produced by (+ 1 1) or (- 4 2) in the same way a function is a piece of data created by calling fn over some other data structures that get interpreted as code.
I have a symbol "a" bound to a function:
(defn a []
(println "Hello, World"))
user=> a
#<user$a__292 user$a__292#97eded>
user=> (a)
Hello, World
nil
Then I use syntax-quote, it "resolves the symbol in the current context, yielding a fully-qualified symbol", according to Clojure documentation. But why can't I use it the same way as unqualified symbol?
user=> `a
user/a
user=> (`a)
java.lang.IllegalArgumentException: Wrong number of args passed to: Symbol (NO_SOURCE_FILE:0)
Second question: if I have a symbol in a list, why can't I evaluate it the same way as if I would evaluate the symbol directly?
user=> (def l '(a 1 2))
#'user/l
user=> 'l
l
user=> (first l)
a
user=> ((first l))
java.lang.IllegalArgumentException: Wrong number of args passed to: Symbol (NO_SOURCE_FILE:0)
I have a suspicion I have a fatal flaw somewhere in the fundamental understanding of how symbols work here. What is wrong with above code?
REPL = read eval print loop. Step through the read-eval process.
READ: Clojure sees the string "(`a)", parses it and ends up with a data structure. At read time, reader macros are expanded and not much else happens. In this case, the reader expands the backquote and ends up with this:
user> (read-string "(`a)")
((quote user/a))
EVAL: Clojure tries to evaluate this object. Evaluation rules vary depending on what kind of object you're looking at.
Some objects evaluate as themselves (numbers, strings, keywords etc.).
A Symbol is evaluated by resolving it in some namespace to obtain some value (usually).
A List is evaluated by macro-expanding the list until there are no macros left, then recursively evaluating the first item in the list to obtain some resulting value, then using the value of the first item in the list to decide what to do. If the first value is a special form, special stuff happens. Otherwise the first value is treated as a function and called with the values of the rest of the list (obtained by recursively evaluating all of the list's items) as parameters.
etc.
Refer to clojure.lang.Compiler/analyzeSeq in the Clojure source to see the evaluation rules for lists, or clojure.lang.Compiler/analyzeSymbol for symbols. There are lots of other evaluation rules there.
Example
Suppose you do this:
user> (user/a)
The REPL ends up doing this internally:
user> (eval '(user/a))
Clojure sees that you're evaluating a list, so it evaluates all items in the list. The first (and only) item:
user> (eval 'user/a)
#<user$a__1811 user$a__1811#82c23d>
a is not a special form and this list doesn't need to be macroexpanded, so the symbol a is looked up in the namespace user and the resulting value here is an fn. So this fn is called.
Your code
But instead you have this:
user> (eval '((quote user/a)))
Clojure evaluates the first item in the list, which is itself a list.
user> (eval '(quote user/a))
user/a
It evaluated the first item in this sub-list, quote, which is a special form, so special rules apply and it returns its argument (the Symbol a) un-evaluated.
The symbol a is the value in this case as the fn was the value up above. So Clojure treats the Symbol itself as a function and calls it. In Clojure, anything that implements the Ifn interface is callable like an fn. It so happens that clojure.lang.Symbol implements Ifn. A Symbol called as a function expects one parameter, a collection, and it looks itself up in that collection. It's meant to be used like this:
user> ('a {'a :foo})
:foo
This is what it tries to do here. But you aren't passing any parameters, so you get the error "Wrong number of args passed to: Symbol" (it expects a collection).
For your code to work you'd need two levels of eval. This works, hopefully you can see why:
user> (eval '((eval (quote user/a))))
Hello, world
user> ((eval (first l)))
Hello, world
Note that in real code, using eval directly is usually a really bad idea. Macros are a better idea by far. I'm only using it here for demonstration.
Look in Compiler.java in the Clojure source to see how this all plays out. It's not too hard to follow.
Using a Symbol as a function is not the same thing as evaluating it. Symbols-as-functions work the same way as keywords-as-functions. Like this:
user=> (declare a)
#'user/a
user=> (def a-map {'a "value"})
#'user/a-map
user=> ('a a-map)
"value"
user=>
This is not how you would normally use a symbol. They are more commonly used for looking up vars in a namespace, and when generating code in a macro.
To break down the layers of indirection, let's define "x" as 1 and see what happens:
user=> (def x 1)
#'user/x
Using def, we have created a "var." The name of the var is the symbol user/x. The def special form returns the var itself to the repl, and this is what we can see printed. Let's try and get a hold of that var:
user=> #'x
#'user/x
The #' syntax is a reader macro that says "give me the var referred to by the following symbol." And in our case, that symbol is "x". We got the same var back as before. Vars are pointers to values, and can be dereferenced:
user=> (deref #'x)
1
But the var needs to be found before it can be dereferenced. This is where the callability of symbols come into play. A namespace is like a map, where the symbols are keys and vars are the values, and when we plainly name a symbol, we implicitly look up its var in our namespace. Like this:
user=> ('x (.getMappings *ns*))
#'user/x
Although, in reality, it is probably more like this:
user=> (.findInternedVar *ns* 'x)
#'user/x
And now we have come full circle on the journey of the unquoted symbol:
user=> (deref (.findInternedVar *ns* 'x))
1
user=> x
1
The two are not entirely equal, though. Because the evaluator does this for all symbols, including deref and *ns*.
The thing about quoting is that you essentially bypass this whole mechanism, and just get the plain symbol back. Like the #' reader macro get plain vars back, the ` and ' reader macros will get plain symbols back, with or without a namespace qualification respectively:
user=> 'x
x
user=> `x
user/x
user=> (def l '(a 1 2))
user=> ((first l))
Turn this into:
user=> (def l `(~a 1 2))
The ~ here resolves the symbol a to its corresponding var, and the backtick makes unquoting work.
In general, you must understand the difference between vars (which are bound to something) and symbols (which are never bound to anything).
I'll try to explain it (in the hope that my exaplanation does not confuse you further):
user=> (def v "content")
#'user/content
-> defines a var in the current namespace under the symbol 'v (fully qualified 'user/v, assuming this is the current namespace), and binds it (the var, not the symbol) to the object "content".
user=> v
"content"
-> resolves v to the var, and gets the bound value
user=> #'v
#'user/v
-> resolves to the var itself
user=> 'v
v
-> does not resolve anything, just a plain symbol (unfortunately, the REPL does not indicate this, printing 'v as v)
user=> `v
user/v
-> as you already quoted, resolves to the symbol in the current context (namespace), but the result is still a symbol (fully qualified), not the var user/v
user=> '(v)
(v)
-> plain quoting, does not resolve anything
user=> `(v)
(user/v)
-> syntax-quote, same as quoting, but resolves symbols to namespace-qualified symbols
user=> `(~v)
("content")
-> resolve the symbol to its var (which is implicitely dereferenced), yielding its bound object