I have a problem with a pair of Clojure macros when a default argument is defined.
In the following situation with 2 macros, where mm02 calls mm01:
(defmacro mm01
[ & [ { :keys [ f1 ] :or { f1 long } :as opts } ]]
`(let []
(println "(2) ~f1" ~f1)))
(defmacro mm02
[ & [ { :keys [ f1 ] :as opts } ]]
`(let []
(println "(1) ~f1" ~f1)
(mm01 ~#opts)))
The evaluation of:
(mm02 { :f1 byte })
prints out:
(1) ~f1 #function[clojure.core/byte]
(2) ~f1 #function[clojure.core/long]
However, I would have expected:
(1) ~f1 #function[clojure.core/byte]
(2) ~f1 #function[clojure.core/byte]
Am I doing anything wrong or I am missing something?
By the way, the evaluation of:
(mm01 { :f1 byte })
prints out:
(2) ~f1 #function[clojure.core/byte]
Thank you very much.
~# expands out a sequence of things into several individual things, inside of a syntax-quote context. Your opts binding is a map, which is conceptually a sequence of map-entries. You can see this in action by playing around in the repl with the expressions your macros will generate: this is often a useful way to look at the intermediate steps of a macro, compared to debugging by trial-and-error on the macro as a whole.
user=> (let [opts {:f1 'long}]
#_=> `(foo ~#opts))
(user/foo [:f1 long])
See the square brackets around :f1 long? That's the problem: your other macro expects to be called with a map, not a vector. As a result, the destructuring fails to find the key you were looking for. To fix this, just remove the # and use an ordinary unquote, not a splicing unquote.
user=> (let [opts {:f1 'long}]
#_=> `(foo ~opts))
(user/foo {:f1 long})
As an additional improvement, you should replace the distracting [& [{...}]] argument stanza with just [{...}]. They behave the same except that the former allows the caller to pass zero arguments (filling in with nils), or any number of extra arguments, which all get ignored. Your version is very slightly more convenient for the caller if they meant to omit an argument and get defaults, but inevitably will lead them to a big hassle of debugging if they leave out an argument, or provide too many, accidentally.
Related
Why do I get the error:
IllegalArgumentException First argument to defn must be a symbol clojure.core/defn (core.clj:277)
When I try to define a function like this:
(defn (symbol "f[]") 1)
Or like this:
(defn (symbol "f") [] 1)
Why aren't those the equivalent of straight forward example below ?
(defn f [] 1)
This is esoteric I know: but it just occurred to me that I might want to name a function dynamically at some point. (No real use case here - just trying to understand Clojure's mind...)
When you pass arguments to a macro, they are not evaluated beforehand. Since defn is a macro, what you're passing it in those two cases are not equivalent.
You are mixing code and data. It is a very common mistake to do. Eg.
(+ 4 5) ; ==> 9
('+ 4 5) ; ==> Error
'+ evaluates to a symbol. It is not the same as the variable + that is code and evaluates for a function. It's easy to check by evaluating them:
+ ; ==> #<core$_PLUS_ clojure.core$_PLUS_#312aa7c>
'+ ; ==> +
defn is a macro that expands to def so your beef is with def. The reason (def (symbol "x") 5) doesn't work is because def happens at compile time. The first arguments is never evaluated, but used for all references to the same identifiers within the same namespace. An expression like (symbol "x") won't work pretty much because of the same reason + and '+ cannot be mixed. You can do this in compile time though:
(defmacro make-fun [name expression]
`(defn ~(symbol name) [] ~expression))
(macroexpand-1 '(make-fun "f" 1))
; ==> (clojure.core/defn f [] 1)
(make-fun "f" 1)
; ==> #'user/f
(f) ; ==> 1
So what is happening is that before the code runs (make-fun "f" 1) gets replaced with (clojure.core/defn f [] 1) and the runtime never ever sees where it came from. While this seems useful you still cannot use a binding or input to make your function:
(def fun-name "f")
(def fun-value 1)
(macroexpand-1 '(make-fun fun-name fun-value))
; ==> (clojure.core/defn fun-name [] fun-value)
Macros are just a way to simplify and abstract on syntax. If you always write a pattern that looks like (defn name [& args] (let ...) you can make the parts that differ bindings in a macro and shorten every place you use the abstraction with the new macro. It is a code translation service. In compile time the arguments are just the literal code that it is suppsoed to replace and you never have the luxury to see if a variable or expression has a certain value since you only knows about the code and never what they actually represent. Thus the errors usually arises in when the code in the end result runs.
In the end you can do anything in runtime with eval. I've seen eval being used in a sensible manner twice in my 19 year run as a professional programmer. You could do:
(defn make-fun [name value]
(eval `(defn ~(symbol name) [] ~value)))
(make-fun fun-name fun-value)
; #'user/f
(f)
; ==> 1
Now while this works you shouldn't do it unless this is some sort of tool to test or do something with code rather than it being a part of the code to be run as a service with the string coming in from a unsafe source. I would have opted for using dictionaries instead such that you do not update your own environment. Imagine if the input was make-fun or some other part of your code that would give the client control over your software.
The answer is what Josh said (defn is a macro; if it was a function then your code really would work in this way). You can define your own defn variation macro that would do what you want or just use eval:
(eval `(defn ~(symbol "f") [] 1))
; => #'user/f
(f)
; => 1
You really don't need to use eval.
You have hit the problem known as "turtles all the way down". Once you try to treat a macro like a function (perhaps passing it to map, for example), you find you cannot do it without writing another macro. The same applies to macro #2, etc.
Thus, you can't compose macros as well as you can compose functions. This is the genesis of the general advice, "Never use a macro when you can use a function."
In this case, defn is a macro, so you have no choice but to write another macro (def behaves the same way, even though it is a special form instead of a macro). Our new macro dyn-defn dynamically creates the function name from a list of strings:
(defn fun-1 [] 1)
(def fun-2 (fn [] 2))
; (def (symbol (str "fun" "-3")) (fn [] 3))
; => Exception: First argument to def must be a Symbol
(defmacro dyn-defn
"Construct a function named dynamically from the supplied strings"
[name-strs & forms]
(let [name-sym (symbol (str/join name-strs)) ]
(spyx name-sym)
`(defn ~name-sym ~#forms)))
(dyn-defn ["fun" "-3"]
[]
3)
with result:
*************** Running tests ***************
:reloading (tst.demo.core)
name-sym => fun-3 ; NOTE: this is evaluated at compile-time
Testing _bootstrap
-------------------------------------
Clojure 1.9.0 Java 1.8.0_161
-------------------------------------
Testing demo.core
Testing tst.demo.core
(fun-1) => 1 ; NOTE: these are all evaluated at run-time
(fun-2) => 2
(fun-3) => 3
Note that the function name is an argument to the defn macro, and must be a symbol, not a function call.
Note:
Correct, you can't tell by looking at it if a form is "calling" a function or a macro. In fact, many "build-in" features of Clojure are constructed from more fundamental parts of the language, whether macros like when (source code) or functions like into (source code).
When I was reading The Joy of Clojure I came across some code.
(fn [~'key ~'r old# new#]
(println old# " -> " new#)
What is the exact behaviour of this declaration ~'some-symbol.
Differences between some-symbol# and '~another-symbol or gensym?
The Joy Of Clojure: (Did not understand)
You’ll see the pattern ~'symbol at times in Clojure
macros for selectively capturing a symbolic name in the body of a
macro. The reason for this bit of awkwardness[11] is that Clojure’s
syntax-quote attempts to resolve symbols in the current context,
resulting in fully qualified symbols. Therefore, ~' avoids that
resolution by unquoting a quote.
You can see an example in the Tupelo library with the Literate Threading Macro. We want the user to type the symbol it and have it recognized by the macro. Here's the definition:
(defmacro it->
"A threading macro like as-> that always uses the symbol 'it'
as the placeholder for the next threaded value "
[expr & forms]
`(let [~'it ~expr
~#(interleave (repeat 'it) forms)
]
~'it))
This is also referred to as an "anaphoric" macro. The user then creates code like this:
(it-> 1
(inc it) ; thread-first or thread-last
(+ it 3) ; thread-first
(/ 10 it) ; thread-last
(str "We need to order " it " items." ) ; middle of 3 arguments
;=> "We need to order 2 items." )
The user includes the special symbol it in their code, which the macro is expecting (& is required in this case).
This is a bit of a special case. In most cases, you want the macro to work no matter what symbols the user picks. That is why most macros use (gensym...) or the reader version with "#" suffix like this example:
(defmacro with-exception-default
"Evaluates body & returns its result. In the event of an exception, default-val is returned
instead of the exception."
[default-val & body]
`(try
~#body
(catch Exception e# ~default-val)))
This is the "normal" case, where the macro creates a "local variable" e# that is guaranteed not to overlap with any user symbol. A similar example shows the spyx macro creating a "local variable" named spy-val# to temporarily hold the result of evaluating the expression expr:
(defmacro spyx
"An expression (println ...) for use in threading forms (& elsewhere). Evaluates the supplied
expression, printing both the expression and its value to stdout, then returns the value."
[expr]
`(let [spy-val# ~expr]
(println (str (spy-indent-spaces) '~expr " => " (pr-str spy-val#)))
spy-val#))
Note that for the (println...) statement, we see the opposite syntax of '~expr -- but that is a topic for another day.
I'm examining the Listing 11.9 of the named book (p.269 of pdf).
Could anyone explain me how tests value is being set (line [tests all-tests :as results])?
thanks
To set the context of the question for people without The Joy of Clojure (a book I do enjoy btw), the macro in question is:
(defmacro with-promises [[n tasks _ as] & body]
(when as
`(let [tasks# ~tasks
n# (count tasks#)
promises# (take n# (repeatedly promise))]
(dotimes [i# n#]
(dothreads!
(fn []
(deliver (nth promises# i#)
((nth tasks# i#))))))
(let [~n tasks#
~as promises#]
~#body))))
And is used thusly:
(defn run-tests [& all-tests]
(with-promises
[tests all-tests :as results]
(into (TestRun. 0 0 0)
(reduce #(merge-with + %1 %2) {}
(for [r results]
(if #r
{:run 1 :passed 1}
{:run 1 :failed 1}))))))
and the final call to run-tests is like:
(run-tests pass fail fail fail pass)
=> #user.TestRun{:run 5, :passed 2, :failed 3}
Ultimately, the latter part of the macro is doing a let assignment and running the body, so you end up with
(let [tests tasks#
results promises#]
(into (TestRun. 0 0 0)
;; rest of body
In the macro, ~n is unquoting the starting back-tick around the '(let so you can just read it as n which is the first parameter to the macro (well, the first parameter of the vector that is the first parameter to the macro).
This all happens after the macro has setup the promises using the custom dothreads! function that uses a thread pool - non of which is important to understand the macro.
You can determine more about the macro by wrapping it in a (pprint (macroexpand-1 '(with-promises ... which generates something like (I've replaced the auto generated names with something simpler, v1, n1, p1 and i1):
(clojure.core/let
[v1 all-tests
n1 (clojure.core/count v1)
p1 (clojure.core/take n1 (clojure.core/repeatedly clojure.core/promise))]
(clojure.core/dotimes
[i1 n1]
(user/dothreads!
(clojure.core/fn
[]
(clojure.core/deliver
(clojure.core/nth p1 i1)
((clojure.core/nth v1 i1))))))
(clojure.core/let
[tests v1
results p1]
(into
(TestRun. 0 0 0)
;; ... rest of main body
which clearly shows the parameters passed in are used as variables in the final let bindings.
However, in this example usage (i.e. run-tests function), the tests variable isn't actually used in the body of the with-promises call, only results is, so you're right to question it, it simply isn't needed.
Looking at the macro definition, there may be further optimizations for this case, as the tasks# binding doesn't seem to give anything extra than wrapping tasks. At first I wondered if this was about immutability in the dothreads! call, or macro-niceness for providing a closure around the usage, rather than directly using the parameter to the macro.
I tried changing the macro to remove tasks# completely and directly use ~tasks, which seems to still work, and as "tests" isn't a required binding variable in the body of run-tests, you can drop both the n parameter from the macro, and the ~n tasks# part of the final let binding without issue.
Actually after reading it several times it finally dawned on me it's to make the whole vector read like a standard destructuring binding.
EDIT: some more explanation on "tests".
This is just a name, it could be "foo-tests", "foo-bar", because ultimately it's used to define something in a let binding.
Had the run-tests body been something like:
(defn run-tests [& all-tests]
(with-promises
[foo all-tests :as results]
(println "foo was set to" foo)
(into (TestRun. 0 0 0)
;; rest of body
you can see how foo (and results) are just used to ultimately define variables (eck - you know what i mean) that can be used in the body part of the call to the macro. The body being everything after the initial vector [foo all-tests :as results] but in the original code, tests is declared but unused.
tests appears to be a function, so :as puts the result (output) of running tests on all-tests.
(edit:)
Upon careful inspection, the with-promises macro appears to be setting the tests to the count of tests.
From what I'm reading (don't know much about macros), the arguments appear to map ("tests" "all-tests" ":as" "results") -> ("n" "tasks" "_" "as") but what I can't quite get is that would imply when requires a value for results ("as") when we're supposed to be creating it. Anyway, the value of tests is set in the final let of the macro.
This code is far too clever, in my humble opinion. Fogus is a master, but this is not his best work.
(If I'm wrong, hopefully someone will be inspired.)
I have an incoming lazy stream lines from a file I'm reading with tail-seq (to contrib - now!) and I want to process those lines one after one with several "listener-functions" that takes action depending on re-seq-hits (or other things) in the lines.
I tried the following:
(defn info-listener [logstr]
(if (re-seq #"INFO" logstr) (println "Got an INFO-statement")))
(defn debug-listener [logstr]
(if (re-seq #"DEBUG" logstr) (println "Got a DEBUG-statement")))
(doseq [line (tail-seq "/var/log/any/java.log")]
(do (info-listener logstr)
(debug-listener logstr)))
and it works as expected. However, there is a LOT of code-duplication and other sins in the code, and it's boring to update the code.
One important step seems to be to apply many functions to one argument, ie
(listen-line line '(info-listener debug-listener))
and use that instead of the boring and error prone do-statement.
I've tried the following seemingly clever approach:
(defn listen-line [logstr listener-collection]
(map #(% logstr) listener-collection))
but this only renders
(nil) (nil)
there is lazyiness or first class functions biting me for sure, but where do I put the apply?
I'm also open to a radically different approach to the problem, but this seems to be a quite sane way to start with. Macros/multi methods seems to be overkill/wrong for now.
Making a single function out of a group of functions to be called with the same argument can be done with the core function juxt:
=>(def juxted-fn (juxt identity str (partial / 100)))
=>(juxted-fn 50)
[50 "50" 2]
Combining juxt with partial can be very useful:
(defn listener [re message logstr]
(if (re-seq re logstr) (println message)))
(def juxted-listener
(apply juxt (map (fn [[re message]] (partial listner re message))
[[#"INFO","Got INFO"],
[#"DEBUG", "Got DEBUG"]]))
(doseq [logstr ["INFO statement", "OTHER statement", "DEBUG statement"]]
(juxted-listener logstr))
You need to change
(listen-line line '(info-listener debug-listener))
to
(listen-line line [info-listener debug-listener])
In the first version, listen-line ends up using the symbols info-listener and debug-listener themselves as functions because of the quoting. Symbols implement clojure.lang.IFn (the interface behind Clojure function invocation) like keywords do, i.e. they look themselves up in a map-like argument (actually a clojure.lang.ILookup) and return nil if applied to something which is not a map.
Also note that you need to wrap the body of listen-line in dorun to ensure it actually gets executed (as map returns a lazy sequence). Better yet, switch to doseq:
(defn listen-line [logstr listener-collection]
(doseq [listener listener-collection]
(listener logstr)))
Given a list of names for variables, I want to set those variables to an expression.
I tried this:
(doall (for [x ["a" "b" "c"]] (def (symbol x) 666)))
...but this yields the error
java.lang.Exception: First argument to def must be a Symbol
Can anyone show me the right way to accomplish this, please?
Clojure's "intern" function is for this purpose:
(doseq [x ["a" "b" "c"]]
(intern *ns* (symbol x) 666))
(doall (for [x ["a" "b" "c"]] (eval `(def ~(symbol x) 666))))
In response to your comment:
There are no macros involved here. eval is a function that takes a list and returns the result of executing that list as code. ` and ~ are shortcuts to create a partially-quoted list.
` means the contents of the following lists shall be quoted unless preceded by a ~
~ the following list is a function call that shall be executed, not quoted.
So ``(def ~(symbol x) 666)is the list containing the symboldef, followed by the result of executingsymbol xfollowed by the number of the beast. I could as well have written(eval (list 'def (symbol x) 666))` to achieve the same effect.
Updated to take Stuart Sierra's comment (mentioning clojure.core/intern) into account.
Using eval here is fine, but it may be interesting to know that it is not necessary, regardless of whether the Vars are known to exist already. In fact, if they are known to exist, then I think the alter-var-root solution below is cleaner; if they might not exist, then I wouldn't insist on my alternative proposition being much cleaner, but it seems to make for the shortest code (if we disregard the overhead of three lines for a function definition), so I'll just post it for your consideration.
If the Var is known to exist:
(alter-var-root (resolve (symbol "foo")) (constantly new-value))
So you could do
(dorun
(map #(-> %1 symbol resolve (alter-var-root %2))
["x" "y" "z"]
[value-for-x value-for-y value-for z]))
(If the same value was to be used for all Vars, you could use (repeat value) for the final argument to map or just put it in the anonymous function.)
If the Vars might need to be created, then you can actually write a function to do this (once again, I wouldn't necessarily claim this to be cleaner than eval, but anyway -- just for the interest of it):
(defn create-var
;; I used clojure.lang.Var/intern in the original answer,
;; but as Stuart Sierra has pointed out in a comment,
;; a Clojure built-in is available to accomplish the same
;; thing
([sym] (intern *ns* sym))
([sym val] (intern *ns* sym val)))
Note that if a Var turns out to have already been interned with the given name in the given namespace, then this changes nothing in the single argument case or just resets the Var to the given new value in the two argument case. With this, you can solve the original problem like so:
(dorun (map #(create-var (symbol %) 666) ["x" "y" "z"]))
Some additional examples:
user> (create-var 'bar (fn [_] :bar))
#'user/bar
user> (bar :foo)
:bar
user> (create-var 'baz)
#'user/baz
user> baz
; Evaluation aborted. ; java.lang.IllegalStateException:
; Var user/baz is unbound.
; It does exist, though!
;; if you really wanted to do things like this, you'd
;; actually use the clojure.contrib.with-ns/with-ns macro
user> (binding [*ns* (the-ns 'quux)]
(create-var 'foobar 5))
#'quux/foobar
user> quux/foobar
5
Evaluation rules for normal function calls are to evaluate all the items of the list, and call the first item in the list as a function with the rest of the items in the list as parameters.
But you can't make any assumptions about the evaluation rules for special forms or macros. A special form or the code produced by a macro call could evaluate all the arguments, or never evaluate them, or evaluate them multiple times, or evaluate some arguments and not others. def is a special form, and it doesn't evaluate its first argument. If it did, it couldn't work. Evaluating the foo in (def foo 123) would result in a "no such var 'foo'" error most of the time (if foo was already defined, you probably wouldn't be defining it yourself).
I'm not sure what you're using this for, but it doesn't seem very idiomatic. Using def anywhere but at the toplevel of your program usually means you're doing something wrong.
(Note: doall + for = doseq.)