While hunting a bug in one of our clojure modules, I stumbled upon a syntax-quoted macro to which somebody passed arguments. After some debugging I realised that this doesn't work as intended, as the arguments do not have the values they should have.
The below is a minimal representation of the situation encountered in the actual code. It should illustrate the point pretty well:
(def testargs {:arg1 "foo"
:arg2 "bar"})
(defmacro argtest
[arg1 arg2]
`(str ~#(arg1 testargs) " " ~#(arg2 testargs))
This works nicely as expected if I call the code like this:
(argtest :arg1 :arg2)
=> "foo bar"
However, if the two arguments are passed in as variables, it does not work so well:
(let [arg1 :arg1 arg2 :arg2]
(argtest arg1 arg2))
=> " "
To get to the bottom of the problem, I changed the macro as follows and ran the tests again:
(defmacro argtest2
[arg1 arg2]
(str arg1 " " arg2))
(argtest2 :arg1 :arg2)
=> ":arg1 :arg2"
(let [argument1 :arg1 argument2 :arg2]
(argtest2 argument1 argument2))
=> "argument1 argument2"
As you can see, what happens is that the variable names are passed as values. How can I avoid this? Should I just try to rewrite the code to avoid a macro, or can I somehow make this work?
An obvious solution would of course be something like this:
(defmacro argtest
[arg1 arg2]
`(str (~arg1 ~testargs) " " (~arg2 ~testargs)))
But since the output of the macro is inserted into another code block before evaluation, this is not possible.
Forms are not evaluated before being passed to macros as they are with functions. Without broader context or knowing what you're trying to do, the simplest solution would be to use a function, as you seem to want the arguments evaluated in all cases, though maybe I'm oversimplifying your problem:
(defn argtest [arg1 arg2]
(str (arg1 testargs) " " (arg2 testargs)))
Related
I can use memfn to create a clojure function that invokes a java function.
(macroexpand '(memfn startsWith prefix))
=> (fn* ([target2780 prefix] (. target2780 (startsWith prefix))))
((memfn startsWith prefix) "abc" "a")
=> true
memfn requires that the function name be a symbol. I'm wondering if I can write a macro to invoke an arbitrary method whose name is provided as a string. That is, I'd like to be able to invoke the following:
(def fn-name "startsWith")
=> #'user/fn-name
(macroexpand '(memfn' fn-name "prefix"))
=> (fn* ([target2780 prefix] (. target2780 (startsWith prefix))))
((memfn fn-name "prefix") "abc" "a")
=> true
The only way I can think to do this involves using read-string.
(defmacro memfn' [fn-name arg-name]
`(memfn ~(read-string fn-name) ~arg-name))
Edit: A version using read-string and eval that actually works the way I want it to.
(defn memfn' [fn-name arg-name]
(eval (read-string (str "(memfn " fn-name " " arg-name ")"))))
Am I missing a fundamental macro building tool to take the string that a symbol references and turn it into a literal without potentially executing code, as read-string might?
Thanks for any ideas!
There's no way to do this, with or without read-string. Your proposed solution doesn't work. The distinction you're really trying to make is not between string and symbol, but between runtime data and compile-time literals. Macros do not evaluate the arguments they receive, so even if fn-name is the name of a var whose value is "startsWith", memfn (or your memfn' macro) will only ever see fn-name.
If you are interested in calling java methods only then you can rely on java.lang.reflect.Method and its invoke method.
Something like this should work for parameterless methods and would not require a macro.
(defn memfn' [m]
(fn [o] (.invoke (.getMethod (-> o .getClass) m nil) o nil)))
((memfn' "length") "clojure")
;=>7
When I was reading The Joy of Clojure I came across some code.
(fn [~'key ~'r old# new#]
(println old# " -> " new#)
What is the exact behaviour of this declaration ~'some-symbol.
Differences between some-symbol# and '~another-symbol or gensym?
The Joy Of Clojure: (Did not understand)
You’ll see the pattern ~'symbol at times in Clojure
macros for selectively capturing a symbolic name in the body of a
macro. The reason for this bit of awkwardness[11] is that Clojure’s
syntax-quote attempts to resolve symbols in the current context,
resulting in fully qualified symbols. Therefore, ~' avoids that
resolution by unquoting a quote.
You can see an example in the Tupelo library with the Literate Threading Macro. We want the user to type the symbol it and have it recognized by the macro. Here's the definition:
(defmacro it->
"A threading macro like as-> that always uses the symbol 'it'
as the placeholder for the next threaded value "
[expr & forms]
`(let [~'it ~expr
~#(interleave (repeat 'it) forms)
]
~'it))
This is also referred to as an "anaphoric" macro. The user then creates code like this:
(it-> 1
(inc it) ; thread-first or thread-last
(+ it 3) ; thread-first
(/ 10 it) ; thread-last
(str "We need to order " it " items." ) ; middle of 3 arguments
;=> "We need to order 2 items." )
The user includes the special symbol it in their code, which the macro is expecting (& is required in this case).
This is a bit of a special case. In most cases, you want the macro to work no matter what symbols the user picks. That is why most macros use (gensym...) or the reader version with "#" suffix like this example:
(defmacro with-exception-default
"Evaluates body & returns its result. In the event of an exception, default-val is returned
instead of the exception."
[default-val & body]
`(try
~#body
(catch Exception e# ~default-val)))
This is the "normal" case, where the macro creates a "local variable" e# that is guaranteed not to overlap with any user symbol. A similar example shows the spyx macro creating a "local variable" named spy-val# to temporarily hold the result of evaluating the expression expr:
(defmacro spyx
"An expression (println ...) for use in threading forms (& elsewhere). Evaluates the supplied
expression, printing both the expression and its value to stdout, then returns the value."
[expr]
`(let [spy-val# ~expr]
(println (str (spy-indent-spaces) '~expr " => " (pr-str spy-val#)))
spy-val#))
Note that for the (println...) statement, we see the opposite syntax of '~expr -- but that is a topic for another day.
I'm using slingshot's throw+ macro to raise an exception that looks like:
(throw+ {:type ::urlparse})
The type checker doesn't like it:
Type Error (stream2es/http.clj:79:17) Bad arguments to apply:
Target: [String t/Any * -> String]
Arguments: (PersistentList String)
in: (clojure.core/apply clojure.core/format (clojure.core/list "throw+: %s" (clojure.core/pr-str %)))
Type Checker: Found 1 error
The macro in slingshot looks like:
(defmacro throw+
([object]
`(throw+ ~object "throw+: %s" (pr-str ~'%)))
([object message]
`(throw+ ~object "%s" ~message))
([object fmt arg & args]
`(let [environment# (s/environment)
~'% ~object
message# (apply format (list ~fmt ~arg ~#args))
stack-trace# (s/stack-trace)]
(s/throw-context ~'% message# stack-trace# environment#)))
([]
`(s/rethrow)))
I've tried various ann ^:no-check forms on apply and format and none works. Since it's a macro, I'm assuming I can't annotate it since it replaces the code that's there. But I also can't rewrite the code in the macro like was suggested in this other answer, because it's in a library. How do I gradually type in this case?
If you’re not able to rewrite the implementation of throw+, I suggest a wrapper macro like this.
(defmacro typed-throw+ [object]
`(let [o# ~object]
(t/tc-ignore
(throw+ o#))
(throw (Exception.)))) ; unreachable
;; other arities are an exercise ..
This way, you still type check the argument, and core.typed still thinks throw+ always throws an exception — it doesn't really know that, but the final throw clause allows core.typed to give the entire expression type Nothing.
The real answer should be we can improve apply to know that applying a non-empty list will satisfy at least one argument, however this answer should work today.
While I may incorrectly interpret the concept of homoiconicity, I've understood it as 'code being data'.
So, I can write code like this:
(def subject "world")
(def helo '(str "Hello " subject))
At this point, helo is only data, but can be executed as code like this:
(eval helo)
which returns "Hello world".
I can also continue to treat helo as data:
(first helo)
(count helo)
which returns respectively str and 3.
So far so good. However, as soon as I wrap the code in a function, I seem to lose the ability to treat code as data:
(defn helofn [subject]
(str "Hello " subject))
How do I decompose helofn? It seems that I can't treat it as data; if I do this:
(count helofn)
I get an exception:
java.lang.UnsupportedOperationException: count not supported on this type: user$helofn
Is there another way to decompose helofn, or am I just expecting too much from homoiconicity?
The helofn definition is data, but you're letting it be evaluated (just as you explicitly evaluated the helo list). If you treated the definition in the same way as helo, then it will remain data, and amenable to whatever transformations you want to apply:
(def helofndata '(defn helofn [subject]
(str "Hello " subject))
=> (second helofndata)
helofn
=> (eval helofndata)
#'user/helofn
defn is just a macro:
(macroexpand '(defn helofn [subject]
(str "Hello " subject)))
(def helofn (clojure.core/fn ([subject] (str "Hello " subject))))
If you define helofn the way you defined helo, you'll be able to treat it as data:
(def helofn '(fn [subject]
(str "Hello " subject)))
Now you can eval and call this function:
((eval helofn) "world")
and to treat it as a data:
(count helofn)
But, when you use defn macro you associates helofn variable with compiled function and not with it's code.
It's not just functions. Let's say you defined hello with the following code:
(def helo (str "Hello " subject))
Now hello is associated with "Hello world" string and not with (str "Hello " subject) code. So, now there is no way to get the code this string was built with.
N.B. If you want to treat clojure code as data you should look into its macros. Any code passed to a macro is treated as data and any data returned by a macro is treated as code.
Homoiconicity is a very powerful concept and I don't think you are expecting too much from it.
defn is actually a macro that uses the def special form to define a function, so:
(defn sq [x]
(* x x))
Is actually equivalent to:
(def sq (fn ([x] (* x x))))
So defn here is receiving the args sq [x] (* x x), then builds the list (def sq (fn ([x] (* x x)))), returns it as the result of the macro and is then eval'ed. This is all done through the manipulation of lists, maps, vectors, symbols, etc., by the defn macro.
The fact that in Clojure you can't get the original list of symbols from which you defined a function, has to do with the fact that in Clojure all code is compiled. This is why evaluating (fn [x] 1) in the REPL returns something like #<user$eval809$fn__810 user$eval809$fn__810#10287d>
. But still, as mentioned in a previous answer, the code that is evaluated is data.
Maybe I'm going too far with this, but if you wanted to have for each function you define, the data from which it was created, you could add it to its metadata by creating your own custom macro.
Here's a naive implementation for such a macro:
(defmacro defn* [x & body ]
(let [form `'~&form
x (vary-meta x assoc :form form)]
`(defn ~x ~#body)))
;=> #'user/defn*
(defn* sq [x]
(* x x))
;=> #'user/sq
(:form (meta #'sq))
;=> (defn* sq [x] (* x x))
&form is an implicit argument (together with &env) that contains the whole (unevaluated) form with which the macro was called (i.e. the data that is evaluated by the compiler).
Hope this helps and it doesn't bring more confusion.
It looks like no based on
get a clojure function's code
and
Can you get the "code as data" of a loaded function in Clojure?
Basically you can get the source from a function defined in a .clj file but there's no reliable way to retrieve the data structures that built a function from the function alone.
EDIT: Also I think you are expecting too much from homoiconicity. The code itself is data yes but it's fairly standard to not be able to retrieve the original source code based on the artifact emitted by that code. Like when I have 2 I have no way of knowing that it was produced by (+ 1 1) or (- 4 2) in the same way a function is a piece of data created by calling fn over some other data structures that get interpreted as code.
(use '[clojure.contrib.trace])
(dotrace [str] (reduce str [\a \b]))
In a nutshell:
That's because trace-fn-call, which is the thing dotrace uses to wrap the functions to be traced, uses str to produce the nice TRACE foo => val output.
Extended explanation:
The dotrace macro does its magic by installing a thread binding for each Var holding a function to be traced; in this case, there is one such Var, clojure.core/str. The replacement looks roughly like so:
(let [f ##'str]
(fn [& args]
(trace-fn-call 'str f args)))
The trace-fn-call, to quote its docstring, "Traces a single call to a function f with args.". In doing so, it calls the traced function, takes note of the return value, prints out a nice informative message of the form TRACE foo => val and returns the value obtained from the traced function so that regular execution may continue.
As mentioned above, this TRACE foo => val message is produced used str; however, in the case at hand, this is actually the function being traced, so a call to it leads to another call to trace-fn-call, which makes its own attempt to produce the tracing output string using str, which leads to another call to trace-fn-call... ultimately leading to the stack blowing up.
A workaround:
The following modified versions of dotrace and trace-fn-call should work fine even in the presence of weird bindings for core Vars (note that futures may not be scheduled promptly; if that's a problem, see below):
(defn my-trace-fn-call
"Traces a single call to a function f with args. 'name' is the
symbol name of the function."
[name f args]
(let [id (gensym "t")]
#(future (tracer id (str (trace-indent) (pr-str (cons name args)))))
(let [value (binding [*trace-depth* (inc *trace-depth*)]
(apply f args))]
#(future (tracer id (str (trace-indent) "=> " (pr-str value))))
value)))
(defmacro my-dotrace
"Given a sequence of function identifiers, evaluate the body
expressions in an environment in which the identifiers are bound to
the traced functions. Does not work on inlined functions,
such as clojure.core/+"
[fnames & exprs]
`(binding [~#(interleave fnames
(for [fname fnames]
`(let [f# #(var ~fname)]
(fn [& args#]
(my-trace-fn-call '~fname f# args#)))))]
~#exprs))
(Rebinding trace-fn-call around a regular dotrace apparently doesn't work; my guess is that's because of clojure.* Var calls still being hard-wired by the compiler, but that's a separate matter. The above will work, anyway.)
An alternative would be to use the above my-dotrace macro together with a my-trace-fn-call function not using futures, but modified to call custom replacements for the clojure.contrib.trace functions using the following in place of str:
(defn my-str [& args] (apply (.getRoot #'clojure.core/str) args))
The replacements are straightforward and tedious and I omit them from the answer.