How can we find out what a reader macro does ?
For instance I know that a regex #"abc" is the same as (re-pattern "abc").
However the following : (macroexpand #"abc" ) yields #"abc", so how can I go from the reader special form to the "normal" function form ?
You can find out what the reader expands to by using read-string:
(read-string "`(rest ~f)")
=>
(clojure.core/seq
(clojure.core/concat (clojure.core/list (quote clojure.core/rest)) (clojure.core/list f)))
(read-string "#(x %&)")
=> (fn* [& rest__92046#] (x rest__92046#))
(read-string "`(~#x)")
=> (clojure.core/seq (clojure.core/concat x))
(read-string "`(~x)")
=> (clojure.core/seq (clojure.core/concat (clojure.core/list x)))
(read-string "'map")
=> (quote map)
(read-string "#'map")
=> (var map)
(read-string "#x")
=> (clojure.core/deref x)
(binding [*print-meta* true]
(pr-str (read-string "(def ^:foo x 'map)")))
=> "(def ^{:foo true} x (quote map))"
(read-string "#\"x\"")
=> #"x"
In your particular case however: The regex is printed back to you the same way because that's how it's printed in clojure.
(class #"abc")
=> java.util.regex.Pattern
Related
Why does this fail:
(eval (with-meta '(fn [] 0) {:stack (gensym "overflow")}))
; Syntax error compiling at (REPL:1:1).
; Unable to resolve symbol: overflow210 in this context
when none of the following fail?
(eval (with-meta '(do [] 0) {:stack (gensym "overflow")}))
; 0
(eval (with-meta '(let [] 0) {:stack (gensym "overflow")}))
; 0
(eval (with-meta '(if true 0 1) {:stack (gensym "overflow")}))
; 0
(eval (with-meta '(println "hello") {:stack (gensym "overflow")}))
; hello
; nil
The examples above are my attempt to find a minimal, reproducible example. I ran into this question when working on a macro, with a simplified example here:
(defmacro my-macro []
(with-meta '(fn [] 0) {:stack (gensym "overflow")}))
(my-macro)
; Syntax error compiling at (REPL:1:1).
; Unable to resolve symbol: overflow156 in this context
while attempting to follow the model explained in this post on testing Clojure macros.
Great question! This was fun to dig into.
It's helpful to first start by trying to set the metadata map to something that works, and retrieve it in each of your examples:
(meta (eval (with-meta '(fn [] 0) {:ten 10})))
;;=> {:ten 10}
(meta (eval (with-meta '(do [] 0) {:ten 10})))
;;=> nil
(meta (eval (with-meta '(let [] 0) {:ten 10})))
;;=> nil
(meta (eval (with-meta '(if true 0 1) {:ten 10})))
;;=> nil
(meta (eval (with-meta '(println "hello") {:ten 10})))
;; printed: hello
;;=> nil
Hopefully this gives you some idea of what's happening here: the metadata isn't returned as part of the value of the non-fn forms, so it's not evaluated. We can test this hypothesis with another value that uses its metadata, like a vector:
(meta (eval (with-meta '[1] {:ten 10})))
;;=> {:ten 10}
but with a gensym:
(eval (with-meta '[1] {:stack (gensym "overflow")}))
;;=> Syntax error compiling at (tmp:localhost:35479(clj)*:25:7).
;;=> Unable to resolve symbol: overflow6261 in this context
You can see where this is emitted in the Clojure compiler, and searching for new MetaExpr will show you the other places where metadata evaluation is emitted (I could see vectors, maps, sets, functions, and reify).
tl,dr: Clojure evaluates the metadata for function forms because it attaches the evaluated metadata to the resulting function. This is also true of the data literals supported in Clojure's syntax. Any other metadata on forms is stripped away by compilation, so it doesn't get evaluated and thus doesn't cause symbol resolution errors.
The problem is your usage of gensym, not the metadata. Observe:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(dotest
(binding [*print-meta* true]
(let [
fn-code-plain '(fn [] 42)
fn-code-meta (with-meta '(fn [] 42) {:alpha true})
fn-code-sym-meta (with-meta '(fn [] 42) {:alpha (gensym "dummy")})
]
(prn fn-code-plain)
(prn fn-code-meta)
\
(prn (eval fn-code-plain))
(prn (eval fn-code-meta))
(newline)
(prn fn-code-sym-meta)
(throws? (eval fn-code-sym-meta))
)
))
with result:
^{:line 7, :column 27} (fn [] 42)
^{:alpha true} (fn [] 42)
#object[user$eval19866$fn__19867 0xac891b5 "user$eval19866$fn__19867#ac891b5"]
^{:alpha true} #object[user$eval19870$fn__19871 0x2d1ab7e0 "user$eval19870$fn__19871#2d1ab7e0"]
^{:alpha dummy19865} (fn [] 42)
The problem is that eval sees the symbol dummy19865 and tries to resolve it. The fact that it was created by gensym is irrelevant. No problem with a keyword:
fn-code-kw-meta (with-meta '(fn [] 42) {:alpha :dummy-42})
<snip>
(prn fn-code-kw-meta)
(prn (eval fn-code-kw-meta))
producing:
^{:alpha :dummy-42} (fn [] 42)
^{:alpha :dummy-42} #object[user$eval19953$fn__19954 0x13253ac7 "user$eval19953$fn__19954#13253ac7"]
or a symbol that is defined:
(def mysym "Forty-Two!")
<snip>
fn-code-mysym-meta (with-meta '(fn [] 42) {:alpha mysym})
<snip>
(prn fn-code-mysym-meta)
(prn (eval fn-code-mysym-meta))
with result:
^{:alpha "Forty-Two!"} (fn [] 42)
^{:alpha "Forty-Two!"} #object[user$eval20082$fn__20083 0x24a63de5 "user$eval20082$fn__20083#24a63de5"]
Summary:
You have demonstrated that eval only attempts symbol resolution of metadata for the fn form, but not other special forms like do, let, if, or with pre-existing functions such as println. If you wish to explore further, you should probably inquire at the Clojure email list:
clojure#googlegroups.com
The above code is based on this template project.
I am trying to use Instaparse to make a simple arithmetic expression evaluator. The parser seems to work fine but I cannot figure out how to evaluate the returned nested vector. Currently I am using postwalk, like this
(ns test5.core
(:require [instaparse.core :as insta])
(:require [clojure.walk :refer [postwalk]])
(:gen-class))
(def WS
(insta/parser
"WS = #'\\s+'"))
(def transform-options
{:IntLiteral read-string})
(def parser
(insta/parser
"AddExpr = AddExpr '+' MultExpr
| AddExpr '-' MultExpr
| MultExpr
MultExpr = MultExpr '*' IntLiteral
| MultExpr '/' IntLiteral
| IntLiteral
IntLiteral = #'[0-9]+'"
:auto-whitespace WS))
(defn parse[input]
(->> (parser input)
(insta/transform transform-options)))
(defn visit [node]
(println node)
(cond
(number? node) node
(string? node) (resolve (symbol node))
(vector? node)
(cond
(= :MultExpr (first node)) (visit (rest node))
(= :AddExpr (first node)) (visit (rest node))
:else node)
:else node))
(defn evaluate [tree]
(println tree)
(postwalk visit tree))
(defn -main
[& args]
(evaluate (parse "1 * 2 + 3")))
postwalk does traverse the vector but I get a nested list as the result, eg
((((1) #'clojure.core/* 2)) #'clojure.core/+ (3))
Use org.clojure/core.match. Base on your current grammar, you can write the evaluation function as:
(defn eval-expr [expr]
(match expr
[:MultExpr e1 "*" e2] (* (eval-expr e1)
(eval-expr e2))
[:MultExpr e1 "/" e2] (/ (eval-expr e1)
(eval-expr e2))
[:AddExpr e1 "+" e2] (+ (eval-expr e1)
(eval-expr e2))
[:AddExpr e1 "-" e2] (- (eval-expr e1)
(eval-expr e2))
[:MultExpr e1] (eval-expr e1)
[:AddExpr e1] (eval-expr e1)
:else expr))
and evaluate with:
(-> "1 * 2 + 3"
parse
eval-expr)
;; => 5
This doesn't use Instaparse or clojure.walk, but here's something I had for evaluating infix math using only reduce:
(defn evaluate
"Evaluates an infix arithmetic form e.g. (1 + 1 * 2)."
[e]
(let [eval-op (fn [op a b]
(let [f (resolve op)]
(f a b)))]
(reduce
(fn [[v op] elem]
(cond
(coll? elem)
(if op
[(eval-op op v (first (evaluate elem))) nil]
[(first (evaluate elem)) nil])
(and op (number? elem))
[(eval-op op v elem) nil]
(number? elem)
[elem nil]
(symbol? elem)
[v elem]
:else
(throw (ex-info "Invalid evaluation" {:v v :op op :elem (type elem)}))))
[0 nil]
e)))
(first (evaluate (clojure.edn/read-string "(1 * 2 + 3)")))
=> 5
(first (evaluate (clojure.edn/read-string "(1 * 2 + (3 * 5))")))
=> 17
This requires the input string to represent a valid Clojure list. I also had this function for grouping multiplication/division:
(defn pemdas
"Groups division/multiplication operations in e into lists."
[e]
(loop [out []
rem e]
(if (empty? rem)
(seq out)
(let [curr (first rem)
next' (second rem)]
(if (contains? #{'/ '*} next')
(recur (conj out (list curr next' (nth rem 2)))
(drop 3 rem))
(recur (conj out curr) (rest rem)))))))
(pemdas '(9.87 + 4 / 3 * 0.41))
=> (9.87 + (4 / 3) * 0.41)
This exact problem is why I first created the Tupelo Forest library.
Please see the talk from Clojure Conj 2017.
I've started some docs here. You can also see live examples here.
Update
Here is how you could use the Tupelo Forest library to do it:
First, define your Abstract Syntax Tree (AST) data using Hiccup format:
(with-forest (new-forest)
(let [data-hiccup [:rpc
[:fn {:type :+}
[:value 2]
[:value 3]]]
root-hid (add-tree-hiccup data-hiccup)
with result:
(hid->bush root-hid) =>
[{:tag :rpc}
[{:type :+, :tag :fn}
[{:tag :value, :value 2}]
[{:tag :value, :value 3}]]]
Show how walk-tree works using a "display interceptor"
disp-interceptor {:leave (fn [path]
(let [curr-hid (xlast path)
curr-node (hid->node curr-hid)]
(spyx curr-node)))}
>> (do
(println "Display walk-tree processing:")
(walk-tree root-hid disp-interceptor))
with result:
Display walk-tree processing:
curr-node => {:tupelo.forest/khids [], :tag :value, :value 2}
curr-node => {:tupelo.forest/khids [], :tag :value, :value 3}
curr-node => {:tupelo.forest/khids [1037 1038], :type :+, :tag :fn}
curr-node => {:tupelo.forest/khids [1039], :tag :rpc}
then define the operators and an interceptor to transform a subtree like (+ 2 3) => 5
op->fn {:+ +
:* *}
math-interceptor {:leave (fn [path]
(let [curr-hid (xlast path)
curr-node (hid->node curr-hid)
curr-tag (grab :tag curr-node)]
(when (= :fn curr-tag)
(let [curr-op (grab :type curr-node)
curr-fn (grab curr-op op->fn)
kid-hids (hid->kids curr-hid)
kid-values (mapv hid->value kid-hids)
result-val (apply curr-fn kid-values)]
(set-node curr-hid {:tag :value :value result-val} [])))))}
] ; end of let form
; imperative step replaces old nodes with result of math op
(walk-tree root-hid math-interceptor)
We can then display the modified AST tree which contains the result of (+ 2 3):
(hid->bush root-hid) =>
[{:tag :rpc}
[{:tag :value, :value 5}]]
You can see the live code here.
(Apologies if this is a duplicate of another question, my search for all those fancy special characters didn't yield anything.)
I'm reading Mastering Clojure Macros and have trouble understanding the following example:
(defmacro inspect-caller-locals []
(->> (keys &env)
(map (fn [k] [`'~k k]))
(into {})))
=> #'user/inspect-caller-locals
(let [foo "bar" baz "quux"]
(inspect-caller-locals))
=> {foo "bar", baz "quux"}
What is the difference between the following and the much simpler 'k?
`'~k
As far as I understand, the innermost unquote ~ should simply reverts the effect of the outermost syntax-quote `, but a short experiment reveals that there's more to it:
(defmacro inspect-caller-locals-simple []
(->> (keys &env)
(map (fn [k] ['k k]))
(into {})))
=> #'user/inspect-caller-locals-simple
(let [foo "bar" baz "quux"]
(inspect-caller-locals-simple))
CompilerException java.lang.RuntimeException: Unable to resolve symbol: k in this context, compiling:(/tmp/form-init4400591386630133028.clj:2:3)
Unfortunately, my usual investigation approach doesn't apply here:
(macroexpand '(let [foo "bar" baz "quux"]
(inspect-caller-locals)))
=> (let* [foo "bar" baz "quux"] (inspect-caller-locals))
(let [foo "bar" baz "quux"]
(macroexpand '(inspect-caller-locals)))
=> {}
What am I missing here?
Let's first establish what the k inside the macro is:
(defmacro inspect-caller-locals []
(mapv (fn [k]
(println (class k)))
(keys &env))
nil)
(let [x 1]
(inspect-caller-locals))
;; Prints:
;; clojure.lang.Symbol
So you each k inside the function is a symbol. If you return a symbol from a macro (ie generate code from it), clojure will lookup the value that it refers to and print it. For instance you could do this:
(defmacro inspect-caller-locals []
(mapv (fn [k]
[(quote x) k]) ;; not the "hard coded" x
(keys &env)))
(let [x 1]
(inspect-caller-locals))
;; Prints:
;; [[1 1]]
What you want however is the actual symbol. The problem (as you noted) is that quote is a special form that DOES NOT EVALUTE whatever you pass it. Ie, the k will not obtain the function parameter but stay k which is not usually defined:
(defmacro inspect-caller-locals []
(mapv (fn [k]
[(quote k) k])
(keys &env)))
(let [x 1]
(inspect-caller-locals))
;; => Error
(let [k 1]
(inspect-caller-locals))
;; Prints:
;; [[1 1]]
You somehow need to evaluate what you pass into quote, this is not however possible since that isn't what quote does. Other functions, such as str don't have that problem:
(defmacro inspect-caller-locals []
(mapv (fn [k]
[(str k) k])
(keys &env)))
(let [x 1]
(inspect-caller-locals))
;; Prints:
;; [["x" 1]]
The trick is to go one level deeper and quote the quote itself so you can pass the symbol to it:
(defmacro inspect-caller-locals []
(mapv (fn [k]
[;; This will evaluate k first but then generate code that
;; wraps that symbol with a quote:
(list (quote quote) k)
;; Or equivalently and maybe easier to understand:
(list 'quote k)
k])
(keys &env)))
(let [x 1]
(inspect-caller-locals))
;; Prints:
;; [[x x 1]]
Or by using the reader that can do this for you:
(defmacro inspect-caller-locals []
(mapv (fn [k]
[`(quote ~k)
`'~k
k])
(keys &env)))
(let [x 1]
(inspect-caller-locals))
;; Prints:
;; [[x x 1]]
Because after all:
(read-string "`'~k")
=> (clojure.core/seq (clojure.core/concat (clojure.core/list (quote quote)) (clojure.core/list k)))
(defmacro inspect-caller-locals []
(mapv (fn [k]
[(clojure.core/seq (clojure.core/concat (clojure.core/list (quote quote)) (clojure.core/list k)))
k])
(keys &env)))
(let [x 1]
(inspect-caller-locals))
;; Prints:
;; [[x 1]]
Some alternative, and equivalent, ways of writing
`'~k
are:
`(quote ~k) ;; expands the ' reader macro to the quote special form
(list 'quote k) ;; avoids syntax quote entirely
You are pretty much right to think that
the innermost unquote ~ should simply reverts the effect of the outermost syntax-quote
The only thing missing from your description there is that you can't pull quote outside of a syntax-quoted expression, since quote is a special form and changes the meaning of what's inside. Otherwise,
'`~k
would be equivalent to 'k - and as you noticed, it's not!
I'll echo #amalloy's general advice, that trying syntax-quoted stuff in the REPL, outside of the context of macros/macroexpansion, is the best way to get your head around these things.
p.s. Also, I'll make a note that I need to fix this confusion by explaining better in a future book edition ;)
How do I modify the :arglist attribute for a clojure fn or macro?
(defn tripler ^{:arglists ([b])} [a] (* 3 a))
(defn ^{:arglists ([b])} quadrupler [a] (* 4 a))
% (meta #'tripler) =>
{:arglists ([a]), :ns #<Namespace silly.testing>, :name tripler, :line 1, :file "NO_SOURCE_PATH"}
% (meta #'quadrupler) =>
{:arglists ([a]), :ns #<Namespace silly.testing>, :name quadrupler, :line 1, :file "NO_SOURCE_PATH"}
Ok, no luck there, so I tried doing the following.
(def tripler
(with-meta trippler
(assoc (meta #'tripler) :arglists '([c]))))
% (with-meta #'tripler) =>
{:ns #<Namespace silly.testing>, :name tripler, :line 1, :file "NO_SOURCE_PATH"}
Hmm, so now the :arglists key is gone? Well, I give up, how do I do this? I would simply like to modify the value of :arglists. The examples above use defn, but I would also like to know how to set the :arglists using a macro (defmacro).
You don't need to do anything as ugly as the suggestions so far. If you take a look at defn's own arglists…
user=> (:arglists (meta #'clojure.core/defn))
([name doc-string? attr-map? [params*] prepost-map? body]
[name doc-string? attr-map? ([params*] prepost-map? body) + attr-map?])
You're looking for attr-map. Here's an example.
user=> (defn foo
"does many great things"
{:arglists '([a b c] [d e f g])}
[arg] arg)
#'user/foo
user=> (doc foo)
-------------------------
user/foo
([a b c] [d e f g])
does many great things
nil
(In that case, arglists is a total lie. Don't do that!)
alter-meta! changes the metadata on a var. The metadata on the function is not relevant, only the var.
(alter-meta! #'tripler assoc :arglists '([b]))
defn does not leave room to mangle the metadata which is OK because it's just a macro that wraps def. You can use def directly instead of defn:
core> (def ^{:arglists '([b])} tripler (fn [a] (* 3 a)))
#'core/tripler
core> (meta #'tripler)
{:arglists ([b]), :ns #<Namespace autotestbed.core>, :name tripler, :line 1, :file "NO_SOURCE_FILE"}
or you define the var tripler with defn:
core> (defn tripler [a] (* 3 a))
#'autotestbed.core/tripler
then redefine the var with the same contents and different metadata:
core> (def ^{:arglists '([b])} tripler tripler)
#'autotestbed.core/tripler
autotestbed.core> (meta #'tripler)
{:arglists ([b]), :ns #<Namespace autotestbed.core>, :name tripler, :line 1, :file "NO_SOURCE_FILE"}
Expanding on amalloy's answer (please give him credit):
user=> (defn foo "prints bar" [] (println "bar"))
#'user/foo
user=> (doc foo)
-------------------------
user/foo
([])
prints bar
nil
user=> (meta #'foo)
{:arglists ([]), :ns #<Namespace user>, :name foo, :doc "prints bar", :line 1, :file "NO_SOURCE_PATH"}
user=> (alter-meta! #'foo assoc :arglists '([blah]))
{:arglists ([blah]), :ns #<Namespace user>, :name foo, :doc "prints bar", :line 1, :file "NO_SOURCE_PATH"}
user=> (doc foo)
-------------------------
user/foo
([blah])
prints bar
nil
user=> (meta #'foo)
{:arglists ([blah]), :ns #<Namespace user>, :name foo, :doc "prints bar", :line 1, :file "NO_SOURCE_PATH"}
user=> (foo)
bar
nil
Sneaky!
I would like to see if a symbol has been "def" ed, but I can't see any ifdef syntax
user> (resolve 'foo)
nil
user> (def foo 3)
#'user/foo
user> (resolve 'foo)
#'user/foo
resolve or ns-resolve may do what you're looking for:
user> (def a 1)
#'user/a
user> (def b)
#'user/b
user> (resolve 'a)
#'user/a
user> (resolve 'b)
#'user/b
user> (resolve 'c)
nil
To get a boolean:
user> (boolean (resolve 'b))
true
EDIT: per MayDaniel's comment, this isn't exactly what you asked for, but it will get you there. Here's an implementation of bounded? (probably not the best name):
(defn bounded? [sym]
(if-let [v (resolve sym)]
(bound? v)
false))
user> (map bounded? ['a 'b 'c])
(true false false)
Can use find-var for this
(bound? (find-var 'user/y))