I'm trying to write a macro which will catch a compile time error in Clojure. Specifically, I would like to catch exceptions thrown when a protocol method, which has not been implemented for that datatype, is called and clojure.lang.Compiler$CompilerException is thrown.
So far I have:
(defmacro catch-compiler-error
[body]
(try
(eval body)
(catch Exception e e)))
But of course, I've been told that eval is evil and that you don't typically need to use it. Is there a way to implement this without using eval?
I'm inclined to believe that eval is appropriate here since I specifically want the code to be evaluated at runtime and not at compile time.
Macros are expanded at compile time. They don't need to eval code; rather, they assemble the code that will be later be evaluated at runtime. In other words, if you want to make sure that the code passed to a macro is evaluated at runtime and not at compile time, that tells you that you absolutely should not eval it in the macro definition.
The name catch-compiler-error is a bit of a misnomer with that in mind; if the code that calls your macro has a compiler error (a missing parenthesis, perhaps), there's not really anything your macro can do to catch it. You could write a catch-runtime-error macro like this:
(defmacro catch-runtime-error
[& body]
`(try
~#body
(catch Exception e#
e#)))
Here's how this macro works:
Take in an arbitrary number of arguments and store them in a sequence called body.
Create a list with these elements:
The symbol try
All the expressions passed in as arguments
Another list with these elements:
The symbol catch
The symbol java.lang.Exception (the qualified version of Exception)
A unique new symbol, which we can refer to later as e#
That same symbol that we created earlier
This is a bit much to swallow all at once. Let's take a look at what it does with some actual code:
(macroexpand
'(catch-runtime-error
(/ 4 2)
(/ 1 0)))
As you can see, I'm not simply evaluating a form with your macro as its first element; that would both expand the macro and evaluate the result. I just want to do the expansion step, so I'm using macroexpand, which gives me this:
(try
(/ 4 2)
(/ 1 0)
(catch java.lang.Exception e__19785__auto__
e__19785__auto__))
This is indeed what we expected: a list containing the symbol try, our body expressions, and another list with the symbols catch and java.lang.Exception followed by two copies of a unique symbol.
You can check that this macro does what you want it to do by directly evaluating it:
(catch-runtime-error (/ 4 2) (/ 1 0))
;=> #error {
; :cause "Divide by zero"
; :via
; [{:type java.lang.ArithmeticException
; :message "Divide by zero"
; :at [clojure.lang.Numbers divide "Numbers.java" 158]}]
; :trace
; [[clojure.lang.Numbers divide "Numbers.java" 158]
; [clojure.lang.Numbers divide "Numbers.java" 3808]
; ,,,]}
Excellent. Let's try it with some protocols:
(defprotocol Foo
(foo [this]))
(defprotocol Bar
(bar [this]))
(defrecord Baz []
Foo
(foo [_] :qux))
(catch-runtime-error (foo (->Baz)))
;=> :qux
(catch-runtime-error (bar (->Baz)))
;=> #error {,,,}
However, as noted above, you simply can't catch a compiler error using a macro like this. You could write a macro that returns a chunk of code that will call eval on the rest of the code passed in, thus pushing compile time back to runtime:
(defmacro catch-error
[& body]
`(try
(eval '(do ~#body))
(catch Exception e#
e#)))
Let's test the macroexpansion to make sure this works properly:
(macroexpand
'(catch-error
(foo (->Baz))
(foo (->Baz) nil)))
This expands to:
(try
(clojure.core/eval
'(do
(foo (->Baz))
(foo (->Baz) nil)))
(catch java.lang.Exception e__20408__auto__
e__20408__auto__))
Now we can catch even more errors, like IllegalArgumentExceptions caused by trying to pass an incorrect number of arguments:
(catch-error (bar (->Baz)))
;=> #error {,,,}
(catch-error (foo (->Baz) nil))
;=> #error {,,,}
However (and I want to make this very clear), don't do this. If you find yourself pushing compile time back to runtime just to try to catch these sorts of errors, you're almost certainly doing something wrong. You'd be much better off restructuring your project so that you don't have to do this.
I'm guessing you've already seen this question, which explains some of the pitfalls of eval pretty well. In Clojure specifically, you definitely shouldn't use it unless you completely understand the issues it raises regarding scope and context, in addition to the other problems discussed in that question.
Related
Recently I came across a use for eval within a macro, which I understand is a bit of a faux pas but let's ignore that for now. What I found surprising, was that eval was able to resolve global vars at macroexpansion time. Below is a contrived example, just to illustrate the situation I'm referring to:
(def list-of-things (range 10))
(defmacro force-eval [args]
(apply + (eval args)))
(macroexpand-1 '(force-eval list-of-things))
; => 45
I would have expected args to resolve to the symbol list-of-things inside force-eval, and then list-of-things to be evaluated resulting in an error due to it being unbound:
"unable to resolve symbol list-of-things in this context"
However, instead list-of-things is resolved to (range 10) and no error is thrown - the macroexpansion succeeds.
Contrast this with attempting to perform the same macroexpansion, but within a local binding context:
(defmacro force-eval [args]
(apply + (eval args)))
(let [list-of-things (range 10)]
(macroexpand-1 '(force-eval list-of-things)))
; => Unable to resolve symbol: list-of-thingss in this context
Note in the above examples I'm assuming list-of-things is not previously bound, e.g. a fresh REPL. One final example illustrates why this is important:
(defmacro force-eval [args]
(apply + (eval args)))
(def list-of-things (range 10 20))
(let [list-of-thing (range 10)]
(macroexpand-1 '(force-eval list-of-things)))
; => 145
The above example shows that the locals are ignored, which is expected behavior for eval, but is a bit confusing when you are expecting the global to not be available at macroexpansion time either.
I seem to have a misunderstanding about what exactly is available at macroexpansion time. I had previously thought that essentially any binding, be it global or local, would not be available until runtime. Apparently this is an incorrect assumption. Is the answer to my confusion simply that global vars are available at macroexpansion time? Or am I missing some further nuance here?
Note: this related post closely describes a similar problem, but the focus there is more on how to avoid inappropriate use of eval. I'm mainly interested in understanding why eval works in the first example and by extension what's available to eval at macroexpansion time.
Of course, vars must be visible at compile time. That's where functions like first and + are stored. Without them, you couldn't do anything.
But keep in mind that you have to make sure to refer to them correctly. In the repl, *ns* will be bound, and so a reference to a symbol will look in the current namespace. If you are running a program through -main instead of the repl, *ns* will not be bound, and only properly qualified vars will be found. You can ensure that you qualify them correctly by using
`(force-eval list-of-things)
instead of
'(force-eval list-of-things)
Note I do not distinguish between global vars and non-global vars. All vars in Clojure are global. Local bindings are not called vars. They're called locals, or bindings, or variables, or some combination of those words.
Clojure is designed with an incremental compilation model. This is poorly documented.
In C and other traditional languages, source code must be compiled, then linked with pre-compiled libraries before the final result can be executed. Once execution begins, no changes to the code can occur until the program is terminated, when new source code can be compiled, linked, then executed. Java is normally used in this manner just like C.
With the Clojure REPL, you can start with zero source code in a live executing environment. You can call existing functions like (+ 2 3), or you can define new functions and variables on the fly (both global & local), and redefine existing functions. This is only possible because core Clojure is already available (i.e. clojure.core/+ etc is already "installed"), so you can combine these functions to define your own new functions.
The Clojure "compiler" works just like a giant REPL session. It reads and evaluates forms from your source code files one at a time, incrementally adding them the the global environment. Indeed, it is a design goal/requirement that the result of compiling and executing source code is identical to what would occur if you just pasted each entire source code file into the REPL (in proper dependency order).
Indeed, the simplest mental model for code execution in Clojure is to pretend it is an interpreter instead of a traditional compiler.
And eval in a macro makes no sense.
Because:
a macro already implicitely contains an eval
at the very final step.
If you use macroexpand-1, you make visible how the code was manipulated in the macro before the evocation of the implicite eval inside the macro.
An eval in a macro is an anti-pattern which might indicate that you should use a function instead of a macro - and in your examle this is exactly the case.
So your aim is to dynamically (in run-time) evoke sth in a macro. This you can only do through an eval applied over a macro call OR you should rather use a function.
(defmacro force-eval [args]
(apply + (eval args)))
;; What you actually mean is:
(defn force-eval [args]
(apply + args))
;; because a function in lisp evaluates its arguments
;; - before applying the function body.
;; That means: args in the function body is exactly
;; `(eval args)`!
(def list-of-things (range 10))
(let [lit-of-things (range 10 13)]
(force-eval list-of-things))
;; => 45
;; so this is exactly the behavior you wanted!
The point is, your construct is a "bad" example for a macro.
Because apply is a special function which allows you to
dynamically rearrange function call structures - so it has
some magic of macros inside it - but in run-time.
With apply you can do quite some meta programming in some cases when you just quote some of your input arguments.
(Try (force-eval '(1 2 3)) it returns 6. Because the (1 2 3) is put together with + at its front by apply and then evaluated.)
The second point - I am thinking of this answer I once gave and this to a dynamic macro call problem in Common Lisp.
In short: When you have to control two levels of evaluations inside a macro (often when you want a macro inject some code in runtime into some code), you need too use eval when calling the macro and evaluate those parts in the macro call which then should be processed in the macro.
I've been into Clojure lately and have avoided macros up until now, so this is my first exposure to them. I've been reading "Mastering Clojure Macros", and on Chapter 3, page 28, I encountered the following example:
user=> (defmacro square [x] `(* ~x ~x))
;=> #'user/square
user=> (map (fn [n] (square n)) (range 10))
;=> (0 1 4 9 16 25 36 49 64 81)
The context is the author is explaining that while simply passing the square macro to map results in an error (can't take value of a macro), wrapping it in a function works because:
when the anonymous function (fn [n] (square n)) gets compiled, the
square expression gets macroexpanded, to (fn [n] (clojure.core/* n
n)). And this is a perfectly reasonable function, so we don’t have any
problems with the compiler
This makes sense to me if we assume the body of the function is evaluated before runtime (at compile, or "definition" time) thus expanding the macro ahead of runtime. However, I always thought that function bodys were not evaluated until runtime, and at compile time you would basically just have a function object with some knowledge of it's lexical scope (but no knowledge of its body).
I'm clearly mixed up on the compile/runtime semantics here, but when I look at this sample I keep thinking that square won't be expanded until map forces it's call, since it's in the body of the anonymous function, which I thought would be unevaluated until runtime. I know my thinking is wrong, because if that was the case, then n would be bound to each number in (range 10), and there wouldn't be an issue.
I know it's a pretty basic question, but macros are proving to be pretty tricky for me to fully wrap my head around at first exposure!
Generally speaking function bodies aren't evaluated at compile time, but macros are always evaluated at compile time because they're always expanded at compile time whether inside a function or not.
You can write a macro that expands to a function, but you still can't refer/pass the macro around as if it were a function:
(defmacro inc-macro [] `(fn [x#] (inc x#)))
=> #'user/inc-macro
(map (inc-macro) [1 2 3])
=> (2 3 4)
defmacro is expanded at compile time, so you can think of it as a function executed during compilation. This will replace every occurrence of the macro "call" with the code it "returns".
You may consider a macros as a syntax rule. For example, in Scheme, a macros is declared with define-syntax form. There is a special step in compiler that substitutes all the macros calls into their content before compiling the code. As a result, there won't be any square calls in your final code. Say, if you wrote something like
(def value (square 3))
the final version after expansion would be
(def value (clojure.core/* 3 3))
There is a special way to check what will be the body of your macros after being expanded:
user=> (defmacro square [x] `(* ~x ~x))
#'user/square
user=> (macroexpand '(square 3))
(clojure.core/* 3 3)
That's why a macros is an ephemeral thing that lives only in source code but not in the compiled version of it. That's why it cannot be passed as a value or referenced somehow.
The best rule regarding macroses is: try to avoid them until you really need them in your work.
What is the idiomatic way of printing values inside a let binding ?
When I started developing in Clojure, I wrote code in the REPL, that I then turned into simple let expressions. Being a beginner, I often made mistakes during this (simple) transformation phase.
(let [a (aFn ...)
b (bFn ... a)]
;; error above
)
So I would transform it back to something like that, basically inlining things :
(println "a is" (aFn ...))
(println "b is" (bFn ... (aFn ...)))
(let [a (aFn ...)
b (bFn ... a)]
;; ...
)
It works most of the time thanks to Clojure being nice (immutability, referential transparency..).
Now I do something along the lines of :
(let [a (aFn ...)
_ (println "a is" a)
b (bFn ... a)
_ (println "b is" b)]
;; ...
)
It is an improvement, but it still feels clumsy. What is the proper way to do this ?
You could define a print function that returns its argument:
(defn cl-print [x] (doto x (print)))
Then, it is only a matter of wrapping your expressions:
(let [a (cl-print (aFn ...))
b (cl-print (bFn ... a))]
...)
I tend to take a totally different approach. I never put print statements in my let bindings. I also think you need to be careful about calling a function just to get the value for debugging purposes. While we would like all our functions to be side-effect free, this is not always the case, so calling the funciton just to get a value to print may have unexpected results. There is also the issue of how printing values can impact on laziness and realising of lazy sequences etc.
My approach is to define some debugging functions, which I stick in a 'debug' namespace. I then call these debug functions when needed from inside the body of the function - not in the let binding section. Often, I also define a debug-level var so that I can have some control over debugging verbosity. this allows me to change one var and increase or decrease the amount of information logged/printed.
I've experimented with 'clever' macros to make debugging easier - but to be honest, these usually take more effort to get right than the benefit they provide.
I like having my debug functions in a separate namespace as this helps me ensure I've not left any debugging code in my production version - or it allows me to leave debug statements in there, but have them 'do nothing' by setting an appropriate debug level.
As mentioned by another post, using a debugger can eliminate/reduce the need to have these print statements or debug functions. However, I think debuggers can be a double edged sword as well. Too often, people get into bad debugging hapits where they rely on trace and inspect rather than thinking about and analysing exactly what is going on. This can tend to development driven by too much trial and error and not enough analysis and understanding.
You could start with something as simple as
(def debug-level 20)
(defn debug [lvl prefix val]
(if (>= lvl debug-level)
(println (str prefix ": " val)))
(defn debug1 [prefix v]
(debug 10 prefix v))
(defn debug2 [prefix v]
(debug 20 prefix v))
etc
and then just call
(debug2 :a a)
in the body of your function to have the value of a printed when debug-level is 20 or higher.
How do I programmatically figure out which Vars may affect the results of a function defined in Clojure?
Consider this definition of a Clojure function:
(def ^:dynamic *increment* 3)
(defn f [x]
(+ x *increment*))
This is a function of x, but also of *increment* (and also of clojure.core/+(1); but I'm less concerned with that). When writing tests for this function, I want to make sure that I control all relevant inputs, so I do something like this:
(assert (= (binding [*increment* 3] (f 1)) 4))
(assert (= (binding [*increment* -1] (f 1)) 0))
(Imagine that *increment* is a configuration value that someone might reasonably change; I don't want this function's tests to need changing when this happens.)
My question is: how do I write an assertion that the value of (f 1) can depend on *increment* but not on any other Var? Because I expect that one day someone will refactor some code and cause the function to be
(defn f [x]
(+ x *increment* *additional-increment*))
and neglect to update the test, and I would like to have the test fail even if *additional-increment* is zero.
This is of course a simplified example – in a large system, there can be lots of dynamic Vars, and they can get referenced through a long chain of function calls. The solution needs to work even if f calls g which calls h which references a Var. It would be great if it didn't claim that (with-out-str (prn "foo")) depends on *out*, but this is less important. If the code being analyzed calls eval or uses Java interop, of course all bets are off.
I can think of three categories of solutions:
Get the information from the compiler
I imagine the compiler does scan function definitions for the necessary information, because if I try to refer to a nonexistent Var, it throws:
user=> (defn g [x] (if true x (+ *foobar* x)))
CompilerException java.lang.RuntimeException: Unable to resolve symbol: *foobar* in this context, compiling:(NO_SOURCE_PATH:24)
Note that this happens at compile time, and regardless of whether the offending code will ever be executed. Thus the compiler should know what Vars are potentially referenced by the function, and I would like to have access to that information.
Parse the source code and walk the syntax tree, and record when a Var is referenced
Because code is data and all that. I suppose this means calling macroexpand and handling each Clojure primitive and every kind of syntax they take. This looks so much like a compilation phase that it would be great to be able to call parts of the compiler, or somehow add my own hooks to the compiler.
Instrument the Var mechanism, execute the test and see which Vars get accessed
Not as complete as the other methods (what if a Var is used in a branch of the code that my test fails to exercise?) but this would suffice. I imagine I would need to redefine def to produce something that acts like a Var but records its accesses somehow.
(1) Actually that particular function doesn't change if you rebind +; but in Clojure 1.2 you can bypass that optimization by making it (defn f [x] (+ x 0 *increment*)) and then you can have fun with (binding [+ -] (f 3)). In Clojure 1.3 attempting to rebind + throws an error.
Regarding your first point you could consider using the analyze library. With it you can quite easily figure out which dynamic vars are used in an expression:
user> (def ^:dynamic *increment* 3)
user> (def src '(defn f [x]
(+ x *increment*)))
user> (def env {:ns {:name 'user} :context :eval})
user> (->> (analyze-one env src)
expr-seq
(filter (op= :var))
(map :var)
(filter (comp :dynamic meta))
set)
#{#'user/*increment*}
I know that this doesn't answer your question, but wouldn't it be a lot less work to just provide two versions of a function where one version has no free variables, and the other version calls the first one with the appropriate top-level defines?
For example:
(def ^:dynamic *increment* 3)
(defn f
([x]
(f x *increment*))
([x y]
(+ x y)))
This way you can write all your tests against (f x y), which doesn't rely on any global state.
All, I'm starting to take a look at the Clojure language, and had a couple questions about something I'm trying to do. The broad objective is to alias the sequence function every? to all?. I'm sure there's a function or macro that does alias-ing (or something along those lines) but I wanted to see if it was possible with some of the basic constructs I know thus far. My approach was going to be to define a function called all? that applies its arguments to the every? implementation.
I'm curious to see if this can be made agnostic, so I wanted to parameter my alias function to take two arguments, the new name (as a Keyword) and the old name (as a function reference). In striving towards this goal, I've encountered two problems.
1) Defining named functions with Keywords throws errors. Apparently it wants clojure.lang.IObj.
user=> (defn :foo "bar")
java.lang.ClassCastException: clojure.lang.Keyword cannot be cast to clojure.lang.IObj (NO_SOURCE_FILE:0)
Is there a function to cast a Keyword to an IObj, or other means to parameterize the name of a newly defined function with some provided value? (In Ruby, define_method amongst other techniques does this)
irb(main)> self.class.instance_eval do
irb(main)* define_method(:foo) { "bar" }
irb(main)> end
=> #<Proc>
irb(main)> foo
=> "bar"
2) Collect all arguments to a function into a single variable. Even basic functions such as (+ 1 2 3 4) take a variable amount of arguments. All the function definition techniques I've seen so far take a specific amount of arguments, with no way to just aggregate everything in a list for handling in the function body. Once again, what I'm going for is done in Ruby like so:
irb(main)> def foo(*args)
irb(main)> p args
irb(main)> end
=> nil
irb(main)> foo(1, 2, 3)
[1, 2, 3]
=> nil
Thanks for any help you can provide me!
I'll answer in bullet points, since the questions can be split neatly into a number of separate issues.
Something which is implicitly contained in what is to follow, but which perhaps warrants a bullet of its own: the top-level objects created by def & Co. (and in particular by defn) are Vars. So what you actually want to do is to alias a Var; functions are just regular values which don't really have names (except they may have a name bound to themselves locally inside their bodies; that's nothing to do with the issue at hand, though).
There is indeed an "aliasing macro" available in Clojure -- clojure.contrib.def/defalias:
(use '[clojure.contrib.def :only [defalias]])
(defalias foo bar)
; => foo can now be used in place of bar
The advantage of this over (def foo bar) is that it copies over metadata (such as the docstring); it even appears to work with macros in the current HEAD, although I recall a bug which prevented that in earlier versions.
Vars are named by symbols, not keywords. Symbol literals in Clojure (and other Lisps) do not start with colons (:foo is a keyword, not a symbol). Thus to define a function called foo you should write
(defn foo [...] ...)
defn is a helper macro easing the creation of new function-holding Vars by allowing the programmer to use a mix of def & fn syntax. So defn is out of question for creating Vars with preexisting values (which might be functions), as is required for creating aliases; use defalias or simply def instead.
To create a variadic function, use the following syntax:
(fn [x y & args] ...)
x and y will be required positional arguments; the rest of the arguments passed to the function (any number of them) will be collected into a seq and available under the name args. You don't have to specify any "required positional arguments" if they are not needed: (fn [& args] ...).
To create a Var holding a variadic function, use
(defn foo [x y & args] ...)
To apply a function to some arguments you've got assembled into a seqable object (such as the args seq in the above examples or perhaps a vector &c.), use apply:
(defn all? [& args]
(apply every? args))
If you want to write a function to create aliases -- as opposed to a macro -- you'll need to investigate the functions intern, with-meta, meta -- and possibly resolve / ns-resolve, depending on whether the function is to accept symbols or Vars. I'll leave filling in the details as an exercise to the reader. :-)
All you need to do is bind the every? function to the all? symbol, which is done via def:
(def all? every?)
For a bit more on this, see Clojure macro to create a synonym for a function
Don't think I can add much to the existing explanations here, except perhaps fill out a couple of blanks in the Ruby traveller's dictionary on argument collection and destructuring:
(defn foo [& args] ; Ruby: def foo(*args)
(println args))
user=> (foo 1 2 3)
(1 2 3)
(defn foo [& args]
(+ args))
user=> (foo 1 2 3)
java.lang.ClassCastException ; + takes numbers, not a list
(defn foo [& args]
(apply + args)) ; apply: as Ruby proc.call(*args)
user=> (foo 1 2 3)
6
(defn foo [& args]
(let [[a b & other] args] ; Ruby: a, b, *other = args
(println a b other)))
user=> (foo 1 2 3)
1 2 (3)