Given the following in one file:
(ns demo.first)
(defmacro ^:private private-macro [a] a)
And the following in another file:
(ns demo.second
(:require [demo.first :as first]))
(first/private-macro 10)
The call to the private macro in demo.second will throw: var: #'demo.first/private-macro is not public, as I expect.
Now, is there a way to have this call succeed, without making the macro public?
For functions, I can do:
(#'first/private-macro 10)
But with a macro, it throws: Wrong number of args (1) passed to: first/private-macro.
I'm looking to unit test this private macro, and personally prefer using private meta over an impl namespace. Which is why I'm hoping there's a solution to this.
Thank You.
UPDATE:
I found out that since defmacro is itself a macro, it first expands into a form which creates the symbol and Var for the macro and add its metadata to it.
Thus:
(defmacro ^:private private-macro [a] a)
First is processed by the defmacro macro, and expanded into:
(do
(clojure.core/defn ^{:private true} private-macro
([&form &env a] a))
(. (var ^{:private true} private-macro)
^{:line 487, :column 49}
(setMacro))
(var ^{:private true} private-macro))
As you can see, what then happens is that:
A private-macro fn is declared with defn, and set to private.
This function takes 3 arguments [&form &env a]. This is why we get the wrong number of argument (1) exception when using #' to call the macro.
The private-macro var is set as a macro by calling its setMacro method.
The private-macro var is returned.
In essence, what is happening is that if you call the function pointed to by the private-macro var, such as is the case when using the (#'private-macro) syntax, you're actually calling the function you see above, which takes 3 arguments. If your macro itself took more than one argument, that function would take 2 + the number of args of your macro.
So I still don't know how to call a private macro:
At first I thought stubbing out &form and &env with nils would work:
(#'first/private-macro nil nil 10)
And for my simple macro above it does, and return 10. But on more complicated macros, which need to be expanded further, it doesn't, and instead I get the macro-expansion returned to me ?!?
Then I thought I could use alter-meta! to remove the private meta from the macro temporarily before calling it. As such:
(alter-meta! #'first/private-macro
(fn [meta] (dissoc meta :private)))
(first/private-macro 10)
(alter-meta! #'first/private-macro
(fn [meta] (assoc meta :private true)))
But this only works at the REPL. Try to compile your code afterwards, and it seems the Compiler itself will throw the var: #'demo.first/private-macro is not public error, even before the alter-meta! has a chance to run, thus failing compilation.
I don't really know why #' doesn't work the same as a normal call to the macro, and why passing nil to the &form and &env doesn't work for all macros. And how to make alter-meta! work at compile time. So if someone does know, please answer away!
And for my simple macro above it does, and return 10. But on more complicated macros, which need to be expanded further, it doesn't, and instead I get the macro-expansion returned to me ?!?
Yes. As you discovered, when you write (defmacro m [x] (list x x)), you:
Define a function m that consumes forms as input and produces forms as output
Tell the compiler to look for calls like (m a) and replace them with the result of calling your m function at compile time
By calling #'m instead, you bypass step 2: there is no call to the macro m, and so the compiler does not call it at compile time or replace the calling code with the result. Since #'m is just a regular function which takes code as input and produces code, when you bypass the special compiler behavior and call it at runtime, you of course get code as a result (which you can't do much with because it's runtime already).
Good news, though: there's rarely a compelling reason to make a macro private anyway, since it can do no harm to let other namespace call it. All the private macro does is expand into code the client could have written by hand anyway. So, if you control this macro, you should probably just make it public. If you don't, then you can just write whatever code the macro would have written for you.
If you absolutely insist on calling someone else's private macro, then you can split the parts (1) and (2) up, in a way: define your own macro whose implementation delegates to the function backing the private var in the other namespace:
(defmacro cheat [& args]
(apply #'m &form &env args))
Because cheat is your own macro, you can call it in the usual way, engaging the compiler's "call this at compile time" mechanism. Then you delegate to the function that generates the code you want, passing &form and &env explicitly.
Related
I'm trying a coding challenge that requires you to create code that compiles infinitely.
My first thought was a macro that expands to itself forever. I wrote up:
(defmacro a []
(a))
(a)
This doesn't actually produce anything, but I expected it to loop forever. Instead however, I get a nonsensical arity exception:
Wrong number of args (-2) passed to: infinite-compile/a, compiling:...
If I try to give it an argument for kicks, it now complains that the macro doesn't expect any arguments.
If I have it actually produce calls to itself:
(defmacro a []
`(a))
(a)
It fails with a StackOverflow, which I expected.
What's going on here? Why does it think I'm passing the macro "-2" arguments? The only possible thing I could think of is it has something to do with the 2 implicit & arguments that are passed to macros, but that's just a shot in the dark and doesn't actually explain what's going on.
To address the answer, this doesn't appear to be an issue with a multi-arity macro since it only has a 0-arity version. Also, explicitly passing the implicit arguments doesn't do anything:
(defmacro a []
(a &form &env))
This yields:
Compiler Exception clojure.lang.ArityException: Wrong number of args (2) passed to: infinite-compile/a, compiling:
You are expecting (a) to be macroexpanded during the definition of a, when a is not yet known to be a macro (and thus expected to be a function), whereas when you quote the form, you are effectively building a macro which expands into a call to itself.
When the compiler macroexpands a macro, it adds the implicit arguments before calling the function associated with the macro: Compiler.java#L6795. Here, you are directly calling a, which is in the scope of the implicit defn (core.clj#L452), without passing the necessary arguments.
I would expect the following to work as you wish (loop):
user=> (defmacro a[]&form)
#'user/a
user=> (a)
But unfortunately, here is the error message I get:
CompilerException java.lang.RuntimeException: Can't take value of a macro: #'user/a, compiling:(/tmp/form-init5239882861501900074.clj:1:1)
... even though:
user=> (defmacro a[](print &form))
#'user/a
user=> (a)
(a)nil
NB. The Common Lisp equivalent is:
(defmacro w(&whole w)w) ;; 23 bytes
Existing names
Note also that once you define a, your cannot change the definition as follows:
(defmacro a[](a 0 1))
... because it complains that a accepts zero arguments. If, however, you define another macro with a name which has not yet been defined, it works:
user=> (defmacro b[](b 0 1))
#'user/b
user=> (b)
StackOverflowError user/b (form-init5239882861501900074.clj:1)
For more description of the problem see the ticket at http://dev.clojure.org/jira/browse/CLJ-1279.
Here's snippet:
(defmacro produce-constantly-fn []
(constantly :value))
(defmacro produce-fn []
(fn [& args] :value))
(defn test-fn []
((produce-fn)))
;; during evaluation of form below it throws:
;; java.lang.IllegalArgumentException
;; No matching ctor found for class clojure.core$constantly$fn__4614
(defn test-constantly-fn []
((produce-constantly-fn)))
Why last function cannot be compiled? The snippet can be considered as some sort macros abuse, but anyway...
I assume you defined your macro body without quoting and you are curious why it results in such a weird error message. If you really meant to define a macro for calling (constantly :value) then you should use quoting and it will work:
(defmacro produce-constantly-fn []
`(constantly :value))
(defn test-constantly-fn []
((produce-constantly-fn)))
=> #'user/test-constantly-fn
(test-constantly-fn)
=> :value
Now going back to your case without quoting. It looks really interesting and mysterious so I did some digging. These are my findings:
When you define a macro:
(defmacro produce-constantly-fn []
(constantly :value))
it will just create a function named produce-constantly-fn and mark it as a macro (it's still a Clojure function).
When you look into the implementation of constantly you will find (docs and meta omitted):
(defn constantly [x]
(fn [& args] x))
Under the hood it will compile to a closure object which will implement IFn and will have a single constructor parameter to close over x parameter. Something like this in Java code:
public class clojure.core$constantly$fn__4614 {
private final Object x;
public clojure.core$constantly$fn__4614(Object x) {
this.x = x;
}
public Object invoke(...) {
return x;
}
// other invoke arities
}
Now when you have following sexp:
(defn test-constantly-fn []
((produce-constantly-fn)))
I noticed that Clojure reader evals (produce-constantly-fn) which should return just a function object (produced by calling (constantly :value)) but I found in debugger that it produces clojure.core$constantly$fn__4614. symbol instead (notice the . at the end of the symbol - it is a Java interop form for calling a constructor). It looks like a function object/value gets somehow converted to a symbol representing its constructor call. I could find that the function value gets converted into Compiler$InvokeExpr object containing references to the compiled class name which is probably somehow converted into the symbol.
The reader tries to resolve clojure.core$constantly$fn__4614. further. It gets transformed by the reader into a call to clojure.core$constantly$fn__4614clojure.core$constantly$fn__4614 class constructor with no parameter.
As you have seen above the constructor of that class requires exactly one constructor thus the compilation fails (in clojure.lang.Compiler.NewExpr constructor body) with:
java.lang.IllegalArgumentException: No matching ctor found for class clojure.core$constantly$fn__4614
I am not sure why the Clojure reader transforms the function value to a symbol with constructor call interop form and causing such behaviour thus I presented only the direct cause of the error message and not the root cause why your code doesn't work. I guess it might be a bug or it was a conscious design decision. From the macro authors it would be better to fail fast and learn that the return value of a macro is not a valid code data but on the other hand it might be very difficult or impossible to determine if the returned data is a valid code or not. It's worth checking on Clojure mailing list.
I think what is going on is: macros resolve before functions.
So if you call macroexpand-1 on the function you get:
(def test-constantly-fn (clojure.core/fn ([] ((produce-constantly-fn)))))
Thus ((produce-constantly-fn)) is called before function and gives listed error.
According to spec, def should intern the var in the current ns (i.e. *ns*). However, the following code does not look anything like it:
(ns namespace-b)
(defn def_something []
(ns namespace-a)
(println *ns*) ;prints namespace-a as it should
(def something 1)
)
(def_something)
(println namespace-b/something) ; prints 1
(println namespace-a/something) ; throws
What am I missing?
Notes:
defn is used just for clarity. Defining and running anonymous function works just as well.
I know that using def inside function is probably not very idiomatic. However, this is just extracted essence of a bigger problem I ran into.
The parser already interns the var to the current namespace at compile time, although it won't be bound immediately:
(defn dd [] (def x 0))
x ;; => #<Unbound Unbound: #'user/x>
The relevant piece of code can be found here, with the second parameter to lookupVar triggering the aforementioned interning for non-existing vars here.
The parses then generates an expression that references the previously created var, so the expression logic never leaves the current namespace.
TL;DR: def is something that the compiler handles in a special kind of way.
The key thing to understand about def is that it is a macro. This means that it does not resolve the namespace or create the binding at runtime, but beforehand, while the code is being compiled.
If you call a function that calls def, that call to def was already resolved to use the namespace in which the function was defined. Similarly, if you call functions inside a function body, the functions to call are resolved at compile time within the namespace where that function was defined.
If you want to generally bind values to namespaces at runtime, you should use the function intern, which lets you explicitly set the namespace to mutate.
All this said, namespace mutation is just that, it's procedural and is not thread safe and does not have nice declarative semantics like other options Clojure makes available. I would strongly suggest finding a way to express your solution that does not involve unsafe runtime mutation.
Is there a way to use the reader with function values, e.g:
(read-string (pr-str +))
RuntimeException Unreadable form clojure.lang.Util.runtimeException
(Util.java:219)
?
As you might already know the output for (pr-str +) is not valid Clojure code that the reader can parse: "#<core$_PLUS_ clojure.core$_PLUS_#ff4805>". The output for function values when using the functions pr, prn, println and such, is intentionally wrapped around the #< reader macro that dispatches to the UnreadableReader, which throws the exception you are seeing.
For the example you provided you can use the print-dup function that works for basic serialization:
(defn string-fn [f]
(let [w (java.io.StringWriter.)]
(print-dup f w)
(str w)))
(let [plus (read-string (string-fn +))]
(plus 1 2))
The serialization done for the + function is actually generating the call to the class' constructor:
#=(clojure.core$_PLUS_. )
This only works of course if the class is already compiled in the Clojure environment where you are reading the string. If you serialized an anonymous function, saving it to a file and then reading it back in, when running a new REPL session, it will most likely not work since the class name for each anonymous function is different and depends on Clojure internals.
For arbitrary functions things get a lot more complicated. Sharing the source code might not even be enough, the function could rely on the usage of any number of other functions or vars that only exist in the source environment. If this is what you are thinking of doing, maybe considering other approaches to the problem you are trying to solve, will eliminate the need to serialize the value of arbitrary functions.
Hope it helps,
If you only need the name, can you just send the symbol with (name '+).
But generally speaking, it is a bad idea to use clojure read, if you want to read it back, as clojure's reader might execute some code in the process. Maybe have a look at the edn reader : clojure.edn/read-string
But maybe you just need to convert the string back to a symbol, in which case the (symbol name) function would be enough.
I was looking through the src of string? fn and had the a few questions. Below is the source of the string? fn -
(def
^{:arglists '([x])
:doc "Return true if x is a String"
:added "1.0"
:static true}
string? (fn ^:static string? [x] (instance? String x)))
What does giving a fn static metadata do?
Why is the static metadata given two times, shouldnt it be enough to specify it either for the fn or for the var?
Why does the anonymous fn have a name ?
For answers to 1. and 2., see d.j.sheldrick's comment on the question. Answer to 3.:
Firstly, although this is not relevant here, named functions can refer to themselves by their name. This allows them to return themselves as values or to call themselves through the usual call mechanism rather than recur to the top. Importantly, this is the correct self-call strategy for functions generating lazy seqs; I've gone into the reasons why in an earlier SO answer (see the part after the "How come you can wrap recursive calls in a lazy sequence..." block quote).
Secondly, functions are compiled to JVM classes. The classes are named by the Clojure compiler based on the namespace in which the function is defined if the function is unnamed; otherwise the name is used to generate a more meaningful name for the class. This is useful for debugging, since it makes stack traces more intelligible.