I have a question regarding how to define functions/macros which call other macros or special forms but where one of the symbols passed in needs to be dynamic.
The simplest version of this question is described below:
We can define variables using def
(def x 0)
But what if we wanted the name x to be determined programmatically so that we could do the equivalent of?
(let [a 'b]
(our-def a 3)) => user/b
We could try to define a function
(defn defn2 [sym val]
(def sym val))
However it does not do what we want
(def2 'y 1) => #'user/sym
At first it seems like a macro works (even though it seems like it would be unnecessary)
(defmacro def3 [sym val]
`(def ~sym ~val))
(def3 z 2) => user/z
but it is just superficial, because we're really right back where we started with regular def.
(let [a 'b]
(def3 a 3)) => user/a
I can do it if I use eval, but it doesn't seem like eval should be necessary
(defn def4 [sym val]
(eval `(def ~sym ~val)))
(let [a 'b]
(def4 a 4)) => user/b
If there are other built-in commands that could achieve this particular example, they are not really what I am looking for since def is just to show a particular example. There are macros more complicated than def that I might want to call and not have to worry about how they were internally implemented.
First: The right way to do this is to use macro that starts with def... since this is the way people have been doing defs and is of little surprise to the user.
To answer you question: Use intern:
(def foo 'bar)
(intern *ns* foo :hi)
(pr bar) ;; => :hi
(intern *ns* foo :hi2)
(pr bar) ;; => :hi2
If you want to use macros do this:
(def z 'aa)
(defmacro def3 [sym val]
`(def ~(eval sym) ~val))
(def3 z 2)
(pr aa) ;; => 2
Related
First, I have no experience with CS and Clojure is my first language, so pardon if the following problem has a solution, that is immediately apparent for a programmer.
The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. My approach revolves around a) storing temporarily the names of the atoms as strings in an atom itself; b) changing those strings to symbols with a function; c) using a function to add and create new atoms. The problem pertains to step "c": calling the function does not create new atoms, but using its body does create them.
All steps taken in the REPL are below (comments follow code blocks):
user=> (def atom-pool
#_=> (atom ["a1" "a2"]))
#'user/atom-pool
'atom-pool is the atom that stores intermediate to-be atoms as strings.
user=> (defn atom-symbols []
#_=> (mapv symbol (deref atom-pool)))
#'user/atom-symbols
user=> (defmacro populate-atoms []
#_=> (let [qs (vec (remove #(resolve %) (atom-symbols)))]
#_=> `(do ~#(for [s qs]
#_=> `(def ~s (atom #{}))))))
#'user/populate-atoms
'populate-atoms is the macro, that defines those atoms. Note, the purpose of (remove #(resolve %) (atom-symbols)) is to create only yet non-existing atoms. 'atom-symbols reads 'atom-pool and turns its content to symbols.
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(nil nil nil)
Here it is confirmed that there are no 'a1', 'a2', 'a-new' atoms as of yet.
user=> (defn new-atom [a]
#_=> (do
#_=> (swap! atom-pool conj a)
#_=> (populate-atoms)))
#'user/new-atom
'new-atom is the function, that first adds new to-be atom as string to `atom-pool. Then 'populate-atoms creates all the atoms from 'atom-symbols function.
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 nil)
Here we see that 'a1 'a2 were created as clojure.lang.Var$Unbound just by defining a function, why?
user=> (new-atom "a-new")
#'user/a2
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 nil)
Calling (new-atom "a-new") did not create the 'a-new atom!
user=> (do
#_=> (swap! atom-pool conj "a-new")
#_=> (populate-atoms))
#'user/a-new
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 #'user/a-new)
user=>
Here we see that resorting explicitly to 'new-atom's body did create the 'a-new atom. 'a-new is a type of clojure.lang.Atom, but 'a1 and 'a2 were skipped due to already being present in the namespace as clojure.lang.Var$Unbound.
Appreciate any help how to make it work!
EDIT: Note, this is an example. In my project the 'atom-pool is actually a collection of maps (atom with maps). Those maps have keys {:name val}. If a new map is added, then I create a corresponding atom for this map by parsing its :name key.
"The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. "
This sounds like a solution looking for a problem. I would generally suggest you try another way of achieving whatever the actual functionality is without generating vars at runtime, but if you must, you should use intern and leave out the macro stuff.
You cannot solve this with macros since macros are expanded at compile time, meaning that in
(defn new-atom [a]
(do
(swap! atom-pool conj a)
(populate-atoms)))
populate-atoms is expanded only once; when the (defn new-atom ...) form is compiled, but you're attempting to change its expansion when new-atom is called (which necessarily happens later).
#JoostDiepenmaat is right about why populate-atoms is not behaving as expected. You simply cannot do this using macros, and it is generally best to avoid generating vars at runtime. A better solution would be to define your atom-pool as a map of keywords to atoms:
(def atom-pool
(atom {:a1 (atom #{}) :a2 (atom #{})}))
Then you don't need atom-symbols or populate-atoms because you're not dealing with vars at compile-time, but typical data structures at run-time. Your new-atom function could look like this:
(defn new-atom [kw]
(swap! atom-pool assoc kw (atom #{})))
EDIT: If you don't want your new-atom function to override existing atoms which might contain actual data instead of just #{}, you can check first to see if the atom exists in the atom-pool:
(defn new-atom [kw]
(when-not (kw #atom-pool)
(swap! atom-pool assoc kw (atom #{}))))
I've already submitted one answer to this question, and I think that that answer is better, but here is a radically different approach based on eval:
(def atom-pool (atom ["a1" "a2"]))
(defn new-atom! [name]
(load-string (format "(def %s (atom #{}))" name)))
(defn populate-atoms! []
(doseq [x atom-pool]
(new-atom x)))
format builds up a string where %s is substituted with the name you're passing in. load-string reads the resulting string (def "name" (atom #{})) in as a data structure and evals it (this is equivalent to (eval (read-string "(def ...)
Of course, then we're stuck with the problem of only defining atoms that don't already exist. We could change the our new-atom! function to make it so that we only create an atom if it doesn't already exist:
(defn new-atom! [name]
(when-not (resolve (symbol name))
(load-string (format "(def %s (atom #{}))" name name))))
The Clojure community seems to be against using eval in most cases, as it is usually not needed (macros or functions will do what you want in 99% of cases*), and eval can be potentially unsafe, especially if user input is involved -- see Brian Carper's answer to this question.
*After attempting to solve this particular problem using macros, I came to the conclusion that it either cannot be done without relying on eval, or my macro-writing skills just aren't good enough to get the job done with a macro!
At any rate, I still think my other answer is a better solution here -- generally when you're getting way down into the nuts & bolts of writing macros or using eval, there is probably a simpler approach that doesn't involve metaprogramming.
Leonardo Borges has put together a fantastic presentation on Monads in Clojure. In it he describes the reader monad in Clojure using the following code:
;; Reader Monad
(def reader-m
{:return (fn [a]
(fn [_] a))
:bind (fn [m k]
(fn [r]
((k (m r)) r)))})
(defn ask [] identity)
(defn asks [f]
(fn [env]
(f env)))
(defn connect-to-db []
(do-m reader-m
[db-uri (asks :db-uri)]
(prn (format "Connected to db at %s" db-uri))))
(defn connect-to-api []
(do-m reader-m
[api-key (asks :api-key)
env (ask)]
(prn (format "Connected to api with key %s" api-key))))
(defn run-app []
(do-m reader-m
[_ (connect-to-db)
_ (connect-to-api)]
(prn "Done.")))
((run-app) {:db-uri "user:passwd#host/dbname" :api-key "AF167"})
;; "Connected to db at user:passwd#host/dbname"
;; "Connected to api with key AF167"
;; "Done."
The benefit of this is that you're reading values from the environment in a purely functional way.
But this approach looks very similar to the partial function in Clojure. Consider the following code:
user=> (def hundred-times (partial * 100))
#'user/hundred-times
user=> (hundred-times 5)
500
user=> (hundred-times 4 5 6)
12000
My question is: What is the difference between the reader monad and a partial function in Clojure?
The reader monad is a set of rules we can apply to cleanly compose readers. You could use partial to make a reader, but it doesn't really give us a way to put them together.
For example, say you wanted a reader that doubled the value it read. You might use partial to define it:
(def doubler
(partial * 2))
You might also want a reader that added one to whatever value it read:
(def plus-oner
(partial + 1))
Now, suppose you wanted to combine these guys in a single reader that adds their results. You'll probably end up with something like this:
(defn super-reader
[env]
(let [x (doubler env)
y (plus-oner env)]
(+ x y)))
Notice that you have to explicitly forward the environment to those readers. Total bummer, right? Using the rules provided by the reader monad, we can get much cleaner composition:
(def super-reader
(do-m reader-m
[x doubler
y plus-oner]
(+ x y)))
You can use partial to "do" the reader monad. Turn let into a do-reader by doing syntactic transformation on let with partial application of the environment on the right-hand side.
(defmacro do-reader
[bindings & body]
(let [env (gensym 'env_)
partial-env (fn [f] (list `(partial ~f ~env)))
bindings* (mapv #(%1 %2) (cycle [identity partial-env]) bindings)]
`(fn [~env] (let ~bindings* ~#body))))
Then do-reader is to the reader monad as let is to the identity monad (relationship discussed here).
Indeed, since only the "do notation" application of the reader monad was used in Beyamor's answer to your reader monad in Clojure question, the same examples will work as is with m/domonad Reader replaced with do-reader as above.
But, for the sake of variety I'll modify the first example to be just a bit more Clojurish with the environment map and take advantage of the fact that keywords can act as functions.
(def sample-bindings {:count 3, :one 1, :b 2})
(def ask identity)
(def calc-is-count-correct?
(do-reader [binding-count :count
bindings ask]
(= binding-count (count bindings))))
(calc-is-count-correct? sample-bindings)
;=> true
Second example
(defn local [modify reader] (comp reader modify))
(def calc-content-len
(do-reader [content ask]
(count content)))
(def calc-modified-content-len
(local #(str "Prefix " %) calc-content-len))
(calc-content-len "12345")
;=> 5
(calc-modified-content-len "12345")
;=> 12
Note since we built on let, we still have destructing at our disposal. Silly example:
(def example1
(do-reader [a :foo
b :bar]
(+ a b)))
(example1 {:foo 2 :bar 40 :baz 800})
;=> 42
(def example2
(do-reader [[a b] (juxt :foo :bar)]
(+ a b)))
(example2 {:foo 2 :bar 40 :baz 800})
;=> 42
So, in Clojure, you can indeed get the functionality of the do notation of reader monad without introducing monads proper. Analagous to doing a ReaderT transform on the identity monad, we can do a syntactic transformation on let. As you surmised, one way to do so is with partial application of the environment.
Perhaps more Clojurish would be to define a reader-> and reader->> to syntactically insert the environment as the second and last argument respectively. I'll leave those as an exercise for the reader for now.
One take-away from this is that while types and type-classes in Haskell have a lot of benefits and the monad structure is a useful idea, not having the constraints of the type system in Clojure allows us to treat data and programs in the same way and do arbitrary transformations to our programs to implement syntax and control as we see fit.
Why don't when-let and if-let support multiple bindings by default?
So:
(when-let [a ...
b ...]
(+ a b))
...instead of:
(when-let [a ...
(when-let [b ...
(+ a b)))
I am aware that I can write my own macro or use a monad (as described here: http://inclojurewetrust.blogspot.com/2010/12/when-let-maybe.html).
Because (for if-let, at least) it's not obvious what to do with the "else" cases.
At least, motivated by Better way to nest if-let in clojure I started to write a macro that did this. Given
(if-let* [a ...
b ...]
action
other)
it would generate
(if-let [a ...]
(if-let [b ...]
action
?))
and it wasn't clear to me how to continue (there are two places for "else").
You can say that there should be a single alternative for any failure, or none for when-let, but if any of the tests mutate state then things are still going to get messy.
In short, it's a little more complicated than I expected, and so I guess the current approach avoids having to make a call on what the solution should be.
Another way of saying the same thing: you're assuming if-let should nest like let. A better model might be cond, which isn't a "nested if" but more an "alternative if", and so doesn't fit well with scopes... or, yet another way of saying it: if doesn't handle this case any better.
Here is when-let*:
(defmacro when-let*
"Multiple binding version of when-let"
[bindings & body]
(if (seq bindings)
`(when-let [~(first bindings) ~(second bindings)]
(when-let* ~(vec (drop 2 bindings)) ~#body))
`(do ~#body)))
Usage:
user=> (when-let* [a 1 b 2 c 3]
(println "yeah!")
a)
;;=>yeah!
;;=>1
user=> (when-let* [a 1 b nil c 3]
(println "damn! b is nil")
a)
;;=>nil
Here is if-let*:
(defmacro if-let*
"Multiple binding version of if-let"
([bindings then]
`(if-let* ~bindings ~then nil))
([bindings then else]
(if (seq bindings)
`(if-let [~(first bindings) ~(second bindings)]
(if-let* ~(vec (drop 2 bindings)) ~then ~else)
~else)
then)))
Usage:
user=> (if-let* [a 1
b 2
c (+ a b)]
c
:some-val)
;;=> 3
user=> (if-let* [a 1 b "Damn!" c nil]
a
:some-val)
;;=> :some-val
EDIT: It turned out bindings should not be leaked in the else form.
If you use cats, then there is a mlet function that you might find useful :
(use 'cats.builtin)
(require '[cats.core :as m])
(require '[cats.monad.maybe :as maybe])
(m/mlet [x (maybe/just 42)
y nil]
(m/return (+ x y)))
;; => nil
As you can see, the mlet short-circuits when encountering a nil value.
(from section 6.5.1 nil)
Functions with closures seem to break when used with eval.
(eval {:fn (let [x "foo"] (fn [] "x"))})
;=> {:fn #<user$eval14716$fn__14717 user$eval14716$fn__14717#1ddd735>}
(eval {:fn (let [x "foo"] (fn [] x))})
;=> IllegalArgumentException No matching ctor found for class user$eval14740$fn__14741
; clojure.lang.Reflector.invokeConstructor (Reflector.java:166)
I don't really know enough about Clojure (or closure) to know if this is a bug or something which intentionally isn't allowed - can anyone shed some light on this?
Edit: Just to be clear, I'm talking specifically about the way eval handles function objects. AFAIK eval is actually designed to work with java objects, including functions; the example given on the clojure website - (eval (list + 1 2 3)) - passes a function object into eval.
Cloure's eval does not perfectly support function objects. It's not necessarily even closures that cause the problem.
For example, this did not work in Clojure 1.0.0:
(eval {:fn (fn [x] x)})
But this did:
(eval (fn [x] x))
The first example got fixed. The following also works:
(eval (let [x "foo"] (fn [] x)))
But the following still does not work:
(eval {:fn (let [x "foo"] (fn [] x))})
I can't pin it down to a single line in the compiler, but it's something about how literal objects (clojure.lang.Compiler$ObjExpr I think) get handled by eval in different contexts: e.g. at the "top" of an expression versus inside another data structure.
In general, I think, you cannot rely on being able to eval function objects in Clojure, regardless of whether or not they are closures. It happens to work for some simple examples, mostly to simplify the explanation of things like (eval (list + 1 2)). Macros should always return literal source code as data structures, not compiled functions.
Try quoting your argument to eval:
(eval '{:fn (let [x "foo"] (fn [] x))})
;=> {:fn #<user$eval345$fn__346 user$eval345$fn__346#17b6dd83>}
((:fn *1))
;=> "foo"
This is not a bug. The equivalent of (eval (list + 1 2 3)) with a "closure" is (eval (list fn [] "foo")), not (eval (fn [] "foo")).
And (eval (list fn [] "foo")) => Can't take value of a macro: #'clojure.core/fn, again indicating that you're not supposed to do things like that (and there's no need for it anyway).
Here is the sample code I want to get to work:
(letfn [(CONC [f] f)
(CONT [f] (str "\newline" f))]
((voodoo "CONC") "hamster"))
Is there some voodo that will make it call the CONC function with hamster as the parameter? That is, is there some way to convert the string "CONC" into a function that is not bound to a namespace but rather to a local binding?
EDIT:
To be clearer, the way this will be called is:
(map #((voodoo (:tag %)) (:value %))
[
{:tag "CONC" :value "hamster"}
{:tag "CONT" :value "gerbil"}
]
)
I'd probably solve this by creating a map of functions indexed by strings:
(def voodoo
{"CONC" (fn [f] f)
"CONT" (fn [f] (str "\newline" f))})
Then your desired code should work directly (exploiting the fact that a map is a function that looks up it's argument)
(map #((voodoo (:tag %)) (:value %))
[
{:tag "CONC" :value "hamster"}
{:tag "CONT" :value "gerbil"}
]
)
Note that the functions here are fully anonymous - you don't need them to be referenced anywhere in the namespace for this to work. In my view this is a good thing, because unless you also need the functions somewhere else then it's best to avoid polluting your top-level namespace too much.
No. Eval does not have access to the local/lexical environment, ever.
Edit: This is not a very good answer, and not really accurate either. You could write voodoo as a macro, and then it doesn't need runtime access to the lexical environment, just compile-time. However, this means it would only work if you know at compile time that the function you want to call is x, and so it wouldn't be very useful - why not just type x instead of (voodoo "x")?
(defmacro voodoo [fname]
(symbol fname))
(letfn [(x [y] (inc y))]
((voodoo "x") 2))
;; 3
(letfn [(x [y] (inc y))]
(let [f "x"]
((voodoo f) 2)))
;; error
Well, it's sort of possible:
(defmacro voodoo [s]
(let [env (zipmap (map (partial list 'quote) (keys &env))
(keys &env))]
`(if-let [v# (~env (symbol ~s))]
v#
(throw (RuntimeException. "no such local")))))
...and now we can do weird stuff like this:
user> (defn example [s]
(letfn [(foo [x] {:foo x})
(bar [x] {:bar x})]
((voodoo s) :quux)))
#'user/example
user> (example "foo")
{:foo :quux}
user> (example "bar")
{:bar :quux}
user> (example "quux")
; Evaluation aborted.
user> *e
#<RuntimeException java.lang.RuntimeException: no such local>
That "Evaluation aborted" means an exception was thrown.
You could also replace the throw branch of the if in voodoo with (resolve (symbol ~s)) to defer to the globals if no local is found:
(defmacro voodoo [s]
(let [env (zipmap (map (partial list 'quote) (keys &env))
(keys &env))]
`(if-let [v# (~env (symbol ~s))]
v#
(resolve (symbol ~s)))))
...and now this works with definition of example as above (though note that if you are experimenting at the REPL, you will need to recompile example after redefining voodoo):
user> (defn quux [x] {:quux x})
#'user/quux
user> (example "quux")
{:quux :quux}
Now, this is an abuse of Clojure's facilities which one would do well to try to do without. If one cannot, one should probably turn to evalive by Michael Fogus; it's a library which provides an "eval-with-locals" facility in the form of an evil function and a couple of utilities. The functionality seems to be well factored too, e.g. something like the ~(zipmap ...) thing above is encapsulated as a macro and evil there appears to be almost a drop-in replacement for eval (add the env parameter and you're good to go). I haven't read the source properly, but I probably will now, looks like fun. :-)
Im not really clear what you are asking for so i'll try a couple answers:
if you have a string that is the name of the function you wish to call:
(def name "+")
((find-var (symbol (str *ns* "/" name))) 1 2 3)
this would give voodoo a deffinition like this:
(defn voodoo [name args] (apply (find-var (symbol (str *ns* "/" name))) args))
#'clojure.core/voodoo
clojure.core=> (voodoo "+" [1 2 3])
6
clojure.core=>
this assumes your function is in the current namepace ns.
if you want to turn a string into a function you could use this pattern
(let [f (eval (read-string "(fn [] 4)"))] (f))