Suppose I've added specific metadata to my vars:
(defn ^:run-at-startup init []
(prn "Initializing...")
:done)
(meta (var init))
; {:arglists ([]), :ns #<Namespace user>, :name init, :end-column 34,
; :run-at-startup true, :column 1, :line 5, :file "NO_SOURCE_FILE", :end-line 5}
Then I would like to lookup all the vars (across different namespaces) that contains it. Is it possible?
Here is why. My app consists of several modules that must be initialized at startup. New modules could be added and existing removed (not at runtime, of course), and it's initializers must be called without knowing any specifics of the module. I think of adding metadata to initializers, then looking it all up and calling.
I would like to know if there are better ways.
So, if you require all the namespaces that contain your non-private initializers, all-ns is able to retrieve a list of those namespaces. If you do not know what namespaces exist, you can probably use e.g. tools.namespace to find out.
The following function finds all vars that contain a certain metadata key set to true, returning a seq of the vars' values.
(defn find-by-var-meta
[metadata-flag]
(->> (all-ns)
(mapcat ns-publics)
(keep
(fn [[_ v]]
(when (-> v meta metadata-flag)
(var-get v))))))
The resulting seq can then be traversed and everything that is a function can be called. So, in your case this should look like this:
(require '[my.namespace.initializers a b c])
(find-by-var-meta :run-at-startup) ;; => seq of initializers from the above ns.
And a quick check in the REPL:
(defn ^:run-at-startup add-one [x] (inc x)) ;; => #'user/add-one
((first (find-by-var-meta :run-at-startup)) 5) ;; => 6
(As seen here, you also don't need to specify a full map for metadata if you only want to set a key - or multiple ones - to true.)
Related
When I attach some metadata to a function and then call it I am not able to access those metadata within that function
(let [I (fn I [x] (println I) (println (meta I)))]
(let [f (with-meta I {:rr 5})]
(println I)
(println f)
(f I)))
I see that the self reference from within the function is not the function instance actually invoked and thus no metadata is available through that self reference. I need the self reference to give me the function instance actually invoked to access those metadata
I think that the problem is that your conflating the value of the function and the identity of the function together. It's a thing many other languages do so it's natural when you're learning Clojure. In your example, I has a reference to itself, and looks up the metadata from that reference, which returns nil. You then create f which is the same as I, but with some metadata. So when you run f it looks up the metadata on I and returns nil. Defining f doesn't change I at all, it just creates a new thing in terms of the old thing. If you want to change something you need to introduce a reference type that you can change. There are several of these, but usually to store functions you'd use a Var (see here for reference)
(defn i [] (meta i))
(i) ;;=> nil
(alter-var-root #'i with-meta {:rr 5})
(i) ;;=> {:rr 5}
Here we define a function in the current namespace called i which just returns it's own metadata. We call it to get nil. Then we alter the global reference with some new metadata, and call it again.
If you wanted a more lexically scoped example, you could use an atom as below:
(let [i (atom nil)
f (fn [] (meta #i))]
(reset! i f)
(prn 'before '>> (#i))
(swap! i with-meta {:rr 5})
(prn 'after '>> (#i)))
However, other than learning how these things fit together, I'm not sure what the goal is. It's probably a bad idea to try and use these structures in a real program that you plan on maintaining.
Rather accidentally, I found a trick that enables functions to read it own metadata. It appears, the Clojure compiler generates metadata support code differently when the original function definition has custom metadata. If it is present, (meta fn-name) works inside the body of the function, otherwise it does not. For example, the following produces the result desired by the OP:
*clojure-version*
;;=> {:major 1, :minor 10, :incremental 0, :qualifier nil}
(let [f1 ^{:foo true} (fn f [] (meta f))
f2 (with-meta f1 {:bar true})]
(prn (f1))
(prn (f2)))
;;=> {:foo true}
;;=> {:bar true}
;;=> nil
We can examine the code generated for a function without the metadata in the original definition - there is just the invoke method
(require '[clojure.pprint :as p])
(let [ff (fn f [] (meta f))]
(p/pprint (seq (.getDeclaredMethods (class ff)))))
;;=> (#object[java.lang.reflect.Method 0x2b56b137 "public java.lang.Object user$eval2171$f__2172.invoke()"])
;;=> nil
And when the metadata is present, additional methods (meta and withMeta) are generated to deal with the metadata.
(let [ff ^:foo (fn f [] (meta f))]
(p/pprint (seq (.getDeclaredMethods (class ff)))))
;;=> (#object[java.lang.reflect.Method 0x3983bd83 "public clojure.lang.IObj user$eval2175$f__2176.withMeta(clojure.lang.IPersistentMap)"]
;;=> #object[java.lang.reflect.Method 0x547d182d "public clojure.lang.IPersistentMap user$eval2175$f__2176.meta()"]
;;=> #object[java.lang.reflect.Method 0x62c3d0fe "public java.lang.Object user$eval2175$f__2176.invoke()"])
;;=> nil
Welcome to Clojure, #xstreamer!
I'm going to suggest something different from what (precisely) you're asking for. I don't know how querying the function's metadata from within the function should work, really. So I'm going to suggest defining the function first, and redefining the function metadata afterwards. This is fairly simple in Clojure.
(defn f
"Boring doc"
[])
(meta #'f)
;; => {:arglists ([]),
;; :doc "Boring doc",
;; :line 32,
;; :column 1,
;; :file "C:/Users/teodorlu/IdeaProjects/th-scratch/src/th/play/core.clj",
;; :name f,
;; :ns #object[clojure.lang.Namespace 0x3b402f0c "th.play.core"]}
Now, redefine it!
(alter-meta! #'f assoc :rr 5)
(meta #'f)
;; => {:arglists ([]),
;; :doc "Boring doc",
;; :line 32,
;; :column 1,
;; :file "C:/Users/teodorlu/IdeaProjects/th-scratch/src/th/play/core.clj",
;; :name f,
;; :ns #object[clojure.lang.Namespace 0x3b402f0c "th.play.core"],
;; :rr 5}
Where assoc sets a value in a map.
(assoc {} :rr 5)
;; {:rr 5}
(assoc {:some :stuff} :more :stuff)
;; {:some :stuff, :more :stuff}
References
If you're confused by the #'f, this is how you get the var representing the binding of f, instead of just the value it refers to. For more information about vars and how to use them, refer to the official reference on vars and the less terse guide from 8th light.
I am trying to print the documentation for all functions in a given namespace by invoking the following expression in a REPL:
(doseq
[f (dir-fn 'clojure.repl)]
(doc f))
However the invocation of this expression returns nil without printing the documentation to the REPL. I know this might have to do with doc being a macro, but I'm a Clojure novice and am not entirely sure how to understand the problem.
Why does this expression return nil without printing the documentation?
How can this expression be modified so that it prints the documentation for each function in a given namespace?
Thanks!
Update: Combined both provided answers:
(defn ns-docs [ns']
(doseq [[symbol var] (ns-interns ns')]
(newline)
(println symbol)
(print " ")
(println (:doc (meta var)))))
(ns-docs 'clojure.repl)
I would, instead, start here:
The Clojure CheatSheet
ClojureDocs.org
Clojure-Doc.org (similar name, but different)
The API & Reference sections at Clojure.org
Note that doc is in the namespace clojure.repl, which reflects its intended usage (by a human in a repl). Here is some code that will also iterate on a namespace & print doc strings (using a different technique):
(doseq [[fn-symbol fn-var] (ns-interns 'demo.core)]
(newline)
(println fn-symbol)
(println (:doc (meta fn-var))))
where demo.core is the namespace of interest.
Note that ns-interns gives you both a symbol and var like:
fn-symbol => <#clojure.lang.Symbol -main>
fn-var => <#clojure.lang.Var #'demo.core/-main>
The meta function has lots of other info you may want to use someday:
(meta fn-var) =>
<#clojure.lang.PersistentArrayMap
{ :arglists ([& args]),
:doc "The Main Man!",
:line 9, :column 1,
:file "demo/core.clj",
:name -main,
:ns #object[clojure.lang.Namespace 0x14c35a06 "demo.core"]}>
While this probably won't help you with answering your question, the problem of evaluating macro's comes up a lot when you are learning Clojure.
Macros are responsible for the evaluation of their arguments. In this case clojure.repl/doc will ignore the current lexical context and assume that the symbol f that you're giving it is the name of a function you want to see the documentation for. It does this because it's intended to be used at the REPL, and is assuming you wouldn't want to type quotes all the time.
As f doesn't exist, it prints nothing. Then doseq returns nil, since it exists to do something for side effects only - hence starting in do. In order to pass an argument to a macro that refuses to respect the lexical context like this, you need to write the code for each element in the list.
You can do this by hand, or by constructing the code as data, and passing it to eval to execute. You can do this in an imperative style, using doseq:
(doseq [f (ns-interns 'clojure.repl)]
(eval `(doc ~(symbol "clojure.repl" (str (first f))))))
or in a slightly more Clojurey way (which will allow you to see the code that it would execute by removing eval from the end and running it at the REPL):
(->> (ns-interns 'clojure.repl)
(map #(list 'clojure.repl/doc (symbol "clojure.repl" (str (first %)))))
(cons `do)
eval)
In both of these we use quote and syntax-quote to construct some code from the list of symbols reflected from the namespace, and pass it to eval to actually execute it. This page on Clojure's weird characters should point you in the right direction for understanding what's going on here.
This an example of why you shouldn't write macro's, unless you've got no other options. Macro's do not compose, and are often difficult to work with. For a more in depth discussion, Fogus's talk and Christophe Grand's talk are both good talks.
Why does this expression return nil without printing the documentation?
Because the doc macro is receiving the symbol f from your loop, instead of a function symbol directly.
How can this expression be modified so that it prints the documentation for each function in a given namespace?
(defn ns-docs [ns']
(let [metas (->> (ns-interns ns') (vals) (map meta) (sort-by :name))]
(for [m metas :when (:doc m)] ;; you could filter here if you want fns only
(select-keys m [:name :doc]))))
(ns-docs 'clojure.repl)
=>
({:name apropos,
:doc "Given a regular expression or stringable thing, return a seq of all
public definitions in all currently-loaded namespaces that match the
str-or-pattern."}
...
)
Then you can print those maps/strings if you want.
Let's say I've defined a record like this:
(defrecord MyRecord [x y z])
And I construct it like this:
(def test (map->MyRecord {:x "1" :y "2" :z "3" :w "ikk"}))
I can do like this:
(:w test) ; Returns "ikk"
Why is :w retained? I'm thinking that since I've created a record that takes x, y and z these are the only "keys" that should be present.
Is there a good way to exclude keys that are not present as arguments in the record declaration without using select-keys?
For example:
(defrecord MyRecord1 [x y z])
(defrecord MyRecord2 [x y w])
(defprotocol MyProtocol
(do-stuff [data]))
(extend-protocol MyProtocol
MyRecord1
(do-stuff [data]
(let [data (select-keys data [:x :y :z])] ; (S1)
...))
MyRecord2
(do-stuff [data]
(let [data (select-keys data [:x :y :w])] ; (S2)
...)))
I want to avoid doing select-keys (S1, S2) manually for each record when I use MyProtocol when records are constructed using map-> with additional data (that I don't care about).
Clojure records implement IPersistentMap (as well as java.util.Map for Java interop), and behaves like a normal map - this means that you can use them wherever and in the same way you would use maps. You can look at a record as a typed map - it's a map, but you can easily do dispatch using multimethods and protocols.
This makes it very easy to start representing your data using plain maps, and then advance to a record when you need the type.
As maps, records support additional keys, but they are treated differently. With your example, (.x test) works, but (.w test) does not, since only the predefined keys become fields in the implementing Java class.
To avoid extra keys, just make your own constructor:
(defn limiting-map->MyRecord
[m]
(map->MyRecord
(select-keys m [:x :y :z])))
It is a feature of defrecord. See: http://clojure.org/datatypes Section deftype and defrecord:
defrecord provides a complete implementation of a persistent map, including:
...
extensible fields (you can assoc keys not supplied with the defrecord definition)
Via Reflection you can see, that your x,y,z params are regular attributes of the object:
user=> (>pprint (.? (map->MyRecord {:w 4})))
(#[__extmap :: (user.MyRecord) | java.lang.Object]
;...
#[x :: (user.MyRecord) | java.lang.Object]
#[y :: (user.MyRecord) | java.lang.Object]
#[z :: (user.MyRecord) | java.lang.Object])
And the additional values are stored in that __extmap:
user=> (.-__extmap (map->MyRecord {:w 4}))
{:w 4}
This means, that there is nothing left for you other than watch out on the places you want to deal with your record as a Map, since new keys can be added at any time:
user=> (let [r (->MyRecord 1 2 3) r (assoc r :w 4)] (keys r))
(:x :y :z :w)
So if you find yourself repeating code like (select-keys myr [:x :y :z]) then extract that as a helper fn.
Adding things like your own c'tors is always a good idea (e.g. if you want to have 0 for missing keys instead of nil e.g.), but this only protects you from yourself and the users following your example.
I'm trying to read metadata for a collection of functions in Clojure, but the var or reader special forms do not work unless they are directly dealing with the symbol.
; this works
(var my-fn)
; this doesn't
(defn val-it [x] (var x))
(val-it my-fn)
Is there any way to get this to work within the scope of another function?
resolve returns the Var or class object corresponding the given symbol in the context of the current namespace. ns-resolve allows you to specify which namespace to resolve the symbol in.
(resolve 'my-fn)
;= #'some.ns/my-fn
If the symbol cannot be resolved to a Var, nil is returned.
(var my-fn) does deal directly with the symbol because it is a special form (the reader receives the form unevaluated).
The metadata you want to read is stored in the var object, not in the function object.
Thus your goal, to read metadata from a list of function objects, is only achievable by traversing all existing vars and comparing their value by equality. I'd only recommend it if the function objects are the only way to start.
(defn meta-of
"Returns a hashmap mapping the function objects in fn-objs to
a set of metadata of vars containing it."
[fn-objs]
(let [fn-objs (set fn-objs)]
(reduce (fn [acc ns]
(reduce (fn [acc var]
(let [val (var-get var)]
(cond-> acc
(contains? fn-objs val)
(update-in [val] (fnil conj #{}) (meta var)))))
acc
(vals (ns-interns ns)))) {} (all-ns))))
(def foo inc) ;; Now there are two vars that have the inc function as their value
(meta-of [inc])
{#<core$inc clojure.core$inc#66d7e31d> ;; <- function object
#{{:ns #<Namespace clojure.core>, ;; <- metadata in clojure.core namespace
:name inc,
:file "clojure/core.clj",
:column 1,
:line 881,
:arglists ([x]),
:added "1.2",
:inline #<core$inc__inliner clojure.core$inc__inliner#24f87069>,
:doc
"Returns a number one greater than num. Does not auto-promote\n longs, will throw on overflow. See also: inc'"}
{:ns #<Namespace user>, ;; <- metadata in user namespace
:name foo,
:file "/tmp/form-init1078564431656334911.clj",
:column 1,
:line 1}}}
Given a list of names for variables, I want to set those variables to an expression.
I tried this:
(doall (for [x ["a" "b" "c"]] (def (symbol x) 666)))
...but this yields the error
java.lang.Exception: First argument to def must be a Symbol
Can anyone show me the right way to accomplish this, please?
Clojure's "intern" function is for this purpose:
(doseq [x ["a" "b" "c"]]
(intern *ns* (symbol x) 666))
(doall (for [x ["a" "b" "c"]] (eval `(def ~(symbol x) 666))))
In response to your comment:
There are no macros involved here. eval is a function that takes a list and returns the result of executing that list as code. ` and ~ are shortcuts to create a partially-quoted list.
` means the contents of the following lists shall be quoted unless preceded by a ~
~ the following list is a function call that shall be executed, not quoted.
So ``(def ~(symbol x) 666)is the list containing the symboldef, followed by the result of executingsymbol xfollowed by the number of the beast. I could as well have written(eval (list 'def (symbol x) 666))` to achieve the same effect.
Updated to take Stuart Sierra's comment (mentioning clojure.core/intern) into account.
Using eval here is fine, but it may be interesting to know that it is not necessary, regardless of whether the Vars are known to exist already. In fact, if they are known to exist, then I think the alter-var-root solution below is cleaner; if they might not exist, then I wouldn't insist on my alternative proposition being much cleaner, but it seems to make for the shortest code (if we disregard the overhead of three lines for a function definition), so I'll just post it for your consideration.
If the Var is known to exist:
(alter-var-root (resolve (symbol "foo")) (constantly new-value))
So you could do
(dorun
(map #(-> %1 symbol resolve (alter-var-root %2))
["x" "y" "z"]
[value-for-x value-for-y value-for z]))
(If the same value was to be used for all Vars, you could use (repeat value) for the final argument to map or just put it in the anonymous function.)
If the Vars might need to be created, then you can actually write a function to do this (once again, I wouldn't necessarily claim this to be cleaner than eval, but anyway -- just for the interest of it):
(defn create-var
;; I used clojure.lang.Var/intern in the original answer,
;; but as Stuart Sierra has pointed out in a comment,
;; a Clojure built-in is available to accomplish the same
;; thing
([sym] (intern *ns* sym))
([sym val] (intern *ns* sym val)))
Note that if a Var turns out to have already been interned with the given name in the given namespace, then this changes nothing in the single argument case or just resets the Var to the given new value in the two argument case. With this, you can solve the original problem like so:
(dorun (map #(create-var (symbol %) 666) ["x" "y" "z"]))
Some additional examples:
user> (create-var 'bar (fn [_] :bar))
#'user/bar
user> (bar :foo)
:bar
user> (create-var 'baz)
#'user/baz
user> baz
; Evaluation aborted. ; java.lang.IllegalStateException:
; Var user/baz is unbound.
; It does exist, though!
;; if you really wanted to do things like this, you'd
;; actually use the clojure.contrib.with-ns/with-ns macro
user> (binding [*ns* (the-ns 'quux)]
(create-var 'foobar 5))
#'quux/foobar
user> quux/foobar
5
Evaluation rules for normal function calls are to evaluate all the items of the list, and call the first item in the list as a function with the rest of the items in the list as parameters.
But you can't make any assumptions about the evaluation rules for special forms or macros. A special form or the code produced by a macro call could evaluate all the arguments, or never evaluate them, or evaluate them multiple times, or evaluate some arguments and not others. def is a special form, and it doesn't evaluate its first argument. If it did, it couldn't work. Evaluating the foo in (def foo 123) would result in a "no such var 'foo'" error most of the time (if foo was already defined, you probably wouldn't be defining it yourself).
I'm not sure what you're using this for, but it doesn't seem very idiomatic. Using def anywhere but at the toplevel of your program usually means you're doing something wrong.
(Note: doall + for = doseq.)