As part of improving Cider's debugger, I need to implement special handling for all possible special-forms. In order words, I need to know all symbols which satisfy special-symbol?.
The doc page on Special Forms, while helpful, doesn't offer all of them.
For instance, after some experimentation, I've learned that
Most of the forms listed there have a * counterpart (let* and loop*, for instance).
There is a clojure.core/import* special-symbol (which I wouldn't have found if not for sheer luck).
Is there a complete list of all special symbols?
Alternatively, is there a way to list all interned symbols? If so, then I could filter over special-symbol?.
Looking at the definition of special-symbol? provides a big clue:
(defn special-symbol?
"Returns true if s names a special form"
{:added "1.0"
:static true}
[s]
(contains? (. clojure.lang.Compiler specials) s))
Thus:
user=> (pprint (keys (. clojure.lang.Compiler specials)))
(&
monitor-exit
case*
try
reify*
finally
loop*
do
letfn*
if
clojure.core/import*
new
deftype*
let*
fn*
recur
set!
.
var
quote
catch
throw
monitor-enter
def)
Related
I am trying to print the documentation for all functions in a given namespace by invoking the following expression in a REPL:
(doseq
[f (dir-fn 'clojure.repl)]
(doc f))
However the invocation of this expression returns nil without printing the documentation to the REPL. I know this might have to do with doc being a macro, but I'm a Clojure novice and am not entirely sure how to understand the problem.
Why does this expression return nil without printing the documentation?
How can this expression be modified so that it prints the documentation for each function in a given namespace?
Thanks!
Update: Combined both provided answers:
(defn ns-docs [ns']
(doseq [[symbol var] (ns-interns ns')]
(newline)
(println symbol)
(print " ")
(println (:doc (meta var)))))
(ns-docs 'clojure.repl)
I would, instead, start here:
The Clojure CheatSheet
ClojureDocs.org
Clojure-Doc.org (similar name, but different)
The API & Reference sections at Clojure.org
Note that doc is in the namespace clojure.repl, which reflects its intended usage (by a human in a repl). Here is some code that will also iterate on a namespace & print doc strings (using a different technique):
(doseq [[fn-symbol fn-var] (ns-interns 'demo.core)]
(newline)
(println fn-symbol)
(println (:doc (meta fn-var))))
where demo.core is the namespace of interest.
Note that ns-interns gives you both a symbol and var like:
fn-symbol => <#clojure.lang.Symbol -main>
fn-var => <#clojure.lang.Var #'demo.core/-main>
The meta function has lots of other info you may want to use someday:
(meta fn-var) =>
<#clojure.lang.PersistentArrayMap
{ :arglists ([& args]),
:doc "The Main Man!",
:line 9, :column 1,
:file "demo/core.clj",
:name -main,
:ns #object[clojure.lang.Namespace 0x14c35a06 "demo.core"]}>
While this probably won't help you with answering your question, the problem of evaluating macro's comes up a lot when you are learning Clojure.
Macros are responsible for the evaluation of their arguments. In this case clojure.repl/doc will ignore the current lexical context and assume that the symbol f that you're giving it is the name of a function you want to see the documentation for. It does this because it's intended to be used at the REPL, and is assuming you wouldn't want to type quotes all the time.
As f doesn't exist, it prints nothing. Then doseq returns nil, since it exists to do something for side effects only - hence starting in do. In order to pass an argument to a macro that refuses to respect the lexical context like this, you need to write the code for each element in the list.
You can do this by hand, or by constructing the code as data, and passing it to eval to execute. You can do this in an imperative style, using doseq:
(doseq [f (ns-interns 'clojure.repl)]
(eval `(doc ~(symbol "clojure.repl" (str (first f))))))
or in a slightly more Clojurey way (which will allow you to see the code that it would execute by removing eval from the end and running it at the REPL):
(->> (ns-interns 'clojure.repl)
(map #(list 'clojure.repl/doc (symbol "clojure.repl" (str (first %)))))
(cons `do)
eval)
In both of these we use quote and syntax-quote to construct some code from the list of symbols reflected from the namespace, and pass it to eval to actually execute it. This page on Clojure's weird characters should point you in the right direction for understanding what's going on here.
This an example of why you shouldn't write macro's, unless you've got no other options. Macro's do not compose, and are often difficult to work with. For a more in depth discussion, Fogus's talk and Christophe Grand's talk are both good talks.
Why does this expression return nil without printing the documentation?
Because the doc macro is receiving the symbol f from your loop, instead of a function symbol directly.
How can this expression be modified so that it prints the documentation for each function in a given namespace?
(defn ns-docs [ns']
(let [metas (->> (ns-interns ns') (vals) (map meta) (sort-by :name))]
(for [m metas :when (:doc m)] ;; you could filter here if you want fns only
(select-keys m [:name :doc]))))
(ns-docs 'clojure.repl)
=>
({:name apropos,
:doc "Given a regular expression or stringable thing, return a seq of all
public definitions in all currently-loaded namespaces that match the
str-or-pattern."}
...
)
Then you can print those maps/strings if you want.
After a while of working with Clojure, I have accumulated some knowledge on its laziness. I know whether a frequently-used API such as map is lazy. However, I still feel dubious when I start using an unfamiliar API such as with-open.
Is there any document that shows a complete list of lazy APIs of Clojure's core module?
You can find functions that return lazy sequences by opening up the Clojure code https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj
and searching for "Returns a lazy"
I am not aware of any curated lists of them.
The rule of thumb is: if it returns a sequence, it will be a lazy sequence, if it returns a value, it will force evaluation.
When using a new function, macro or special form, read the docstring. Most development environments have a key to show the docstring, or at least navigate to the source (where you can see the docstring), and there is always http://clojure.org/api/api.
In the case of with-open:
with-open
macro
Usage: (with-open bindings & body)
bindings => [name init ...]
Evaluates body in a try expression with names bound to the values
of the inits, and a finally clause that calls (.close name) on each
name in reverse order.
We can see that the result of calling with-open is evaluation of the expression with a final close. So we know that there is nothing lazy about it. However that doesn't mean you don't need to think about laziness inside with-open, quite the opposite!
(with-open [r (io/reader "myfile")]
(line-seq r))
This is a common trap. line-seq returns a lazy sequence! The problem here is that the lazy sequence will be realized after the file is closed, because the file is closed when exiting the scope of with-open. So you need to fully process the lazy sequence before exiting the with-open scope.
My advice is to avoid trying to think about your program as having 'lazy bits' and 'immediate bits', but instead just be mindful that when io or side-effects are involved you need to take care of when things happen as well as what should happen.
digging on a Timothy Pratley's proposal to search in doc:
let's make it fun!
your repl has everything that you need to find out a list of lazy functions.
first of all, there is a clojure.repl/doc macro, which prints documentation to out in repl
user> (doc +)
-------------------------
clojure.core/+
([] [x] [x y] [x y & more])
Returns the sum of nums. (+) returns 0. Does not auto-promote
longs, will throw on overflow. See also: +'
nil
unfortunately we can't get a string of it simply, but we can always rebind the *out* to be a StringWriter, and then get its string value.
so, whan we want to take all the symbols from clojure.core namespace, get their docs, write them all to string, and find every one that contains "returns a lazy". Here comes the help: clojure.core/ns-publics, returning a map of public names to their vars:
user> (take 10 (ns-publics 'clojure.core))
([primitives-classnames #'clojure.core/primitives-classnames]
[+' #'clojure.core/+']
[decimal? #'clojure.core/decimal?]
[restart-agent #'clojure.core/restart-agent]
[sort-by #'clojure.core/sort-by]
[macroexpand #'clojure.core/macroexpand]
[ensure #'clojure.core/ensure]
[chunk-first #'clojure.core/chunk-first]
[eduction #'clojure.core/eduction]
[tree-seq #'clojure.core/tree-seq])
so we just need to get all the keys from there and lookup for their docs.
Let's make a macro for that:
user> (defmacro all-docs []
(let [names (keys (ns-publics 'clojure.core))]
`(binding [*out* (java.io.StringWriter.)]
(do ~#(map #(list `doc %) names))
(str *out*))))
#'user/all-docs
it does just what i've said, gets all publics' docs to string.
now we simply process it:
user> (def all-doc-items (clojure.string/split
(all-docs)
#"-------------------------"))
#'user/all-doc-items
user> (nth all-doc-items 10)
"\nclojure.core/tree-seq\n([branch? children root])\n Returns a lazy sequence of the nodes in a tree, via a depth-first walk.\n branch? must be a fn of one arg that returns true if passed a node\n that can have children (but may not). children must be a fn of one\n arg that returns a sequence of the children. Will only be called on\n nodes for which branch? returns true. Root is the root node of the\n tree.\n"
and now just filter them:
user> (def all-lazy-fns (filter #(re-find #"(?i)returns a lazy" %) all-doc-items))
#'user/all-lazy-fns
user> (count all-lazy-fns)
30
user> (println (take 3 all-lazy-fns))
(
clojure.core/tree-seq
([branch? children root])
Returns a lazy sequence of the nodes in a tree, via a depth-first walk.
branch? must be a fn of one arg that returns true if passed a node
that can have children (but may not). children must be a fn of one
arg that returns a sequence of the children. Will only be called on
nodes for which branch? returns true. Root is the root node of the tree.
clojure.core/keep-indexed
([f] [f coll])
Returns a lazy sequence of the non-nil results of (f index item). Note,
this means false return values will be included. f must be free of
side-effects. Returns a stateful transducer when no collection is
provided.
clojure.core/take-nth
([n] [n coll])
Returns a lazy seq of every nth item in coll. Returns a stateful
transducer when no collection is provided.
)
nil
And now use these all-lazy-fns however you want.
In the Clojure Koans :fin is being used in atoms.clj. Here is one example:
(def atomic-clock (atom 0))
"Atomic atoms are atomic"
(= 20 (do
(compare-and-set! atomic-clock 100 :fin)
#atomic-clock))
I understand that :fin is the new value. But why is it being used instead of number?
I've searched the documentation for :fin being used and cannot find its use other than in the Koans.
Thanks.
In Clojure, the colon makes something called a keyword, or a key:
user=> (type :xyz)
clojure.lang.Keyword
Keys are used in maps, because they're easy to check for equality and to convert to strings. You can see that, later in the same file, :fin is used again in a check for equality:
(= :fin (do
(compare-and-set! __ __ __)
#atomic-clock)))
I have an incoming lazy stream lines from a file I'm reading with tail-seq (to contrib - now!) and I want to process those lines one after one with several "listener-functions" that takes action depending on re-seq-hits (or other things) in the lines.
I tried the following:
(defn info-listener [logstr]
(if (re-seq #"INFO" logstr) (println "Got an INFO-statement")))
(defn debug-listener [logstr]
(if (re-seq #"DEBUG" logstr) (println "Got a DEBUG-statement")))
(doseq [line (tail-seq "/var/log/any/java.log")]
(do (info-listener logstr)
(debug-listener logstr)))
and it works as expected. However, there is a LOT of code-duplication and other sins in the code, and it's boring to update the code.
One important step seems to be to apply many functions to one argument, ie
(listen-line line '(info-listener debug-listener))
and use that instead of the boring and error prone do-statement.
I've tried the following seemingly clever approach:
(defn listen-line [logstr listener-collection]
(map #(% logstr) listener-collection))
but this only renders
(nil) (nil)
there is lazyiness or first class functions biting me for sure, but where do I put the apply?
I'm also open to a radically different approach to the problem, but this seems to be a quite sane way to start with. Macros/multi methods seems to be overkill/wrong for now.
Making a single function out of a group of functions to be called with the same argument can be done with the core function juxt:
=>(def juxted-fn (juxt identity str (partial / 100)))
=>(juxted-fn 50)
[50 "50" 2]
Combining juxt with partial can be very useful:
(defn listener [re message logstr]
(if (re-seq re logstr) (println message)))
(def juxted-listener
(apply juxt (map (fn [[re message]] (partial listner re message))
[[#"INFO","Got INFO"],
[#"DEBUG", "Got DEBUG"]]))
(doseq [logstr ["INFO statement", "OTHER statement", "DEBUG statement"]]
(juxted-listener logstr))
You need to change
(listen-line line '(info-listener debug-listener))
to
(listen-line line [info-listener debug-listener])
In the first version, listen-line ends up using the symbols info-listener and debug-listener themselves as functions because of the quoting. Symbols implement clojure.lang.IFn (the interface behind Clojure function invocation) like keywords do, i.e. they look themselves up in a map-like argument (actually a clojure.lang.ILookup) and return nil if applied to something which is not a map.
Also note that you need to wrap the body of listen-line in dorun to ensure it actually gets executed (as map returns a lazy sequence). Better yet, switch to doseq:
(defn listen-line [logstr listener-collection]
(doseq [listener listener-collection]
(listener logstr)))
I'm writing a small application that includes a password change function with validators for password quality. Currently the validators are specified in a map like so:
(def validations
{:min-length 6
:max-length 32})
The validations map is defined in the validations namespace, but I plan to move it to a configuration namespace later on. My decision to use a map in this way was to make configuration straight-forward for non-programmers.
There are a number of validation functions in the validations namespace that usually take the form:
(defn min-length [n s]
{:req (str "be at least " n " characters long")
:pass? (>= (.length (or s "")) n)})
So with the function above (min-length 3 "clojure") would return {:req "be at least 3 characters long", :pass? true}.
I can validate a password with this function in the validation namespace with this function:
(defn validate-new-password [password]
(into {} (for [[k v] validations]
[k (eval (list (-> k name symbol) v password))])))
The result being something like:
>(validate-new-password "clojure")
{:min-length {:req "be at least 6 characters long", :pass? true},
:max-length {:req "be no longer than 32 characters long", :pass? true},
:min-digits {:req "have at least 1 digit", :pass? false},
:allow-whitespace {:req "not contain spaces or tabs", :pass? true},
:allow-dict-words {:req "not be a dictionary word", :pass? false}}
What is the most practical way to resolve the validation functions when the validate-new-password function is called from outside the validation namespace?
I've tried a number of approaches over the past weeks but I've never been happy with the resulting form (and none of them have worked!).
Generally I guess the question is "how are symbols in a :require'd namespace resolved when called by functions within that namespace?"
I'm also interested on any general comments about my implementation.
There is no need for eval, as in 98% of the cases.
(defn validate-new-password
[password]
(into {} (for [[k v] validations]
[k ((->> k name (symbol "your.name.space") resolve) v password)])))
To answer the general question (from the penultimate paragraph of the question), symbols are normally resolved at compile time -- you have to go out of your way (that is, use resolve1) to postpone resolution to runtime.
That said, you probably don't need to postpone resolution in this case, you simply need to :require the configuration namespace in the validations namespace and refer to config/validations in the relevant spot.
A minor complication would arise if you wanted your config namespace to depend on the validations namespace (i.e. :require or :use it itself, or else another namespace which depends on it). In that case you could provide a handful of configuration setters in the validations namespace and use those from config. E.g.:
(def validations (atom default-validations))
(defn set-validations! [vs] (reset! validations vs))
(defn add-validation! [k v] (swap! validations assoc k v))
Then put #validations in place of validations in validate-new-password.
Alternatively, you could put your default validations in a Var and provide functions to rebind the Var (presumably this will only need to happen once, or else very rarely, so using a Var shouldn't be a problem):
(def validations default-validations)
(defn set-validations! [vs] (.bindRoot validations vs))
(defn add-validation! [k v] (alter-var-root validations assoc k v))
Now if you actually cannot predict, at the time of writing the code, which namespace you'll need to resolve your symbol in, then kotarak's answer (with its use of resolve) is the way to go.
1 You could say eval also allows you to do it, although technically it performs compilation at runtime, so symbol resultion in eval'd forms still occurs at compilation time. The more important thing to remember about it is that it's hardly ever needed.