I am using Nathan Marz's wonderful Specter library. I am doing syntax tree transformations with it, among other things. Suppose that there is a nested data structure:
(def expr
'([:var price (5)]
[:var output ("")]
(clojure.core/cond
(< price 4)
[:var output ("6")]
(= price 5)
[:var output ("==")]
:else
[:var output ("7")])))
I can apply a transformation to all :var nodes via:
(transform [(walker #(and (sequential? %1) (= :var (first %1))))]
transform-fn expr)
However, i'd like to pass the complete path of the navigated-to node to transform-fn, so that to distinguish between parent :var entries and nested ones.
More generally, the action of transform-fn should depend on the complete path of the node being operated on. In a sense, this is similar to inserting VAL for each node visited by walker. How can this be achieved?
Thanks!
Related
I need to turn the following input into output by applying the following two rules:
remove all vectors that have "nope" as last item
remove each map that does not have at least one vector with "ds1" as last item
(def input
[{:simple1 [:from [:simple1 'ds1]]}
{:simple2 [:from-any [[:simple2 'nope] [:simple2 'ds1]]]}
{:walk1 [:from [:sub1 :sub2 'ds1]]}
{:unaffected [:from [:unaffected 'nope]]}
{:replaced-with-nil [:from [:the-original 'ds1]]}
{:concat1 [:concat [[:simple1 'ds1] [:simple2 'ds1]]]}
{:lookup-word [:lookup [:word 'word :word 'ds1]]}])
(def output
[{:simple1 [:from [:simple1 'ds1]]}
{:simple2 [:from-any [[:simple2 'ds1]]]}
{:walk1 [:from [:sub1 :sub2 'ds1]]}
{:replaced-with-nil [:from [:the-original 'ds1]]}
{:concat1 [:concat [[:simple1 'ds1] [:simple2 'ds1]]]}
{:lookup-word [:lookup [:word 'word :word 'ds1]]}])
I was wondering if performing this transformation is possible with zippers?
I'd recommend clojure.walk instead for this kind of general tree transformation. It can take a bit of fiddling to get the replacement functions right but it works nicely with any nesting of Clojure data structures, which AFAIK can be a bit more challenging in a zipper based approach.
We're looking to shrink our tree, so postwalk is my go-to here. It takes a function f and a tree root and goes through the tree, replacing each leaf value with (f leaf), then their parents and their parents etc. until finally replacing the root. (prewalk is similar but proceeds from root and down to leaves, so it's usually more natural when you're growing the tree by splitting branches.)
The strategy here is to somehow construct a function that prunes any branch which meets our removal criteria, but returns any other value unchanged.
(ns shrink-tree
(:require [clojure.walk :refer [postwalk]]))
(letfn[(rule-1 [node]
(and (vector? node)
(= 'nope (last node))))
(rule-2 [node]
(and
(map? node)
(not-any? #(and (vector? %) (= 'ds1 (last %)))
(tree-seq vector? seq (-> node vals first)))))
(remove-marked [node]
(if (coll? node)
(into (empty node) (remove (some-fn rule-1 rule-2) node))
node))]
(= output (postwalk remove-marked input)))
;; => true
Here the fns rule-1 and rule-2 try to turn your rules into predicates and remove-marked:
If a node is a collection, returns the same collection, less any members for which rule1 or rule2 return truthy when called with that member. To check for either one at the same time we combine the predicates with some-fn.
Otherwise returns the same node. This is how we keep values like 'ds1 or :from-any around.
You might also want to consider looking at specter. It supports these sorts of transformations by allowing you to select and transform arbitrarily complex structures.
After a while of working with Clojure, I have accumulated some knowledge on its laziness. I know whether a frequently-used API such as map is lazy. However, I still feel dubious when I start using an unfamiliar API such as with-open.
Is there any document that shows a complete list of lazy APIs of Clojure's core module?
You can find functions that return lazy sequences by opening up the Clojure code https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj
and searching for "Returns a lazy"
I am not aware of any curated lists of them.
The rule of thumb is: if it returns a sequence, it will be a lazy sequence, if it returns a value, it will force evaluation.
When using a new function, macro or special form, read the docstring. Most development environments have a key to show the docstring, or at least navigate to the source (where you can see the docstring), and there is always http://clojure.org/api/api.
In the case of with-open:
with-open
macro
Usage: (with-open bindings & body)
bindings => [name init ...]
Evaluates body in a try expression with names bound to the values
of the inits, and a finally clause that calls (.close name) on each
name in reverse order.
We can see that the result of calling with-open is evaluation of the expression with a final close. So we know that there is nothing lazy about it. However that doesn't mean you don't need to think about laziness inside with-open, quite the opposite!
(with-open [r (io/reader "myfile")]
(line-seq r))
This is a common trap. line-seq returns a lazy sequence! The problem here is that the lazy sequence will be realized after the file is closed, because the file is closed when exiting the scope of with-open. So you need to fully process the lazy sequence before exiting the with-open scope.
My advice is to avoid trying to think about your program as having 'lazy bits' and 'immediate bits', but instead just be mindful that when io or side-effects are involved you need to take care of when things happen as well as what should happen.
digging on a Timothy Pratley's proposal to search in doc:
let's make it fun!
your repl has everything that you need to find out a list of lazy functions.
first of all, there is a clojure.repl/doc macro, which prints documentation to out in repl
user> (doc +)
-------------------------
clojure.core/+
([] [x] [x y] [x y & more])
Returns the sum of nums. (+) returns 0. Does not auto-promote
longs, will throw on overflow. See also: +'
nil
unfortunately we can't get a string of it simply, but we can always rebind the *out* to be a StringWriter, and then get its string value.
so, whan we want to take all the symbols from clojure.core namespace, get their docs, write them all to string, and find every one that contains "returns a lazy". Here comes the help: clojure.core/ns-publics, returning a map of public names to their vars:
user> (take 10 (ns-publics 'clojure.core))
([primitives-classnames #'clojure.core/primitives-classnames]
[+' #'clojure.core/+']
[decimal? #'clojure.core/decimal?]
[restart-agent #'clojure.core/restart-agent]
[sort-by #'clojure.core/sort-by]
[macroexpand #'clojure.core/macroexpand]
[ensure #'clojure.core/ensure]
[chunk-first #'clojure.core/chunk-first]
[eduction #'clojure.core/eduction]
[tree-seq #'clojure.core/tree-seq])
so we just need to get all the keys from there and lookup for their docs.
Let's make a macro for that:
user> (defmacro all-docs []
(let [names (keys (ns-publics 'clojure.core))]
`(binding [*out* (java.io.StringWriter.)]
(do ~#(map #(list `doc %) names))
(str *out*))))
#'user/all-docs
it does just what i've said, gets all publics' docs to string.
now we simply process it:
user> (def all-doc-items (clojure.string/split
(all-docs)
#"-------------------------"))
#'user/all-doc-items
user> (nth all-doc-items 10)
"\nclojure.core/tree-seq\n([branch? children root])\n Returns a lazy sequence of the nodes in a tree, via a depth-first walk.\n branch? must be a fn of one arg that returns true if passed a node\n that can have children (but may not). children must be a fn of one\n arg that returns a sequence of the children. Will only be called on\n nodes for which branch? returns true. Root is the root node of the\n tree.\n"
and now just filter them:
user> (def all-lazy-fns (filter #(re-find #"(?i)returns a lazy" %) all-doc-items))
#'user/all-lazy-fns
user> (count all-lazy-fns)
30
user> (println (take 3 all-lazy-fns))
(
clojure.core/tree-seq
([branch? children root])
Returns a lazy sequence of the nodes in a tree, via a depth-first walk.
branch? must be a fn of one arg that returns true if passed a node
that can have children (but may not). children must be a fn of one
arg that returns a sequence of the children. Will only be called on
nodes for which branch? returns true. Root is the root node of the tree.
clojure.core/keep-indexed
([f] [f coll])
Returns a lazy sequence of the non-nil results of (f index item). Note,
this means false return values will be included. f must be free of
side-effects. Returns a stateful transducer when no collection is
provided.
clojure.core/take-nth
([n] [n coll])
Returns a lazy seq of every nth item in coll. Returns a stateful
transducer when no collection is provided.
)
nil
And now use these all-lazy-fns however you want.
Clojure noob here.
I want to pull the front and rest out of a Set. Doing (front #{1}) and (rest #{1}) produce 1 and () respectively, which is mostly what I'd expect.
However in the code below, I use the destructuring [current-node & open-nodes] #{start} in my loop to pull something out of the set (at this point I don't really care about if it was the first or last item. I just want this form working) and it breaks.
Here's my function, half-implementing a grid search:
(defn navigate-to [grid start dest]
"provides route from start to dest, not including start"
(loop [[current-node & open-nodes] #{start} ;; << throws exception
closed-nodes #{}]
(if (= dest current-node)
[] ;; todo: return route
(let [all-current-neighbours (neighbours-of grid current-node) ;; << returns a set
open-neighbours (set/difference all-current-neighbours closed-nodes)]
(recur (set/union open-nodes open-neighbours)
(conj closed-nodes current-node))))))
When stepping through (with Cider), on the start of the first loop, it throws this exception:
UnsupportedOperationException nth not supported on this type: PersistentHashSet clojure.lang.RT.nthFrom (RT.java:933)
I could use a nested let form that does first/rest manually, but that seems wasteful. Is there a way to get destructured Sets working like this in the loop form? Is it just not supported on Sets?
Sets are unordered, so positional destructuring doesn't make much sense.
According to the documentation for Special Forms, which treats destructuring as well, sequential (vector) binding is specified to use nth and nthnext to look up the elements to bind.
Vector binding-exprs allow you to bind names to parts of sequential things (not just vectors), like vectors, lists, seqs, strings, arrays, and anything that supports nth.
Clojure hash sets (being instances of java.util.Set) do not support lookup by index.
I don't know the context of your example code, but in any case pouring the set contents into an ordered collection, for example (vec #{start}), would make the destructuring work.
As mentioned by others you cannot bind a set to a vector literal, because a set is not sequential. So even this simple let fails with nth not supported:
(let [[x] #{1}])
You could work around this by "destructuring" the set with the use of first and disj:
(loop [remaining-nodes #{start}
closed-nodes #{}]
(let [current-node (first remaining-nodes)
open-nodes (disj remaining-nodes current-node)]
;; rest of your code ...
))
Using (rest remaining-nodes) instead of (disj remaining-nodes current-node) could be possible, but as sets are unordered, rest is in theory not obliged to take out the same element as was extracted with first. Anyway disj will do the job.
NB: be sure to detect remaining-nodes being nil, which could lead to an endless loop.
Algorithm for returning the route
For implementing the missing part in the algorithm (returning the route) you could maintain
a map of paths. It would have one path for each visited node: a vector with the nodes leading from the start node to that node, keyed by that node.
You could use reduce to maintain that map of paths as you visit new nodes. With a new function used together with that reduce and an added nil test, the program could look like this:
(defn add-path [[path paths] node]
"adds a node to a given path, which is added to a map of paths, keyed by that node"
[path (assoc paths node (conj path node))])
(defn navigate-to [grid start dest]
"provides route from start to dest, including both"
(loop [remaining-nodes #{start}
closed-nodes #{}
paths (hash-map start [start])]
(let [current-node (first remaining-nodes)
current-path (get paths current-node)
all-current-neighbours (neighbours-of grid current-node)
open-neighbours (set/difference all-current-neighbours closed-nodes)]
(if (contains? #{dest nil} current-node)
current-path ;; search complete
(recur (set/union (disj remaining-nodes current-node) open-neighbours)
(conj closed-nodes current-node)
(second (reduce add-path [current-path paths] open-neighbours)))))))
The essence of the algorithm is still the same, although I merged the original let with the one needed for destructuring the nodes. This is not absolutely needed, but it probably makes the code more readable.
Test
I tested this with a poor-mans definition of grid and neighbours-of, based on this graph (digits are nodes, bars indicate linked nodes:
0--1 2
| | |
3--4--5
|
6--7--8
This graph seems a good candidate for a test as it has a loop, a dead end, and is connected.
The graph is encoded with grid being a vector, where each element represents a node. An element's index in that vector is the node's identifier. The content of each element is a set of neighbours, making the neighbours-of function a trivial thing (your implementation will be different):
(def grid [#{1 3} #{0 4} #{5}
#{0 4 6} #{1 3 5} #{2 4}
#{3 7} #{6 8} #{7} ])
(defn neighbours-of [grid node]
(get grid node))
Then the test is to find the route from node 0 to node 8:
(println (navigate-to grid 0 8))
Output is:
[0 1 4 3 6 7 8]
This outcome demonstrates that the algoritm does not guarantee a shortest route, only that a route will be found if it exists. I suppose the outcome could be different on different engines, depending on how the Conjure internals decide which element to take from a set with first.
After removing one of the necessary node links, like the one between node 7 and 8, the output is nil.
NB: I found this an interesting question, and probably went a bit too far in my answer.
I've got a recursive function building a list:
(defn- traverse-dir
"Traverses the (source) directory, preorder"
[src-dir dst-root dst-step ffc!]
(let [{:keys [options]} *parsed-args*
uname (:unified-name options)
[dirs files] (list-dir-groomed (fs/list-dir src-dir))
... recursive call of traverse-dir is the last expression of dir-handler
(doall (concat (map-indexed (dir-handler) dirs) (map-indexed (file-handler) files))))) ;; traverse-dir
The list, built by traverse-dir, is recursive, while I want a flat one:
flat-list (->> (flatten recursive-list) (partition 2) (map vec))
Is there a way of building the flat list in the first place? Short of using mutable lists, that is.
I don't quite understand your context with a dir-handler that is called with nothing and returns a function which expects indices and directories, list-dir-groomed and all of that, but I'd recommend a look at tree-seq:
(defn tree-seq
"Returns a lazy sequence of the nodes in a tree, via a depth-first walk.
branch? must be a fn of one arg that returns true if passed a node
that can have children (but may not). children must be a fn of one
arg that returns a sequence of the children. Will only be called on
nodes for which branch? returns true. Root is the root node of the
tree."
{:added "1.0"
:static true}
[branch? children root]
(let [walk (fn walk [node]
(lazy-seq
(cons node
(when (branch? node)
(mapcat walk (children node))))))]
(walk root)))
My go-to use here is
(tree-seq #(.isDirectory %) #(.listFiles %) (clojure.java.io/as-file file-name))
but your context might mean that doesn't work. You can change to different functions for getting child files if you need to sanitize those, or you can just use filter on the output. If that's no good, the same pattern of a local fn from nodes into pre-walks that handles children by recursively mapcatting itself over them seems pretty applicable.
I am new to clojure as well as to Functional Programming. I am trying to traverse a tree in pre-order using:
(def preordercoll [])
(deftrace preorder [mytree]
(if-not (empty? mytree)
(do (println "position"(value mytree))
(cons (value mytree) (preorder (left-child mytree)))
(cons (value mytree) (preorder (right-child mytree))))
)preordercoll) )
(preorder [45[65 [90 nil nil] [81 nil nil]] [72[82 nil nil][96 nil nil]]])
I am unable to append values of node in some list, like I tried using 'conj' operation on global variable preordercoll, but yes it doesn't work like object oriented , so I tried using cons, but only few values are returned, that too in improper order. Can anyone guide me what mistake am making?
I also thought of using Partial function but could not find how to supply value of node in recursive manner. Am not asking for code but please draw me in right direction to get collection of values in pre-order.
you're on the right track, just have the nesting of the cons calls a little off. First a note on evaluation of forms in clojure. One of the key ideas is that every form evaluates to something* which is why there is no "return" statement in the language, because you could say everything would be a return statement so what's the point in having it. In the case of a do expression the return value of the expression is the last statement so:
(do 1 2 3)
returns (evaluates to) 3. In the do expression in your code it returns the result of the second cons, and the first cons has no effect.
(do (println "position"(value mytree))
(cons (value mytree) (preorder (left-child mytree))) ;; <-- this does nothing
(cons (value mytree) (preorder (right-child mytree))))
instead it sounds like you would like an expression that starts with the result of calling preorder on the left tree, then concatinates the result of calling preorder on the right tree, then attaches the current node's value to the front of that.
(let [left-side (preorder (left-child mytree))
right-side (preorder (left-child mytree))
this-value (value mytree)]
(do (println "position" this-value)
(cons this-value (concat right-side left-side))
*(for the pedants) "except the ignore reader maco #_"