I have created the following graph using Clojure Zipper
A
/ | \
B C D
/ \
E F
using the following code:
(require '[clojure.zip :as z])
(def g (z/vector-zip ["A" ["B" "C" "D"["E" "F"]]]))
Now I want to create a Visualization in d3, So that I want to represent the graph in EDN format like,
[{:from "A" :to "B"}
{:from "A" :to "C"}
{:from "A" :to "D"}
{:from "D" :to "E"}
{:from "D" :to "F"}]
I've tried this
(loop [t g]
(if-not (z/end? t)
(do
(if-not (z/branch? t)
(println {:from (-> t (get 1) :ppath :l) :to (z/node t)})
)
(recur (z/next t))
)
)
)
The only problem is with child E & F, I could not track its parent node D.
You could slightly change the syntax for your tree there to have bascially a pair of parent and childs stored in a vector and then roll your own zipper. E.g.
(def v [\a [\b [\c [\z]] [\d [\e \f]]]])
(def g
(z/zipper
vector? ; a vector/pair is a branch
#(concat (second %)) ; might be a smarter way to get the childs
nil ; don't support edit
v))
(loop [t (z/next g)] ; skip first
(if-not (z/end? t)
(do
(println {
:from (-> t z/up z/node first) ; parents are always vectors
:to (if (z/branch? t) (-> t z/node first) (z/node t))}) ; if vector, then first
(recur (z/next t)))))
;;=> {:from a, :to b}
;;=> {:from a, :to c}
;;=> {:from c, :to z}
;;=> {:from a, :to d}
;;=> {:from d, :to e}
;;=> {:from d, :to f}
I think that #cfrick answer makes sens. It is more convenient to have pairs of [parent [children]] in your tree vector. But nonetheless, here is a solution where you can keep your structure and use vector-zip.
parent-node takes the current zipper and returns the parent node in the define structure. E.g when zipper is on node :c, parent-node returns :a.
edge-seq build a lazy-seq of the graph edges. It can be useful to filter specific edges.
(defn edges [vtree]
(letfn [(parent-node [vz]
(if-not (-> vz z/node vector?)
(-> vz
z/up
z/left
(#(when % (z/node %))))))
(edge-seq [vz]
(if-not (z/end? vz)
(if-let [p (parent-node vz)]
(cons {:from p :to (z/node vz)}
(lazy-seq (edge-seq (-> vz z/next))))
(edge-seq (-> vz z/next)))))]
(let [vz (z/vector-zip vtree)]
(edge-seq vz))))
With the following vector/tree :
user> (def v [:a [:b :c :d [:e [:h] :f] :g]])
#'user/v
user> (pprint (edges v))
({:from :a, :to :b}
{:from :a, :to :c}
{:from :a, :to :d}
{:from :d, :to :e}
{:from :e, :to :h}
{:from :d, :to :f}
{:from :a, :to :g})
Keep only edges from :d
user> (pprint (filter #(= :d (:from %)) (edges v)))
({:from :d, :to :e} {:from :d, :to :f})
Related
Getting data from the database as a list of maps (LazySeq) leaves me in need of transforming it into a map of maps.
I tried to 'assoc' and 'merge', but that didn't bring the desired result because of the nesting.
This is the form of my data:
(def data (list {:structure 1 :cat "A" :item "item1" :val 0.1}
{:structure 1 :cat "A" :item "item2" :val 0.2}
{:structure 1 :cat "B" :item "item3" :val 0.4}
{:structure 2 :cat "A" :item "item1" :val 0.3}
{:structure 2 :cat "B" :item "item3" :val 0.5}))
I would like to get it in the form
=> {1 {"A" {"item1" 0.1}
"item2" 0.2}}
{"B" {"item3" 0.4}}
2 {"A" {"item1" 0.3}}
{"B" {"item3" 0.5}}}
I tried
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply merge-with into))
This gives
{1 {"A" {"item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
By merging I lose some entries, but I can't think of any other way. Is there a simple way? I was even about to try to use specter.
Any thoughts would be appreciated.
If I'm dealing with nested maps, first stop is usually to think about update-in or assoc-in - these take a sequence of the nested keys. For a problem like this where the data is very regular, it's straightforward.
(assoc-in {} [1 "A" "item1"] 0.1)
;; =>
{1 {"A" {"item1" 0.1}}}
To consume a sequence into something else, reduce is the idiomatic choice. The reducing function is right on the edge of the complexity level I'd consider an anonymous fn for, so I'll pull it out instead for clarity.
(defn- add-val [acc line]
(assoc-in acc [(:structure line) (:cat line) (:item line)] (:val line)))
(reduce add-val {} data)
;; =>
{1 {"A" {"item1" 0.1, "item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
Which I think was the effect you were looking for.
Roads less travelled:
As your sequence is coming from a database, I wouldn't worry about using a transient collection to speed the aggregation up. Also, now I think about it, dealing with nested transient maps is a pain anyway.
update-in would be handy if you wanted to add up any values with the same key, for example, but the implication of your question is that structure/cat/item tuples are unique and so you just need the grouping.
juxt could be used to generate the key structure - i.e.
((juxt :structure :cat :item) (first data))
[1 "A" "item1"]
but it's not clear to me that there's any way to use this to make the add-val fn more readable.
You may continue to use your existing code. Only the final merge has to change:
(defn deep-merge [& xs]
(if (every? map? xs)
(apply merge-with deep-merge xs)
(apply merge xs)))
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply deep-merge))
;; =>
{1
{"A" {"item1" 0.1, "item2" 0.2},
"B" {"item3" 0.4}},
2
{"A" {"item1" 0.3},
"B" {"item3" 0.5}}}
Explanation: your original (apply merge-with into) only merge one level down. deep-merge from above will recurse into all nested maps to do the merge.
Addendum: #pete23 - one use of juxt I can think of is to make the function reusable. For example, we can extract arbitrary fields with juxt, then convert them to nested maps (with yet another function ->nested) and finally do a deep-merge:
(->> data
(map (juxt :structure :cat :item :val))
(map ->nested)
(apply deep-merge))
where ->nested can be implemented like:
(defn ->nested [[k & [v & r :as t]]]
{k (if (seq r) (->nested t) v)})
(->nested [1 "A" "item1" 0.1])
;; => {1 {"A" {"item1" 0.1}}}
One sample application (sum val by category):
(let [ks [:cat :val]]
(->> data
(map (apply juxt ks))
(map ->nested)
(apply (partial deep-merge-with +))))
;; => {"A" 0.6000000000000001, "B" 0.9}
Note deep-merge-with is left as an exercise for our readers :)
(defn map-values [f m]
(into {} (map (fn [[k v]] [k (f v)])) m))
(defn- transform-structures [ss]
(map-values (fn [cs]
(into {} (map (juxt :item :val) cs))) (group-by :cat ss)))
(defn transform [data]
(map-values transform-structures (group-by :structure data)))
then
(transform data)
Is there a short form/macro that allows me to do
(defn f [a b c]
{a b c})
instead of
(defn f [a b c]
{:a a :b b :c c})
(defmacro as-map [& syms]
(zipmap (map keyword syms) syms))
Usage:
(def a 42)
(def b :foo)
(as-map a b)
;;-> {:a 42 :b :foo}
Note that to support namespaced keywords, you'd have to drop support for ns aliases if you want to keep it as short:
(defmacro as-map [& syms]
(zipmap (map keyword syms) (map (comp symbol name) syms)))
Usage:
(def a 42)
(def b :foo)
(as-map example/a foo-of/b)
;;-> {:example/a 42 :foo-of/b :foo}
Advice: Likely not a good idea, saves you a few keyboard hits at the cost of readability and expressivity and flexibility in naming local bindings.
This shows the steps. Remove the println's for actual use:
(ns clj.core
(:gen-class))
(defmacro hasher [& args]
(let [keywords (map keyword args)
values args
keyvals-list (interleave keywords values)
]
(println "keywords " keywords)
(println "values " values)
(println "keyvals-list " keyvals-list)
`(hash-map ~#keyvals-list)
)
)
(def a 1)
(def b 2)
(println \newline "result: " (hasher a b))
> lein run
keywords (:a :b)
values (a b)
keyvals-list (:a a :b b)
result: {:b 2, :a 1}
This is an old snippet of mine I've had kicking around for a while.
(declare ^:private restructure*)
(defn ^:private restructure-1 [m [e k]]
(cond
(= :strs e) (reduce #(assoc %1 (name %2) %2) m k)
(= :keys e) (reduce #(assoc %1 (keyword (namespace %2) (name %2)) %2) m k)
:else (assoc m k (restructure* e))))
(defn ^:private restructure* [form]
(if-not (map? form)
form
(as-> {} v
(reduce restructure-1 v form)
`(hash-map ~#(mapcat identity v)))))
(defmacro restructure [form]
(restructure* form))
The idea is that it provides the complement of clojure.core/destructure which goes from a destructuring form to bindings, this captures bindings and constructs a datastructure.
(let [x 1 y 2 z 3]
(restructure {:keys [x y z]}))
;; => {:x 1 :y 2 :z 3}
I have a collection of maps
(def a '({:id 9345 :value 3 :type "orange"}
{:id 2945 :value 2 :type "orange"}
{:id 145 :value 3 :type "orange"}
{:id 2745 :value 6 :type "apple"}
{:id 2345 :value 6 :type "apple"}))
I want to group this first by value, followed by type.
My output should look like:
{
:orange [{
:value 3,
:id [9345, 145]
}, {
:value 2,
:id [2935]
}],
:apple [{
:value 6,
:id [2745, 2345]
}]
}
How would I do this in Clojure? Appreciate your answers.
Thanks!
Edit:
Here is what I had so far:
(defn by-type-key [data]
(group-by #(get % "type") data))
(reduce-kv
(fn [m k v] (assoc m k (reduce-kv
(fn [sm sk sv] (assoc sm sk (into [] (map #(:id %) sv))))
{}
(group-by :value (map #(dissoc % :type) v)))))
{}
(by-type-key a))
Output:
=> {"orange" {3 [9345 145], 2 [2945]}, "apple" {6 [2745 2345], 3 [125]}}
I just couldnt figure out how to proceed next...
Your requirements are a bit inconsistent (or rather irregular) - you use :type values as keywords in the result, but the rest of the keywords are carried through. Maybe that's what you must do to satisfy some external formats - otherwise you need to either use the same approach as with :type through, or add a new keyword to the result, like :group or :rows and keep the original keywords intact. I will assume the former approach for the moment (but see below, I will get to the shape as you want it,) so the final shape of data is like
{:orange
{:3 [9345 145],
:2 [2945]},
:apple
{:6 [2745 2345]}
}
There is more than one way of getting there, here's the gist of one:
(group-by (juxt :type :value) a)
The result:
{["orange" 3] [{:id 9345, :value 3, :type "orange"} {:id 145, :value 3, :type "orange"}],
["orange" 2] [{:id 2945, :value 2, :type "orange"}],
["apple" 6] [{:id 2745, :value 6, :type "apple"} {:id 2345, :value 6, :type "apple"}]}
Now all rows in your collection are grouped by the keys you need. From this, you can go and get the shape you want, say to get to the shape above you can do
(reduce
(fn [m [k v]]
(let [ks (map (comp keyword str) k)]
(assoc-in m ks
(map :id v))))
{}
(group-by (juxt :type :value) a))
The basic idea is to get the rows grouped by the key sequence (and that's what group-by and juxt do,) and then combine reduce and assoc-in or update-in to beat the result into place.
To get exactly the shape you described:
(reduce
(fn [m [k v]]
(let [type (keyword (first k))
value (second k)
ids (map :id v)]
(update-in m [type]
#(conj % {:value value :id ids}))))
{}
(group-by (juxt :type :value) a))
It's a bit noisy, and it might be harder to see the forest for the trees - that's why I simplified the shape, to highlight the main idea. The more regular your shapes are, the shorter and more regular your functions become - so if you have control over it, try to make it simpler for you.
I would do the transform in two stages (using reduce):
the first to collect the values
the second for formating
The following code solves your problem:
(def a '({:id 9345 :value 3 :type "orange"}
{:id 2945 :value 2 :type "orange"}
{:id 145 :value 3 :type "orange"}
{:id 2745 :value 6 :type "apple"}
{:id 2345 :value 6 :type "apple"}))
(defn standardise [m]
(->> m
;; first stage
(reduce (fn [out {:keys [type value id]}]
(update-in out [type value] (fnil #(conj % id) [])))
{})
;; second stage
(reduce-kv (fn [out k v]
(assoc out (keyword k)
(reduce-kv (fn [out value id]
(conj out {:value value
:id id}))
[]
v)))
{})))
(standardise a)
;; => {:orange [{:value 3, :id [9345 145]}
;; {:value 2, :id [2945]}],
;; :apple [{:value 6, :id [2745 2345]}]}
the output of the first stage is:
(reduce (fn [out {:keys [type value id]}]
(update-in out [type value] (fnil #(conj % id) [])))
{}
a)
;;=> {"orange" {3 [9345 145], 2 [2945]}, "apple" {6 [2745 2345]}}
You may wish to use the built-in function group-by. See http://clojuredocs.org/clojure.core/group-by
In my Clojure webapp I have various model namespaces with functions that take a map as an agrument and somehow insert that map into a database. I would like to be able take out only the desired keys from the map before I do the insert.
A basic example of this is:
(let [msg-keys [:title :body]
msg {:title "Hello" :body "This is an example" :somekey "asdf" :someotherkey "asdf"}]
(select-keys msg msg-keys))
;; => {:title "Hello" :body "This is an example"}
select-keys is not an option when the map is somewhat complex and I would like to select a specific set of nested keys:
(let [person {:name {:first "john" :john "smith"} :age 40 :weight 155 :something {:a "a" :b "b" :c "c" :d "d"}}]
(some-select-key-fn person [:name [:first] :something [:a :b]]))
;; => {:name {:first "john"} :something {:a "a" :b "b"}}
Is there a way to do this with the core functions? Is there a way do this purely with destructuring?
This topic was discussed in the Clojure Google Group along with a few solutions.
Destructuring is probably the closest to a "core" capability, and may be a fine solution if your problem is rather static and the map has all of the expected keys (thus avoiding nil). It could look like:
(let [person {:name {:first "john" :john "smith"} :age 40 :weight 155 :something {:a "a" :b "b" :c "c" :d "d"}}
{{:keys [first]} :name {:keys [a b]} :something} person]
{:name {:first first} :something {:a a :b b}})
;; => {:name {:first "john"}, :something {:a "a", :b "b"}}
Below is a survey of the solutions in the Clojure Google Group thread, applied to your sample map. They each have a different take on how to specify the nested keys to be selected.
Here is Christophe Grand's solution:
(defprotocol Selector
(-select [s m]))
(defn select [m selectors-coll]
(reduce conj {} (map #(-select % m) selectors-coll)))
(extend-protocol Selector
clojure.lang.Keyword
(-select [k m]
(find m k))
clojure.lang.APersistentMap
(-select [sm m]
(into {}
(for [[k s] sm]
[k (select (get m k) s)]))))
Using it requires a slightly modified syntax:
(let [person {:name {:first "john" :john "smith"} :age 40 :weight 155 :something {:a "a" :b "b" :c "c" :d "d"}}]
(select person [{:name [:first] :something [:a :b]}]))
;; => {:something {:b "b", :a "a"}, :name {:first "john"}}
Here is Moritz Ulrich's solution (he cautions that it doesn't work on maps with seqs as keys):
(defn select-in [m keyseq]
(loop [acc {} [k & ks] (seq keyseq)]
(if k
(recur
(if (sequential? k)
(let [[k ks] k]
(assoc acc k
(select-in (get m k) ks)))
(assoc acc k (get m k)))
ks)
acc)))
Using it requires another slightly modified syntax:
(let [person {:name {:first "john" :john "smith"} :age 40 :weight 155 :something {:a "a" :b "b" :c "c" :d "d"}}]
(select-in person [[:name [:first]] [:something [:a :b]]]))
;; => {:something {:b "b", :a "a"}, :name {:first "john"}}
Here is Jay Fields's solution:
(defn select-nested-keys [m top-level-keys & {:as pairs}]
(reduce #(update-in %1 (first %2) select-keys (last %2)) (select-keys m top-level-keys) pairs))
It uses a different syntax:
(let [person {:name {:first "john" :john "smith"} :age 40 :weight 155 :something {:a "a" :b "b" :c "c" :d "d"}}]
(select-nested-keys person [:name :something] [:name] [:first] [:something] [:a :b]))
;; => {:something {:b "b", :a "a"}, :name {:first "john"}}
Here is Baishampayan Ghose's solution:
(defprotocol ^:private IExpandable
(^:private expand [this]))
(extend-protocol IExpandable
clojure.lang.Keyword
(expand [k] {k ::all})
clojure.lang.IPersistentVector
(expand [v] (if (empty? v)
{}
(apply merge (map expand v))))
clojure.lang.IPersistentMap
(expand [m]
(assert (= (count (keys m)) 1) "Number of keys in a selector map can't be more than 1.")
(let [[k v] (-> m first ((juxt key val)))]
{k (expand v)}))
nil
(expand [_] {}))
(defn ^:private extract* [m selectors expand?]
(let [sels (if expand? (expand selectors) selectors)]
(reduce-kv (fn [res k v]
(if (= v ::all)
(assoc res k (m k))
(assoc res k (extract* (m k) v false))))
{} sels)))
(defn extract
"Like select-keys, but can select nested keys.
Examples -
(extract [{:b {:c [:d]}} :g] {:a 1 :b {:c {:d 1 :e 2}} :g 42 :xxx 11})
;=> {:g 42, :b {:c {:d 1}}}
(extract [:g] {:a 1 :b {:c {:d 1 :e 2}} :g 42 :xxx 11})
;=> {:g 42}
(extract [{:b [:c]} :xxx] {:a 1 :b {:c {:d 1 :e 2}} :g 42 :xxx 11})
;=> {:b {:c {:d 1, :e 2}}, :xxx 11}
Also see - exclude"
[selectors m]
(extract* m selectors true))
It uses another syntax (and the parameters are reversed):
(let [person {:name {:first "john" :john "smith"} :age 40 :weight 155 :something {:a "a" :b "b" :c "c" :d "d"}}]
(extract [{:name [:first]} {:something [:a :b]}] person))
;; => {:name {:first "john"}, :something {:a "a", :b "b"}}
Your best bet is probably to use select keys on each nested portion of the structure.
(-> person
(select-keys [:name :something])
(update-in [:name] select-keys [:first])
(update-in [:something] select-keys [:a :b]))
You could of course use a generalized version of the above to implement the syntax you suggest in a function (with a reduce rather than a -> form, most likely, and recursive calls for each nested selection of keys). Destructuring would not help much, it makes binding nested data convenient, but isn't really that useful for constructing values.
Here is how I would do it with reduce and recursion:
(defn simplify
[m skel]
(if-let [kvs (not-empty (partition 2 skel))]
(reduce (fn [m [k nested]]
(if nested
(update-in m [k] simplify nested)
m))
(select-keys m (map first kvs))
kvs)
m))
note that your proposed argument format is inconvenient, so I changed it slightly
user=> (simplify {:name {:first "john" :john "smith"}
:age 40
:weight 155
:something {:a "a" :b "b" :c "c" :d "d"}}
[:name [:first nil] :something [:a nil :b nil]])
{:something {:b "b", :a "a"}, :name {:first "john"}}
the syntax you propose will require a more complex implementation
the motivation is for checking what has changed in a deeply nest map, kind of like a reverse of update-in.
This is a simple example:
(def p1 {:a {:a1 :1 :a2 :2}
:b {:b1 :1 :b2 :2}})
(def p2 (update-in p1 [:a :a1] (constantly :updated))
;; => {:a {:a1 :updated :a2 :2}
;; :b {:b1 :1 :b2 :2}}
(what-changed? p1 p2)
;; => {:keys [:a :a1] :value :updated)
(what-changed? p2 p1)
;; => {:keys [:a :a1] :value :2)
I'm hoping that because clojure maps are persistent data-structures, there may be a smart algorithm to figure this out by looking at the underlying structure as opposed to walking through the nested maps and comparing the difference.
I would imagine a straight-forward recursive algorithm like (if (= map1 map2) :no-change (recursively-check-children ...)) already takes good advantage of the structure sharing, unless you're dealing with very wide trees (lots of branches per node). In any case you probably want to check out clojure.data/diff which solves a general, non-recursive, version of your problem, and also starts out by doing a straight (= a b)
Persistent data structures is only about implementation and not about "looking at the underlying structure".
As Joost said (+1) you can use "diff". It only needs to convert the answer using your "{:keys ... :value ...}" pattern:
(def p1 {:a {:a1 :1 :a2 :2}
:b {:b1 :1 :b2 {:b11 :11 :b22 :22}}})
(def p2 {:a {:a1 :updated1 :a2 :2}
:b {:b1 :1 :b2 {:b11 :updated2 :b22 :updated3}}})
(defn what-changed?* [m]
(if (not (map? m))
[(list m)]
(apply concat (map (fn [k]
(map (fn [nest-k]
(conj nest-k k))
(nested-keys (m k))))
(keys m)))))
(defn what-changed? [m1 m2]
(map (fn [l] {:keys (drop-last l) :value (last l)})
(nested-keys (second (data/diff m1 m2)))))
Test:
(what-changed? p1 p2)
-> ({:keys (:a :a1), :value :updated1}
{:keys (:b :b2 :b11), :value :updated2}
{:keys (:b :b2 :b22), :value :updated3})
(what-changed? p2 p1)
-> ({:keys (:a :a1), :value :1}
{:keys (:b :b2 :b11), :value :11}
{:keys (:b :b2 :b22), :value :22})
BTW in your case you can modify hashmap by "assoc-in" instead of "update-in":
(assoc-in {:a {:a1 :1 :a2 :2}
:b {:b1 :1 :b2 :2}}
[:a :a1] :updated)
-> {:a {:a2 :2, :a1 :updated}
:b {:b2 :2, :b1 :1}}