I have one map that looks like
{:a {:b {:c {:d [[1 2 3]]}
:e "Hello"}}}
and another map that looks like {:a {:b {:c {:d [[4 5 6]]}}}}. How can I merge these two maps so that the result looks like this?
{:a {:b {:c {:d [[1 2 3] [4 5 6]]}
:e "Hello"}}}
For such a simple use-case, you might choose to stick with core Clojure functions:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(dotest
(let [x {:a {:b {:c {:d [[1 2 3]]}
:e "Hello"}}}
y {:a {:b {:c {:d [[4 5 6]]}}}}
yseq (get-in y [:a :b :c :d])
r1 (update-in x [:a :b :c :d] into yseq)
r2 (update-in x [:a :b :c :d] #(into % yseq)) ]
(is= r1 r2
{:a {:b {:c {:d [[1 2 3]
[4 5 6]]},
:e "Hello"}}})))
As shown for r2, I sometimes think it is clearer to use a self-contained closure function to explicitly show where the old value % is being used. I am often even more explicit, writing the r2 closure as:
(fn [d-val]
(into d-val yseq))
instead of using the #(...) reader macro.
You can use deep-merge-with from the deprecated clojure-contrib.map-utils:
(defn deep-merge-with [f & maps]
(apply
(fn m [& maps]
(if (every? map? maps)
(apply merge-with m maps)
(apply f maps)))
maps))
(def m1
{:a {:b {:c {:d [[1 2 3]]}
:e "Hello"}}})
(def m2
{:a {:b {:c {:d [[4 5 6]]}}}})
(deep-merge-with into m1 m2)
;; => {:a {:b {:c {:d [[1 2 3] [4 5 6]]}
;; :e "Hello"}}}
Related
Hi l have 2 maps like these (with the possibility of having multiple maps)
(def map1 {:a {:b 1} :c ["dog"]})
(def map2 {:a {:b 2} :c ["cat"]})
l need to have this as a return {:a {:b 3} :c ["dog" "cat"]}
How can l do this?
Solution using generic functions
You could use generic functions - let's call it here o+ (operator +) - by making use of Clojure's multiple dispatch ability. By multiple dispatch you avoid if - else or cond clauses - for distinguishing the argument types/classes - thereby the code stays extensible (further cases can be added without having to change existing code). The function at the end of the defmulti definition is the dispatch function. In the defmethod form - right before the actual function arguments list (vector), you list the dispatch case for which then Clojure looks when deciding which method to use.
(defmulti o+ (fn [& args] (mapv class args)))
(defmethod o+ [Number Number] [x y] (+ x y)) ;; Numbers get added
(defmethod o+ [clojure.lang.PersistentVector clojure.lang.PersistentVector]
[x y] (into (empty x) (concat x y))) ;; Vectors concatentated
(defmethod o+ [clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap]
[x y] (merge-with o+ x y)) ;; Maps merged-with #'o+ (recursive definition!)
Test
You can fuse then two maps with the (now recursively defined) o+ operator:
(def map1 {:a {:b 1} :c ["dog"]})
(def map2 {:a {:b 2} :c ["cat"]})
(o+ map1 map2) ;; or originally: (merge-with o+ map1 map2)
;; => {:a {:b 3}, :c ["dog" "cat"]}
Even deeper nested maps are covered
Due to the recursive character of this definition - this works also with more deeply nested maps as long as they have the same structure - and use only the defined class cases (otherwise one could add further cases):
(def mapA {:a {:b 2 :c {:d 1 :e ["a"] :f 3}} :g ["b"]})
(def mapB {:a {:b 3 :c {:d 4 :e ["b" "e"] :f 5}} :g ["c"]})
(o+ mapA mapB)
;;=> {:a {:b 5, :c {:d 5, :e ["a" "b" "e"], :f 8}}, :g ["b" "c"]}
Use reduce to add more maps at once
And as long as you can apply o+ on two objects, you can process an arbitrary number of maps using reduce:
(def mapA {:a {:b 2 :c {:d 1 :e ["a"] :f 3}} :g ["b"]})
(def mapB {:a {:b 3 :c {:d 4 :e ["b" "e"] :f 5}} :g ["c"]})
(def mapC {:a {:b 1 :c {:d 3 :e ["c"] :f 1}} :g ["a"]})
(reduce o+ [mapA mapB mapC]) ;; this vector could contain much more maps!
;; => {:a {:b 6, :c {:d 8, :e ["a" "b" "e" "c"], :f 9}}, :g ["b" "c" "a"]}
;; or we could define in addition:
(defmethod o+ :default [& args] (reduce o+ args))
;; which makes `o+` to a variadic function (function which can be
;; called with as many arguments you want)
;; then, whenever we add more than two arguments, it will be activated:
(o+ mapA mapB mapC)
;; => {:a {:b 6, :c {:d 8, :e ["a" "b" "e" "c"], :f 9}}, :g ["b" "c" "a"]}
;; and this also works:
(o+ 1 2 3 4 5 6)
;; => 21
(o+ ["a"] ["b" "c"] ["d"])
;; => ["a" "b" "c" "d"]
You can use merge-with with a custom function to resolve conflicts when merging maps.
(defn my-merge [a b]
(merge-with (fn [a b]
(if (map? a)
(merge-with + a b)
(into a b)))
a b))
Or defined recursively:
(defn my-merge [a b]
(cond (vector? a) (into a b)
(number? a) (+ a b)
(map? a) (merge-with my-merge a b)
:else (throw (ex-info "Unsupported values" {:values [a b]}))))
Here's an idea. I didn't test this, and I'm quite sure a more clever solution will come... especially now that I said that this is an answer (the someone-said-something-wrong-on-the-internet effect) :)
(reduce
(fn [accum item]
(let [b-accum (get-in accum [:a :b])
b-item (get-in item [:a :b])
new-b (+ b-accum b-item)
new-c (vec (concat (:c accum) (:c item))]
{:a {:b new-b} :c new-c}))
[map1 map2])
(require '[net.cgrand.xforms :as x])
(let [map1 {:a {:b 1} :c ["dog"]}
map2 {:a {:b 2} :c ["cat"]}]
(->> [map1 map2]
(into {}
(x/multiplex
{:a (comp (map :a) (map :b) (x/reduce +) (x/into [:b]) (x/into {}))
:c (comp (mapcat :c) (x/reduce conj))}))))
I'm wondering if anyone can help me find the right function to use with merge-with to get the desired merging of map values as a single vector.
Thanks!
; works great -single vector
(merge-with vector {:a "b"} {:a "d"} {:a "c"})
; {:a ["b" "d"]}
; uh-oh... now we are beginning to nest each set
(merge-with vector {:a "b"} {:a "d"} {:a "c"})
;{:a [["b" "d"] "c"]}
; what I want:
; {:a ["b" "d" "c"]}
though the approach with flatten solves your concrete problem, it is not universal. Based on your question i would guess that you need a map of keyword to vector as a result. And it works, when all the maps contain exactly same keys. But guess the following corner cases:
user> (merge-with (comp flatten vector) {:a "b"})
;;=> {:a "b"} oops! you following processing probably wants {:a ["b"]}
user> (merge-with (comp flatten vector) {:a "b"} {:c "d"})
;;=> {:a "b", :c "d"} once again!
user> (merge-with (comp flatten vector) {:a ["b"]} {:a ["c" ["d"]]})
;;=> {:a ("b" "c" "d")}
;; here i can see some inconsistent behavior, breaking the initial data form: would't you rather want {:a [["b"] ["c" ["d"]]]} ?
so, given that you are doing something for production, rather then learning,
i would advice the following approach: you can make the function, merging maps, but also handling the single (or first) key appearing in the result the special way:
(defn smart-merge-with [first-val-fn merge-fn & args]
(when (seq args)
(reduce (fn [acc items-map]
(reduce (fn [acc [k v]]
(if (contains? acc k)
(update acc k merge-fn v)
(assoc acc k (first-val-fn v))))
acc items-map))
{} args)))
now you can just wrap the first value into a vector, and then, when there is another value with the same key appears just add it to that vector:
user> (smart-merge-with vector conj {:a 10 :b 30} {:a 20 :c 30} {:c 1} {:d 100})
;;=> {:a [10 20], :b [30], :c [30 1], :d [100]}
user> (smart-merge-with vector conj {:a [10] :b 30} {:a 20 :c 30} {:c 1} {:d 100})
{:a [[10] 20], :b [30], :c [30 1], :d [100]}
in addition, now you can add more sophisticated logic to the maps' merging, like for example some accumulation:
user> (smart-merge-with (fn [x] {:items [x] :sum x})
(fn [x y] (-> x
(update :items conj y)
(update :sum + y)))
{:a 10 :b 20} {:b 30 :c 40} {:c 1 :d 2})
;;=> {:a {:items [10], :sum 10},
;; :b {:items [20 30], :sum 50},
;; :c {:items [40 1], :sum 41},
;; :d {:items [2], :sum 2}}
From this answer we can use the same principle:
(merge-with (comp #(into [] % ) flatten vector) {:a "b"} {:a "d"} {:a "c"})
{:a ["b" "d" "c"]}
Or roll you own function:
(merge-with #(if (vector? %1) (conj %1 %2) (vector %1 %2)) {:a "b"} {:a "d"} {:a "c"})
Let's say I have a map (m) like this:
(def m {:a 1 :b 2 :c {:d 3 :e 4} :e { ... } ....})
I'd like to create a new map only containing :a, :b and :d from m, i.e. the result should be:
{:a 1 :b 2 :d 3}
I know that I can use select-keys to easily get :a and :b:
(select-keys m [:a :b])
But what's a good way to also get :d? I'm looking for something like this:
(select-keys* m [:a :b [:c :d]])
Does such a function exists in Clojure or what's the recommended approach?
In pure Clojure I would do it like this:
(defn select-keys* [m paths]
(into {} (map (fn [p]
[(last p) (get-in m p)]))
paths))
(select-keys* m [[:a] [:b] [:c :d]]) ;;=> {:a 1, :b 2, :d 3}
I prefer keeping the type of a path regular, so a sequence of keys for all paths. In clojure.spec this would read as
(s/def ::nested-map (s/map-of keyword?
(s/or :num number? :map ::nested-map)))
(s/def ::path (s/coll-of keyword?))
(s/fdef select-keys*
:args (s/cat :m ::nested-map
:paths (s/coll-of ::path)))
As an alternative you can use destructing on a function, for example:
(def m {:a 1 :b 2 :c {:d 3 :e 4}})
(defn get-m
[{a :a b :b {d :d} :c}]
{:a 1 :b b :d d})
(get-m m) ; => {:a 1, :b 2, :d 3}
You can use clojure.walk.
(require '[clojure.walk :as w])
(defn nested-select-keys
[map keyseq]
(w/postwalk (fn [x]
(if (map? x)
(select-keys x keyseq)
(identity x))) map))
(nested-select-keys {:a 1 :b {:c 2 :d 3}} [:a :b :c])
; => {:a 1, :b {:c 2}}
I'm not aware of such a function being part of Clojure. You'll probably have to write it yourself. I've came up with this :
(defn select-keys* [m v]
(reduce
(fn [aggregate next]
(let [key-value (if (vector? next)
[(last next)
(get-in m next)]
[next
(get m next)])]
(apply assoc aggregate key-value)))
{}
v))
Require paths to be vectors so you can use peek (much faster than last). Reduce over the paths like this:
(defn select-keys* [m paths]
(reduce (fn [r p] (assoc r (peek p) (get-in m p))) {} paths))
(select-keys* m [[:a] [:b] [:c :d]]) ;;=> {:a 1, :b 2, :d 3}
Of course, this assumes that all your terminal keys are unique.
What would be a quick way to keep only certain keys from a hash-map?
(def m {:a 1 :b 2 :c 3 :d 4})
explicit version:
((fn [{:keys [b c]}] {:b b :c c})
m)
;= {:b 2, :c 3}
select-keys:
(select-keys m [:b :c])
I am struggling with the following problem...
Given a collection of maps
[
{:a 1 :b 1 :c 1 :d 1}
{:a 1 :b 2 :c 1 :d 2}
{:a 1 :b 2 :c 2 :d 3}
{:a 2 :b 1 :c 1 :d 5}
{:a 2 :b 1 :c 1 :d 6}
{:a 2 :b 1 :c 1 :d 7}
{:a 2 :b 2 :c 1 :d 7}
{:a 2 :b 3 :c 1 :d 7}
]
want to reduce/transform to...
{
1 {:b [1 2] :c [1 2] :d [1 2 3]}
2 {:b [1 2 3] :c 1 :d [5 6 7]}
}
group-by :a (primary key) and accumulate the distinct values for other keys.
I can do this in a brute force/imperative way, but struggling to figure out how to solve this in clojure way.
Thanks
Here is an admittedly inelegant, first-draft solution:
(defn reducing-fn [list-of-maps grouping-key]
(reduce (fn [m [k lst]]
(assoc m k (dissoc (reduce (fn [m1 m2]
(apply hash-map
(apply concat
(for [[k v] m2]
[k (conj (get m1 k #{}) v)]))))
{}
lst)
grouping-key)))
{}
(group-by #(grouping-key %) list-of-maps)))
user> (reducing-fn [{:a 1 :b 1 :c 1 :d 1}
{:a 1 :b 2 :c 1 :d 2}
{:a 1 :b 2 :c 2 :d 3}
{:a 2 :b 1 :c 1 :d 5}
{:a 2 :b 1 :c 1 :d 6}
{:a 2 :b 1 :c 1 :d 7}
{:a 2 :b 2 :c 1 :d 7}
{:a 2 :b 3 :c 1 :d 7}]
:a)
=> {2 {:c #{1}, :b #{1 2 3}, :d #{5 6 7}}, 1 {:c #{1 2}, :b #{1 2}, :d #{1 2 3}}}
Will try and figure out a more polished approach tomorrow, heading off to bed right now :)
(use 'clojure.set)
(def data
[
{:a 1 :b 1 :c 1 :d 1}
{:a 1 :b 2 :c 1 :d 2}
{:a 1 :b 2 :c 2 :d 3}
{:a 2 :b 1 :c 1 :d 5}
{:a 2 :b 1 :c 1 :d 6}
{:a 2 :b 1 :c 1 :d 7}
{:a 2 :b 2 :c 1 :d 7}
{:a 2 :b 3 :c 1 :d 7}
]
)
(defn key-join
"join of map by key , value is distinct."
[map-list]
(let [keys (keys (first map-list))]
(into {} (for [k keys] [k (vec (set (map #(% k) map-list)))]))))
(defn group-reduce [key map-list]
(let [sdata (set map-list)
group-value (project sdata [key])]
(into {}
(for [m group-value] [(key m) (key-join (map #(dissoc % key) (select #(= (key %) (key m)) sdata)))]))))
;;other version fast than group-reduce
(defn gr [key map-list]
(let [gdata (group-by key map-list)]
(into {} (for [[k m] gdata][k (dissoc (key-join m) key)]))))
user=> (group-reduce :a data)
{1 {:c [1 2], :b [1 2], :d [1 2 3]}, 2 {:c [1], :b [1 2 3], :d [5 6 7]}}
user=> (gr :a data)
{1 {:c [1 2], :b [1 2], :d [1 2 3]}, 2 {:c [1], :b [1 2 3], :d [5 6 7]}}
Another solution:
(defn pivot [new-key m]
(apply merge
(for [[a v] (group-by new-key m)]
{a (let [ks (set (flatten (map keys (map #(dissoc % new-key) v))))]
(zipmap ks (for [k ks] (set (map k v)))))})))
ETA: new-key would be the :a key here and m is your input map.
The first "for" destructures the group-by. That's where you're partitioning the data by the input "new-key." "for" generates a list - it's like Python's list comprehension. Here we're generating a list of maps, each with one key, whose value is a map. First we need to extract the relevant keys. These keys are held in the "ks" binding. We want to accumulate distinct values. While we could do this using reduce, since keywords are also functions, we can use them to extract across the collection and then use "set" to reduce down to distinct values. "zipmap" ties together our keys and their associated values. Then outside the main "for," we need to convert this list of maps into a single map whose keys are the distinct values of "a".
Another solution:
(defn transform
[key coll]
(letfn [(merge-maps
[coll]
(apply merge-with (fnil conj #{}) {} coll))
(process-key
[[k v]]
[k (dissoc (merge-maps v) key)])]
(->> coll
(group-by #(get % key))
(map process-key)
(into (empty coll)))))
Code untested, though.
EDIT: Of course it doesn't work, because of merge-with trying to be too clever.
(defn transform
[key coll]
(letfn [(local-merge-with
[f m & ms]
(reduce (fn [m [k v]] (update-in m [k] f v))
m
(for [m ms e m] e)))
(merge-maps
[coll]
(apply local-merge-with (fnil conj #{}) {} coll))
(process-key
[[k v]]
[k (dissoc (merge-maps v) key)])]
(->> coll
(group-by #(get % key))
(map process-key)
(into (empty coll)))))