I am given a list maps:
({:a 1 :b ["red" "blue"]} {:a 2 :b ["green"]} {:a 1 :b ["yellow"]} {:a 2 :b ["orange"]})
and I need to combine them to ultimately look like this:
({:a 1 :b ["red" "blue" "yellow"]} {:a 2 :b ["green" "orange"]})
Where the maps are combined based off the value of the key "a".
So far, I have this
(->> (sort-by :a)
(partition-by :a)
(map (partial apply merge)))
But the merge will overwrite the vector in "b" with the last one giving me
({:a 1 :b ["yellow"]} {:a 2 :b ["orange"]})
Instead of merge, use merge-with, i.e.
(->> a (sort-by :a)
(partition-by :a)
(map (partial
apply
merge-with (fn [x y] (if (= x y) x (into x y))))))
Outputs
({:a 1, :b ["red" "blue" "yellow"]} {:a 2, :b ["green" "orange"]})
You don't really want to sort or partition anything: that is just CPU time spent on nothing, and at the end you have an awkward data structure (a list of maps) instead of something that would be more convenient, a map keyed by the "important" :a value.
Rather, I would write this as a reduce, where each step uses merge-with on the appropriate subsection of the eventual map:
(defn combine-by [k ms]
(reduce (fn [acc m]
(update acc (get m k)
(partial merge-with into)
(dissoc m k)))
{}, ms))
after which we have
user=> (combine-by :a '({:a 1 :b ["red" "blue"]}
{:a 2 :b ["green"]}
{:a 1 :b ["yellow"]}
{:a 2 :b ["orange"]}))
{1 {:b ["red" "blue" "yellow"]}, 2 {:b ["green" "orange"]}}
which is usefully keyed for easily looking up a map by a specific :a, or if you prefer to get the results back out as a list, you can easily unroll it.
Related
This makes sense:
user=> (into {} [[:a 1] [:b 2]])
{:a 1, :b 2}
But why does this generate an error?
user=> (into {} (partition 2 [:a 1 :b 2]))
ClassCastException clojure.lang.Keyword cannot be cast to java.util.Map$Entry clojure.lang.ATransientMap.conj (ATransientMap.java:44)
Just to be sure:
user=> (partition 2 [:a 1 :b 2])
((:a 1) (:b 2))
Does into have a problem with lazy sequences? If so, why?
Beyond an explanation of why this doesn't work, what is the recommended way to conj a sequence of key-value pairs like [:a 1 :b 2] into a map? (apply conj doesn't seem to work, either.)
You can apply the sequence to assoc:
(apply assoc {:foo 1} [:a 1 :b 2])
=> {:foo 1, :a 1, :b 2}
Does into have a problem with lazy sequences? If so, why?
No, into is commonly used with lazily evaluated sequences. This is lazy, but each key/value tuple is a vector, which is why it works when into is reducing the pairs into the map:
(into {} (map vector (range 3) (repeat :x)))
=> {0 :x, 1 :x, 2 :x}
This doesn't work because the key/value pairs are lists:
(into {} (map list (range 3) (repeat :x)))
So the difference isn't laziness; it's due to into using reduce using conj on the map, which only works with vector key/value pairs (or MapEntrys):
(conj {} [:a 1]) ;; ok
(conj {} (MapEntry. :a 1)) ;; ok
(conj {} '(:a 1)) ;; not ok
Update: assoc wrapper for applying empty/nil sequences as suggested in comments:
(defn assoc*
([m] m)
([m k v & kvs]
(apply assoc m k v kvs)))
The recommended way – (assuming the seq arg is non-empty, as pointed out by the OP) – would be
Clojure 1.9.0
user=> (apply assoc {} [:a 1 :b 2])
{:a 1, :b 2}
The version with partition doesn't work because the blocks that partition returns are seqs and those are not treated as map entries when conj'd on to a map the way vectors and actual map entries are.
E.g. (into {} (map vec) (partition 2 [:a 1 :b 2])) would work because here the pairs get converted to vectors before conjing.
Still the approach with assoc is preferable unless there's some particular circumstance that makes into convenient (like, say, if you have a bunch of transducers that you want to use for preprocessing your partition-generated pairs etc.).
Clojure treats a 2-vec such as [:a 1] as equivalent to a MapEntry, doing what amounts to "automatic type conversion". I try to avoid this and always be explicit.
(first {:a 1}) => <#clojure.lang.MapEntry [:a 1]>
(conj {:a 1} [:b 2]) => <#clojure.lang.PersistentArrayMap {:a 1, :b 2}>
So we see that a MapEntry prints like a vector but has a different type (just like a Clojure seq prints like a list but has a different type). seq converts a Clojure map into a sequence of MapEntry's, and first gets us the first one (most Clojure functions call (seq ...) on any input collections before any other processing).
Notice that conj does the inverse type conversion, treating the vector [:b 2] as if it were a MapEntry. However, conj won't perform automatic type conversion for a list or a seq:
(throws? (conj {:a 1} '(:b 2)))
(throws? (into {:a 1} '(:b 2)))
into has the same problem since it is basically just (reduce conj <1st-arg> <2nd-seq>).
The other answers already have 3 ways that work:
(assoc {} :b 2) => {:b 2}
(conj {} [:b 2]) => {:b 2}
(into {} [[:a 1] [:b 2]]) => {:a 1, :b 2}
However, I would avoid those and stick to either hash-map or sorted-map, both of which avoid the problem of empty input seqs:
(apply hash-map []) => {} ; works for empty input seq
(apply hash-map [:a 1 :b 2]) => {:b 2, :a 1}
If your input sequence is a list of pairs, flatten is sometimes helpful:
(apply sorted-map (flatten [[:a 1] [:b 2]])) => {:a 1, :b 2}
(apply hash-map (flatten '((:a 1) (:b 2)))) => {:a 1, :b 2}
P.S.
Please be note that these are not the same:
java.util.Map$Entry (listed in jdk docs as "Map.Entry")
clojure.lang.MapEntry
P.P.S
If you already have a map and want to merge in a (possibly empty) sequence of key-value pairs, just use a combination of into and hash-map:
(into {:a 1} (apply hash-map [])) => {:a 1}
(into {:a 1} (apply hash-map [:b 2])) => {:a 1, :b 2}
I have a nested map like so:
{:A {:B {:A {:B {:A {:B 0}}}}}}
I want to count the occurrences of the pair [:A :B] so that in the case above, the result is 3.
My initial idea was to use clojure.walk/postwalk to traverse the map and increment a counter.
Is there a more optimal way of achieving this?
tree-seq works out nice in this case:
(defn count-ab [data]
(->> data
(tree-seq coll? seq)
(keep #(get-in % [:A :B]))
count))
user> (def data {:A {:B {:A {:B {:A {:B 0}}}}}})
#'user/data
user> (count-ab data)
3
user> (def data-1 {:A {:B {:C {:A {:B {:A {:B 0}}}}}}})
#'user/data-1
user> (count-ab data-1)
3
user> (def data-2 {:A {:X {:A {:B 100}}
:B {:C {:A {:B {:A {:B 0}}}}}}})
#'user/data-2
user> (count-ab data-2)
4
Because it's nested map, my pragmatic idea is to traverse recursively and count:
(defn do-count [m]
(loop [m m
counter 0]
(if-let [m* (get-in m [:A :B])]
(recur m* (inc counter))
counter)))
(do-count {:A {:B {:A {:B {:A {:B 0}}}}}}) ;==> 3
I would suggest writing a function to turn the deeply nested map into a flat map keyed by paths
(defn flatten-map
"unwind deeply nested map into map of paths and vals"
([m] (into {} (flatten-map [] m)))
([path m]
(if (map? m)
(mapcat (fn [[k v]]
(flatten-map (conj path k) v))
m)
[[path m]])))
you can use this to count the adjacent [:a :b] keys like this
(->> {:A {:B {:A {:B {:A {:B 0}}}}}}
flatten-map
keys
(mapcat #(partition 2 1 %))
(filter #(= [:A :B] %))
count)
Let's say I have a map (m) like this:
(def m {:a 1 :b 2 :c {:d 3 :e 4} :e { ... } ....})
I'd like to create a new map only containing :a, :b and :d from m, i.e. the result should be:
{:a 1 :b 2 :d 3}
I know that I can use select-keys to easily get :a and :b:
(select-keys m [:a :b])
But what's a good way to also get :d? I'm looking for something like this:
(select-keys* m [:a :b [:c :d]])
Does such a function exists in Clojure or what's the recommended approach?
In pure Clojure I would do it like this:
(defn select-keys* [m paths]
(into {} (map (fn [p]
[(last p) (get-in m p)]))
paths))
(select-keys* m [[:a] [:b] [:c :d]]) ;;=> {:a 1, :b 2, :d 3}
I prefer keeping the type of a path regular, so a sequence of keys for all paths. In clojure.spec this would read as
(s/def ::nested-map (s/map-of keyword?
(s/or :num number? :map ::nested-map)))
(s/def ::path (s/coll-of keyword?))
(s/fdef select-keys*
:args (s/cat :m ::nested-map
:paths (s/coll-of ::path)))
As an alternative you can use destructing on a function, for example:
(def m {:a 1 :b 2 :c {:d 3 :e 4}})
(defn get-m
[{a :a b :b {d :d} :c}]
{:a 1 :b b :d d})
(get-m m) ; => {:a 1, :b 2, :d 3}
You can use clojure.walk.
(require '[clojure.walk :as w])
(defn nested-select-keys
[map keyseq]
(w/postwalk (fn [x]
(if (map? x)
(select-keys x keyseq)
(identity x))) map))
(nested-select-keys {:a 1 :b {:c 2 :d 3}} [:a :b :c])
; => {:a 1, :b {:c 2}}
I'm not aware of such a function being part of Clojure. You'll probably have to write it yourself. I've came up with this :
(defn select-keys* [m v]
(reduce
(fn [aggregate next]
(let [key-value (if (vector? next)
[(last next)
(get-in m next)]
[next
(get m next)])]
(apply assoc aggregate key-value)))
{}
v))
Require paths to be vectors so you can use peek (much faster than last). Reduce over the paths like this:
(defn select-keys* [m paths]
(reduce (fn [r p] (assoc r (peek p) (get-in m p))) {} paths))
(select-keys* m [[:a] [:b] [:c :d]]) ;;=> {:a 1, :b 2, :d 3}
Of course, this assumes that all your terminal keys are unique.
I am wrapping my head around state in Clojure. I come from languages where state can be mutated. For example, in Python, I can create a dictionary, put some string => integer pairs inside, and then walk over the dictionary and increment the values.
How would I do this in idiomatic Clojure?
(def my-map {:a 1 :b 2})
(zipmap (keys my-map) (map inc (vals my-map)))
;;=> {:b 3, :a 2}
To update only one value by key:
(update-in my-map [:b] inc) ;;=> {:a 1, :b 3}
Since Clojure 1.7 it's also possible to use update:
(update my-map :b inc)
Just produce a new map and use it:
(def m {:a 3 :b 4})
(apply merge
(map (fn [[k v]] {k (inc v) }) m))
; {:b 5, :a 4}
To update multiple values, you could also take advantage of reduce taking an already filled accumulator, and applying a function on that and every member of the following collection.
=> (reduce (fn [a k] (update-in a k inc)) {:a 1 :b 2 :c 3 :d 4} [[:a] [:c]])
{:a 2, :c 4, :b 2, :d 4}
Be aware of the keys needing to be enclosed in vectors, but you can still do multiple update-ins in nested structures like the original update in.
If you made it a generalized function, you could automatically wrap a vector over a key by testing it with coll?:
(defn multi-update-in
[m v f & args]
(reduce
(fn [acc p] (apply
(partial update-in acc (if (coll? p) p (vector p)) f)
args)) m v))
which would allow for single-level/key updates without the need for wrapping the keys in vectors
=> (multi-update-in {:a 1 :b 2 :c 3 :d 4} [:a :c] inc)
{:a 2, :c 4, :b 2, :d 4}
but still be able to do nested updates
(def people
{"keith" {:age 27 :hobby "needlefelting"}
"penelope" {:age 39 :hobby "thaiboxing"}
"brian" {:age 12 :hobby "rocket science"}})
=> (multi-update-in people [["keith" :age] ["brian" :age]] inc)
{"keith" {:age 28, :hobby "needlefelting"},
"penelope" {:age 39, :hobby "thaiboxing"},
"brian" {:age 13, :hobby "rocket science"}}
To slightly improve #Michiel Brokent's answer. This will work if the key already doesn't present.
(update my-map :a #(if (nil? %) 1 (inc %)))
I've been toying with the same idea, so I came up with:
(defn remap
"returns a function which takes a map as argument
and applies f to each value in the map"
[f]
#(into {} (map (fn [[k v]] [k (f v)]) %)))
((remap inc) {:foo 1})
;=> {:foo 2}
or
(def inc-vals (remap inc))
(inc-vals {:foo 1})
;=> {:foo 2}
I am struggling with the following problem...
Given a collection of maps
[
{:a 1 :b 1 :c 1 :d 1}
{:a 1 :b 2 :c 1 :d 2}
{:a 1 :b 2 :c 2 :d 3}
{:a 2 :b 1 :c 1 :d 5}
{:a 2 :b 1 :c 1 :d 6}
{:a 2 :b 1 :c 1 :d 7}
{:a 2 :b 2 :c 1 :d 7}
{:a 2 :b 3 :c 1 :d 7}
]
want to reduce/transform to...
{
1 {:b [1 2] :c [1 2] :d [1 2 3]}
2 {:b [1 2 3] :c 1 :d [5 6 7]}
}
group-by :a (primary key) and accumulate the distinct values for other keys.
I can do this in a brute force/imperative way, but struggling to figure out how to solve this in clojure way.
Thanks
Here is an admittedly inelegant, first-draft solution:
(defn reducing-fn [list-of-maps grouping-key]
(reduce (fn [m [k lst]]
(assoc m k (dissoc (reduce (fn [m1 m2]
(apply hash-map
(apply concat
(for [[k v] m2]
[k (conj (get m1 k #{}) v)]))))
{}
lst)
grouping-key)))
{}
(group-by #(grouping-key %) list-of-maps)))
user> (reducing-fn [{:a 1 :b 1 :c 1 :d 1}
{:a 1 :b 2 :c 1 :d 2}
{:a 1 :b 2 :c 2 :d 3}
{:a 2 :b 1 :c 1 :d 5}
{:a 2 :b 1 :c 1 :d 6}
{:a 2 :b 1 :c 1 :d 7}
{:a 2 :b 2 :c 1 :d 7}
{:a 2 :b 3 :c 1 :d 7}]
:a)
=> {2 {:c #{1}, :b #{1 2 3}, :d #{5 6 7}}, 1 {:c #{1 2}, :b #{1 2}, :d #{1 2 3}}}
Will try and figure out a more polished approach tomorrow, heading off to bed right now :)
(use 'clojure.set)
(def data
[
{:a 1 :b 1 :c 1 :d 1}
{:a 1 :b 2 :c 1 :d 2}
{:a 1 :b 2 :c 2 :d 3}
{:a 2 :b 1 :c 1 :d 5}
{:a 2 :b 1 :c 1 :d 6}
{:a 2 :b 1 :c 1 :d 7}
{:a 2 :b 2 :c 1 :d 7}
{:a 2 :b 3 :c 1 :d 7}
]
)
(defn key-join
"join of map by key , value is distinct."
[map-list]
(let [keys (keys (first map-list))]
(into {} (for [k keys] [k (vec (set (map #(% k) map-list)))]))))
(defn group-reduce [key map-list]
(let [sdata (set map-list)
group-value (project sdata [key])]
(into {}
(for [m group-value] [(key m) (key-join (map #(dissoc % key) (select #(= (key %) (key m)) sdata)))]))))
;;other version fast than group-reduce
(defn gr [key map-list]
(let [gdata (group-by key map-list)]
(into {} (for [[k m] gdata][k (dissoc (key-join m) key)]))))
user=> (group-reduce :a data)
{1 {:c [1 2], :b [1 2], :d [1 2 3]}, 2 {:c [1], :b [1 2 3], :d [5 6 7]}}
user=> (gr :a data)
{1 {:c [1 2], :b [1 2], :d [1 2 3]}, 2 {:c [1], :b [1 2 3], :d [5 6 7]}}
Another solution:
(defn pivot [new-key m]
(apply merge
(for [[a v] (group-by new-key m)]
{a (let [ks (set (flatten (map keys (map #(dissoc % new-key) v))))]
(zipmap ks (for [k ks] (set (map k v)))))})))
ETA: new-key would be the :a key here and m is your input map.
The first "for" destructures the group-by. That's where you're partitioning the data by the input "new-key." "for" generates a list - it's like Python's list comprehension. Here we're generating a list of maps, each with one key, whose value is a map. First we need to extract the relevant keys. These keys are held in the "ks" binding. We want to accumulate distinct values. While we could do this using reduce, since keywords are also functions, we can use them to extract across the collection and then use "set" to reduce down to distinct values. "zipmap" ties together our keys and their associated values. Then outside the main "for," we need to convert this list of maps into a single map whose keys are the distinct values of "a".
Another solution:
(defn transform
[key coll]
(letfn [(merge-maps
[coll]
(apply merge-with (fnil conj #{}) {} coll))
(process-key
[[k v]]
[k (dissoc (merge-maps v) key)])]
(->> coll
(group-by #(get % key))
(map process-key)
(into (empty coll)))))
Code untested, though.
EDIT: Of course it doesn't work, because of merge-with trying to be too clever.
(defn transform
[key coll]
(letfn [(local-merge-with
[f m & ms]
(reduce (fn [m [k v]] (update-in m [k] f v))
m
(for [m ms e m] e)))
(merge-maps
[coll]
(apply local-merge-with (fnil conj #{}) {} coll))
(process-key
[[k v]]
[k (dissoc (merge-maps v) key)])]
(->> coll
(group-by #(get % key))
(map process-key)
(into (empty coll)))))