How to convert in Clojure a sequence with maps structure (:data), like this:
{:data ({2019-09-20 3.673202} {2018-01-01 4.673202} {2018-01-02 5.673202}) }
into a map of key-value pairs, like this one:
{:data {2019-09-20 3.673202, 2018-01-01 4.673202, 2018-01-02 5.673202}}
Another option is into:
(let [data [ map1 map2 map3 ...]
(into {} data))
(let [data '({"2019-09-20" 3.673202} {"2018-01-01" 4.673202} {"2018-01-02" 5.673202})]
(apply merge data))
=> {"2019-09-20" 3.673202, "2018-01-01" 4.673202, "2018-01-02" 5.673202}
Related
Getting data from the database as a list of maps (LazySeq) leaves me in need of transforming it into a map of maps.
I tried to 'assoc' and 'merge', but that didn't bring the desired result because of the nesting.
This is the form of my data:
(def data (list {:structure 1 :cat "A" :item "item1" :val 0.1}
{:structure 1 :cat "A" :item "item2" :val 0.2}
{:structure 1 :cat "B" :item "item3" :val 0.4}
{:structure 2 :cat "A" :item "item1" :val 0.3}
{:structure 2 :cat "B" :item "item3" :val 0.5}))
I would like to get it in the form
=> {1 {"A" {"item1" 0.1}
"item2" 0.2}}
{"B" {"item3" 0.4}}
2 {"A" {"item1" 0.3}}
{"B" {"item3" 0.5}}}
I tried
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply merge-with into))
This gives
{1 {"A" {"item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
By merging I lose some entries, but I can't think of any other way. Is there a simple way? I was even about to try to use specter.
Any thoughts would be appreciated.
If I'm dealing with nested maps, first stop is usually to think about update-in or assoc-in - these take a sequence of the nested keys. For a problem like this where the data is very regular, it's straightforward.
(assoc-in {} [1 "A" "item1"] 0.1)
;; =>
{1 {"A" {"item1" 0.1}}}
To consume a sequence into something else, reduce is the idiomatic choice. The reducing function is right on the edge of the complexity level I'd consider an anonymous fn for, so I'll pull it out instead for clarity.
(defn- add-val [acc line]
(assoc-in acc [(:structure line) (:cat line) (:item line)] (:val line)))
(reduce add-val {} data)
;; =>
{1 {"A" {"item1" 0.1, "item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
Which I think was the effect you were looking for.
Roads less travelled:
As your sequence is coming from a database, I wouldn't worry about using a transient collection to speed the aggregation up. Also, now I think about it, dealing with nested transient maps is a pain anyway.
update-in would be handy if you wanted to add up any values with the same key, for example, but the implication of your question is that structure/cat/item tuples are unique and so you just need the grouping.
juxt could be used to generate the key structure - i.e.
((juxt :structure :cat :item) (first data))
[1 "A" "item1"]
but it's not clear to me that there's any way to use this to make the add-val fn more readable.
You may continue to use your existing code. Only the final merge has to change:
(defn deep-merge [& xs]
(if (every? map? xs)
(apply merge-with deep-merge xs)
(apply merge xs)))
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply deep-merge))
;; =>
{1
{"A" {"item1" 0.1, "item2" 0.2},
"B" {"item3" 0.4}},
2
{"A" {"item1" 0.3},
"B" {"item3" 0.5}}}
Explanation: your original (apply merge-with into) only merge one level down. deep-merge from above will recurse into all nested maps to do the merge.
Addendum: #pete23 - one use of juxt I can think of is to make the function reusable. For example, we can extract arbitrary fields with juxt, then convert them to nested maps (with yet another function ->nested) and finally do a deep-merge:
(->> data
(map (juxt :structure :cat :item :val))
(map ->nested)
(apply deep-merge))
where ->nested can be implemented like:
(defn ->nested [[k & [v & r :as t]]]
{k (if (seq r) (->nested t) v)})
(->nested [1 "A" "item1" 0.1])
;; => {1 {"A" {"item1" 0.1}}}
One sample application (sum val by category):
(let [ks [:cat :val]]
(->> data
(map (apply juxt ks))
(map ->nested)
(apply (partial deep-merge-with +))))
;; => {"A" 0.6000000000000001, "B" 0.9}
Note deep-merge-with is left as an exercise for our readers :)
(defn map-values [f m]
(into {} (map (fn [[k v]] [k (f v)])) m))
(defn- transform-structures [ss]
(map-values (fn [cs]
(into {} (map (juxt :item :val) cs))) (group-by :cat ss)))
(defn transform [data]
(map-values transform-structures (group-by :structure data)))
then
(transform data)
I am totally new to clojure.
I have a JSON like: { "1": true, "2": false, "3": true, "4": false }
I want to create an array of keys for which the value is true in clojure. In this example the array should be ["1", "3"].
Please help me. Any help would be appreciated.
there are also couple of short and simple snippets for that:
user> (filter m (keys m))
;;=> ("1" "3")
user> (keep (fn [[k v]] (when v k)) m)
;;=> ("1" "3")
user> (for [[k v] m :when v] k)
;;=> ("1" "3")
If you're fine with using a vector instead of an array (since you're usually using vectors in Clojure anyway), you can do something like.
(defn keys-for-truthy-vals [m]
(->> m (filter val) (mapv key)))
Note The mapv is only so the map call returns a vector. If you want a seq, just use map.
The same as already provided, just staying in maps.
(keys (filter val m))
If your map is a Something like (->> (filter (fn [[k v]] v) a) (map (fn [[k v]] k))) will work. You can't do it with just a map because you need to drop certain values, so there will need to be some reducing or filtering.
There is built-in function in the Tupelo library for this:
(submap-by-vals map-arg keep-vals & opts)
Returns a new map containing entries with the specified vals. Throws for missing vals,
unless `:missing-ok` is specified. Usage:
(submap-by-vals {:a 1 :b 2 :A 1} #{1 } ) => {:a 1 :A 1}
(submap-by-vals {:a 1 :b 2 :A 1} #{1 9} :missing-ok ) => {:a 1 :A 1}
You could then just use the keys function on the resulting map.
Maybe this?
(->> foo (filter second) keys)
where foo is a map.
What is the idiomatic way of counting certain properties of a nested map of maps in Clojure?
Given the following datastructure:
(def x {
:0 {:attrs {:attributes {:dontcare "something"
:1 {:attrs {:abc "some value"}}}}}
:1 {:attrs {:attributes {:dontcare "something"
:1 {:attrs {:abc "some value"}}}}}
:9 {:attrs {:attributes {:dontcare "something"
:5 {:attrs {:xyz "some value"}}}}}})
How can i produce the desired output:
(= (count-attributes x) {:abc 2, :xyz 1})
This is my best effort so far:
(defn count-attributes
[input]
(let [result (for [[_ {{attributes :attributes} :attrs}] x
:let [v (into {} (remove (comp not :attrs) (vals attributes)))]]
(:attrs v))]
(frequencies result)))
Which produces the following:
{{:abc "some value"} 2, {:xyz "some value"} 1}
I like building such functions with threadding so the steps are easier to read
user> (->> x
vals ; first throw out the keys
(map #(get-in % [:attrs :attributes])) ; get the nested maps
(map vals) ; again throw out the keys
(map #(filter map? %)) ; throw out the "something" ones.
flatten ; we no longer need the sequence sequences
(map vals) ; and again we don't care about the keys
flatten ; the map put them back into a list of lists
frequencies) ; and then count them.
{{:abc "some value"} 2, {:xyz "some value"} 1}
(remove (comp not :attrs) is a lot like select-keys
for [[_ {{attributes :attributes} :attrs}] reminds me of get-in
I find tree-seq very useful for these cases:
(frequencies (filter #(and (map? %) (not-any? map? (vals %))) (tree-seq map? vals x)))
How can I convert the values of 'mymap' to a list of Doubles instead of a list of Strings, at the same time as mymap is created?
(use '[clojure.string :only (join split)])
;(def raw-data (slurp "http://ichart.finance.yahoo.com/table.csv?s=INTC"))
;Downloaded and removed the first line
(def raw-data (slurp "table-INTC.csv"))
(def raw-vector-list
(map
#(split % #",") ; anonymous map function to split by comma
(split raw-data #"\n"))) ; split raw data by new line
(pr (take 1 raw-vector-list))
(def mymap
(zipmap
;construct composite key out of symbol and date which is head of the list
(map #(str "INTC-" %) (map first raw-vector-list))
;How do i convert these values to Double instead of Strings?
(map rest raw-vector-list)))
(pr (take 1 mymap))
(def mymap
(zipmap
(map #(str "NAT-" %) (map first raw-vector-list))
(map #(map (fn [v] (Double/parseDouble v)) %)
(map rest raw-vector-list))))
(pprint (take 1 mymap))
-> (["NAT-1991-09-30" (41.75 42.25 41.25 42.25 3.62112E7 1.03)])
Another version
(def mymap
(map (fn [[date & values]]
[(str "NAT-" date)
(map #(Double/parseDouble %) values)])
;; Drop first non-parsable element in raw-vector-list
;; ["Date" "Open" "High" "Low" "Close" "Volume" "Adj Close"]
(drop 1 raw-vector-list)))
So for the tail/rest portion of this data. You are mapping an anonymous, map function, to a list of strings, and then mapping the type conversion to the elements in each sublist.
(def mymap
(zipmap
(map #(str "NAT-" %) (map first raw-vector-list))
(map #(map (fn [v] (Double/parseDouble v)) %)
(map rest raw-vector-list))))
How can I pull out the type conversion into a function like below...And then utilize my custom method?
(defn str-to-dbl [n] (Double/parseDouble n))
This code complains about nested #'s.
(def mymap
(zipmap
(map #(str "NAT-" %) (map first raw-vector-list))
(map #(map #(str-to-double %)
(map rest raw-vector-list))))
When we call keys on a map, is the order of keys in the resulting seq guaranteed to be the same as the order of values when we call vals on the same map?
In other words, is it ok to map a function f over the contents of a map like this:
(map #(f %1 %2) (keys m) (vals m))
If not, is there an equivalent to perl's each in clojure? Or perhaps the inverse function of zipmap?
You can iterate over the map, you get key val pairs,
(map (fn [[key val]]
(println key val)) {:a :b :c :d})
pretty much all clojure data structures are seqable.