I am exploring the exciting world of Clojure, but I am stopped on this...
I have two vectors, different in length, stored in vars.
(def lst1 ["name" "surname" "age"])
(def lst2 ["Jimi" "Hendrix" "28" "Sam" "Cooke" "33" "Buddy" "Holly" "23"])
I want to interleave them and obtain a map, with keys from first list and values from the second, like the following one:
{"name" "Jimi" , "surname" "Hendrix" , "age" "28" ,
"name" "Sam" , "surname" "Cooke" , "age" "33" ... }
even the following solution, with proper keys, would be ok:
{:name "Jimi" , :surname "Hendrix" , :age" "28" , ... }
I can use interleave-all function from Medley library and then apply the hash-map fn:
(apply hash-map
(vec (interleave-all (flatten (repeat 3 lst1)) lst2)))
=> {"age" "23", "name" "Buddy", "surname" "Holly"}
but returns just last musician. This persistent hashmap is not ordered, but is not the point.
I later tried to pair keys and values, maybe for a possible future use of assoc, who knows...
(map vector
(for [numMusicians (range 0 3) , keys (range 0 3)] (-> lst1 (nth keys) (keyword)))
(for [values (range 0 9)] (-> lst2 (nth values) (str)))
)
Returns a lazy sequence with paired vectors and proper keywords.
=> ([:name "Jimi"] [:surname "Hendrix"] [:age "28"] [:name "Sam"] ...)
Now I want to try into that should return
a new coll consisting of to-coll with ALL of the items of from-coll
conjoined.
(into {}
(map vector
(for [numMusicians (range 0 3) , keys (range 0 3)] (-> lst1 (nth keys) (keyword)))
(for [values (range 0 9)] (-> lst2 (nth values) (str)))
))
But again:
=> {:name "Buddy", :surname "Holly", :age "23"}
just the last musician, this time in a persistent array map.
I want a map with all my dead musicians. Someone knows where I am wrong?
Edit:
Thank you guys! Have managed the fn this way:
(use 'clojure.set)
(->> (partition 3 lst2) (map #(zipmap % lst1)) (map map-invert))
=> ({"name" "Jimi", "surname" "Hendrix", "age" "28"} {"name" "Sam", "surname" "Cooke", "age" "33"} {"name" "Buddy", "surname" "Holly", "age" "23"})
Each key can only exist once in a map. So your later values overwrite the earlier ones.
To get a list of maps per artiste you could do something like:
(def lst1 ["name" "surname" "age"])
(def lst2 ["Jimi" "Hendrix" "28" "Sam" "Cooke" "33" "Buddy" "Holly" "23"])
(->> (partition 3 lst2) ; Split out the seperate people
(map (fn [artist-seq] (zipmap lst1 artist-seq)))) ; Use zipmap to connect the keys and values.
This should work for any number of people as long as all the values are there, in the right order
Although you want the following form:
{"name" "Jimi" , "surname" "Hendrix" , "age" "28" ,
"name" "Sam" , "surname" "Cooke" , "age" "33" ... }
This is not allowed because keys are collided. You can't add "name" as a key several times. Key should be unique in a map.
But you can construct a list of map with the following code:
user=> (->> (map (fn [ks vs] (interleave ks vs)) (repeat 3 lst1) (partition 3 lst2))
(map #(apply hash-map %)))
({"age" "28", "name" "Jimi", "surname" "Hendrix"} {"age" "33", "name" "Sam", "surname" "Cooke"} {"age" "23", "name" "Buddy", "surname" "Holly"})
UPDATE
#status203's solution which uses zipmap looks much better.
user=> (->> (partition 3 lst2)
(map #(zipmap lst1 %)))
({"age" "28", "surname" "Hendrix", "name" "Jimi"} {"age" "33", "surname" "Cooke", "name" "Sam"} {"age" "23", "surname" "Holly", "name" "Buddy"})
Related
Getting data from the database as a list of maps (LazySeq) leaves me in need of transforming it into a map of maps.
I tried to 'assoc' and 'merge', but that didn't bring the desired result because of the nesting.
This is the form of my data:
(def data (list {:structure 1 :cat "A" :item "item1" :val 0.1}
{:structure 1 :cat "A" :item "item2" :val 0.2}
{:structure 1 :cat "B" :item "item3" :val 0.4}
{:structure 2 :cat "A" :item "item1" :val 0.3}
{:structure 2 :cat "B" :item "item3" :val 0.5}))
I would like to get it in the form
=> {1 {"A" {"item1" 0.1}
"item2" 0.2}}
{"B" {"item3" 0.4}}
2 {"A" {"item1" 0.3}}
{"B" {"item3" 0.5}}}
I tried
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply merge-with into))
This gives
{1 {"A" {"item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
By merging I lose some entries, but I can't think of any other way. Is there a simple way? I was even about to try to use specter.
Any thoughts would be appreciated.
If I'm dealing with nested maps, first stop is usually to think about update-in or assoc-in - these take a sequence of the nested keys. For a problem like this where the data is very regular, it's straightforward.
(assoc-in {} [1 "A" "item1"] 0.1)
;; =>
{1 {"A" {"item1" 0.1}}}
To consume a sequence into something else, reduce is the idiomatic choice. The reducing function is right on the edge of the complexity level I'd consider an anonymous fn for, so I'll pull it out instead for clarity.
(defn- add-val [acc line]
(assoc-in acc [(:structure line) (:cat line) (:item line)] (:val line)))
(reduce add-val {} data)
;; =>
{1 {"A" {"item1" 0.1, "item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
Which I think was the effect you were looking for.
Roads less travelled:
As your sequence is coming from a database, I wouldn't worry about using a transient collection to speed the aggregation up. Also, now I think about it, dealing with nested transient maps is a pain anyway.
update-in would be handy if you wanted to add up any values with the same key, for example, but the implication of your question is that structure/cat/item tuples are unique and so you just need the grouping.
juxt could be used to generate the key structure - i.e.
((juxt :structure :cat :item) (first data))
[1 "A" "item1"]
but it's not clear to me that there's any way to use this to make the add-val fn more readable.
You may continue to use your existing code. Only the final merge has to change:
(defn deep-merge [& xs]
(if (every? map? xs)
(apply merge-with deep-merge xs)
(apply merge xs)))
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply deep-merge))
;; =>
{1
{"A" {"item1" 0.1, "item2" 0.2},
"B" {"item3" 0.4}},
2
{"A" {"item1" 0.3},
"B" {"item3" 0.5}}}
Explanation: your original (apply merge-with into) only merge one level down. deep-merge from above will recurse into all nested maps to do the merge.
Addendum: #pete23 - one use of juxt I can think of is to make the function reusable. For example, we can extract arbitrary fields with juxt, then convert them to nested maps (with yet another function ->nested) and finally do a deep-merge:
(->> data
(map (juxt :structure :cat :item :val))
(map ->nested)
(apply deep-merge))
where ->nested can be implemented like:
(defn ->nested [[k & [v & r :as t]]]
{k (if (seq r) (->nested t) v)})
(->nested [1 "A" "item1" 0.1])
;; => {1 {"A" {"item1" 0.1}}}
One sample application (sum val by category):
(let [ks [:cat :val]]
(->> data
(map (apply juxt ks))
(map ->nested)
(apply (partial deep-merge-with +))))
;; => {"A" 0.6000000000000001, "B" 0.9}
Note deep-merge-with is left as an exercise for our readers :)
(defn map-values [f m]
(into {} (map (fn [[k v]] [k (f v)])) m))
(defn- transform-structures [ss]
(map-values (fn [cs]
(into {} (map (juxt :item :val) cs))) (group-by :cat ss)))
(defn transform [data]
(map-values transform-structures (group-by :structure data)))
then
(transform data)
May be, it is a stupid question, but it may help many of newbies. How do I add a key-value pair to the map?
I mean something like:
(defn init-item [v item]
(let [{:keys [id value]} item]
(-> v
(assoc :{ID_AS_A_KEY} value))))
And I get:
(init-item {} {:id "123456789" :value [:name "King" :surname "Leonid"]})
user=> {:123456789 [:name "King" :surname "Leonid"]}
Just don't do it. Use the string itself as your map key. There's no reason to make it a keyword. It's much easier to work with if you leave it alone.
(defn init-item [v item]
(assoc v (:id item) (:value item)))
I think this is what you meant to do:
(defn init-item
[dest-map item]
(let [item-id-str (:id item)
item-val (:value item)
item-id-kw (keyword item-id-str)]
(assoc dest-map item-id-kw item-val)))
(let [all-items {:a 1 :b 2 :c 3}
item-1 {:id "123456789"
:value [:name "King" :surname "Leonid"]}]
(init-item all-items item-1)
;=> {:a 1, :b 2, :c 3, :123456789 [:name "King" :surname "Leonid"]}
Clojure has functions name, symbol, and keyword to convert between strings and symbols/keywords. Since you already have the ID as a string, you just need to call keyword to convert it.
Be sure to always keep a browser tab open to The Clojure CheatSheet.
I've got this list of fields (that's Facebook's graph API fields list).
["a" "b" ["c" ["t"] "d"] "e" ["f"] "g"]
I want to generate a map out of it. The convention is following, if after a key vector follows, then its an inner object for the key. Example vector could be represented as a map as:
{"a" "value"
"b" {"c" {"t" "value"} "d" "value"}
"e" {"f" "value"}
"g" "value"}
So I have this solution so far
(defn traverse
[data]
(mapcat (fn [[left right]]
(if (vector? right)
(let [traversed (traverse right)]
(mapv (partial into [left]) traversed))
[[right]]))
(partition 2 1 (into [nil] data))))
(defn facebook-fields->map
[fields default-value]
(->> fields
(traverse)
(reduce #(assoc-in %1 %2 nil) {})
(clojure.walk/postwalk #(or % default-value))))
(let [data ["a" "b" ["c" ["t"] "d"] "e" ["f"] "g"]]
(facebook-fields->map data "value"))
#=> {"a" "value", "b" {"c" {"t" "value"}, "d" "value"}, "e" {"f" "value"}, "g" "value"}
But it is fat and difficult to follow. I am wondering if there is a more elegant solution.
Here's another way to do it using postwalk for the whole traversal, rather than using it only for default-value replacement:
(defn facebook-fields->map
[fields default-value]
(clojure.walk/postwalk
(fn [v] (if (coll? v)
(->> (partition-all 2 1 v)
(remove (comp coll? first))
(map (fn [[l r]] [l (if (coll? r) r default-value)]))
(into {}))
v))
fields))
(facebook-fields->map ["a" "b" ["c" ["t"] "d"] "e" ["f"] "g"] "value")
=> {"a" "value",
"b" {"c" {"t" "value"}, "d" "value"},
"e" {"f" "value"},
"g" "value"}
Trying to read heavily nested code makes my head hurt. It is worse when the answer is something of a "force-fit" with postwalk, which does things in a sort of "inside out" manner. Also, using partition-all is a bit of a waste, since we need to discard any pairs with two non-vectors.
To me, the most natural solution is a simple top-down recursion. The only problem is that we don't know in advance if we need to remove one or two items from the head of the input sequence. Thus, we can't use a simple for loop or map.
So, just write it as a straightforward recursion, and use an if to determine whether we consume 1 or 2 items from the head of the list.
If the 2nd item is a value, we consume one item and add in
:dummy-value to make a map entry.
If the 2nd item is a vector, we recurse and use that
as the value in the map entry.
The code:
(ns tst.demo.core
(:require [clojure.walk :as walk] ))
(def data ["a" "b" ["c" ["t"] "d"] "e" ["f"] "g"])
(defn parse [data]
(loop [result {}
data data]
(if (empty? data)
(walk/keywordize-keys result)
(let [a (first data)
b (second data)]
(if (sequential? b)
(recur
(into result {a (parse b)})
(drop 2 data))
(recur
(into result {a :dummy-value})
(drop 1 data)))))))
with result:
(parse data) =>
{:a :dummy-value,
:b {:c {:t :dummy-value}, :d :dummy-value},
:e {:f :dummy-value},
:g :dummy-value}
I added keywordize-keys at then end just to make the result a little more "Clojurey".
Since you're asking for a cleaner solution as opposed to a solution, and because I thought it was a neat little problem, here's another one.
(defn facebook-fields->map [coll]
(into {}
(keep (fn [[x y]]
(when-not (vector? x)
(if (vector? y)
[x (facebook-fields->map y)]
[x "value"]))))
(partition-all 2 1 coll)))
I have a file containing some text like:
1|apple|sweet
2|coffee|bitter
3|gitpush|relief
I want to work with this input using a map. In Java or Python, I would have made a nested map like:
{1: {thing: apple, taste: sweet},
2: {thing: coffee, taste: bitter},
3: {thing: gitpush, taste: relief}}
Or even a list inside the map like:
{1: [apple, sweet],
2: [coffee, bitter],
3: [grape, sour]}
The end goal is to access the last two column's data efficiently using the first column as the key.
I want to do this in Clojure and I am new to it. So far, I have succeeded in creating a list of map using the following code:
(def cust_map (map (fn [[id name taste]]
(hash-map :id (Integer/parseInt id)
:name name
:taste taste ))
(map #(str/split % #"\|") (line-seq (clojure.java.io/reader path)))))
And I get this, but it's not what I want.
({1, apple, sweet},
{2, coffee, bitter},
{3, gitpush, relief})
It would be nice if you can show me how to do the most efficient of, or both nested map and list inside map in Clojure. Thanks!
When you build a map with hash-map, the arguments are alternative keys and values. For example:
(hash-map :a 0 :b 1)
=> {:b 1, :a 0}
From what I understand, you want to have a unique key, the integer, which maps to a compound object, a map:
(hash-map 0 {:thing "apple" :taste "sweet"})
Also, you do not want to call map, which would result in a sequence of maps. You want to have a single hash-map being built.
Try using reduce:
(reduce (fn [map [id name taste]]
(merge map
(hash-map (Integer/parseInt id)
{:name name :taste taste})))
{}
'(("1" "b" "c")
("2" "d" "e")))
--- edit
Here is the full test program:
(import '(java.io BufferedReader StringReader))
(def test-input (line-seq
(BufferedReader.
(StringReader.
"1|John Smith|123 Here Street|456-4567
2|Sue Jones|43 Rose Court Street|345-7867
3|Fan Yuhong|165 Happy Lane|345-4533"))))
(def a-map
(reduce
(fn [map [id name address phone]]
(merge map
(hash-map (Integer/parseInt id)
{:name name :address address :phone phone})))
{}
(map #(clojure.string/split % #"\|") test-input)))
a-map
=> {1 {:name "John Smith", :address "123 Here Street", :phone "456-4567"}, 2 {:name "Sue Jones", :address "43 Rose Court Street", :phone "345-7867"}, 3 {:name "Fan Yuhong", :address "165 Happy Lane", :phone "345-4533"}}
I agree with #coredump that this is not concise, yet a quick solution to your code is using a list (or any other collection) and a nested map:
(def cust_map (map (fn [[id name taste]]
(list (Integer/parseInt id)
(hash-map :name name
:taste taste)))
(map #(clojure.string/split % #"\|") (line-seq (clojure.java.io/reader path)))))
This may be a somewhat naive view on my part, as I'm not all that experienced with Clojure, but any time I want to make a map from a collection I immediately think of zipmap:
(require '[clojure.java.io :as io :refer [reader]])
(defn lines-from [fname]
(line-seq (io/reader fname)))
(defn nested-map [fname re keys]
"fname : full path and filename to the input file
re : regular expression used to split file lines into columns
keys : sequence of keys for the trailing columns in each line. The first column
of each line is assumed to be the line ID"
(let [lines (lines-from fname)
line-cols (map #(clojure.string/split % re) lines) ; (["1" "apple" "sweet"] ["2" "coffee" "bitter"] ["3" "gitpush" "relief"])
ids (map #(Integer/parseInt (first %)) line-cols) ; (1 2 3)
rest-cols (map rest line-cols) ; (("apple" "sweet") ("coffee" "bitter") ("gitpush" "relief"))
rest-maps (map #(zipmap keys %) rest-cols)] ; ({:thing "apple", :taste "sweet"} {:thing "coffee", :taste "bitter"} {:thing "gitpush", :taste "relief"})
(zipmap ids rest-maps)))
(nested-map "C:/Users/whatever/q50663848.txt" #"\|" [:thing :taste])
produces
{1 {:thing "apple", :taste "sweet"}, 2 {:thing "coffee", :taste "bitter"}, 3 {:thing "gitpush", :taste "relief"}}
I've shown the intermediate results of each step in the let block as a comment so you can see what's going on. I've also tossed in lines-from, which is just my thin wrapper around line-seq to keep myself from having to type in BufferedReader. and StringReader. all the time. :-)
I have three text files as http://paste.debian.net/plain/1027720. As the third file is in the following format
Third File
salesID | custID | prodID | itemCount
1|1|1|3
2|2|2|3
I want to display the table such that custID should be replaced by the customer name and the prodID by the product description,
as follows:
1: ["John" "shoes" "3"]
What I did till now is :
(def data (slurp "cust.txt"))
(->> (for [line (clojure.string/split data #"[ ]*[\r\n]+[ ]*")]
(-> line (clojure.string/split #"\|") rest vec))
(map vector (rest (range))))
How I can retreive and map the values accordingly?
EDIT
"demo_1.txt"
content id|name|address|phone-number
1|John|123 Street|456-4567
2|Smith|123 Here Street|456-4567
"demo_2.txt"
prodID | item | Cost
1|shoes|14.96
2|milk|1.98
The processing of this data is similar to how I process CSV files. I like to split the problem into functions that do line to vector and vector to map, using the first row as the header for each.
(defn line->vec [s]
(s/split s #"\|"))
(defn vec->map [desc row]
(into {}
(map vector desc row))) ; Map accepts multiple collections
(defn file->maps [filename]
; Destructuring here, for easy capturing of header row
(let [[desc & lines] (->> (slurp filename)
(s/split-lines)
(map line->vec))
desc-keys (map keyword desc)]
(for [line lines]
(vec->map desc-keys line))))
For your demo files, you can use group-by to generate a map, sort of like an index (I manually fixed the header formatting, but you'd want to do it with a utility fn):
For (group-by :content-id (file->maps "demo_1.txt"))
{"1" [{:address "123 Street",
:phone-number "456-4567",
:name "John",
:content-id "1"}],
"2" [{:address "123 Here Street",
:phone-number "456-4567",
:name "Smith",
:content-id "2"}]}
For (group-by :prodID (file->maps "demo_2.txt"))
{"1" [{:item "shoes", :prodID "1", :cost "14.96"}],
"2" [{:item "milk", :prodID "2", :cost "1.98"}]}
And then replace each column with its index value:
(defn replace-value [index idx-key m k]
(update m k #(get-in index [% 0 idx-key])))
(defn -main [& args]
(let [customers (group-by :content-id (file->maps "demo1.txt"))
products (group-by :prodID (file->maps "demo2.txt"))]
; Use customers and products to replace some data
(->> (file->maps "demo_3.txt")
(map #(replace-value customers :name % :content-id))
(map #(replace-value products :item % :prodID)))))
And the result:
({:prodID "shoes", :content-id "John", :salesID "1", :itemCount "3"}
{:prodID "milk", :content-id "Smith", :salesID "2", :itemCount "3"})
Then it should be straightforward to convert those maps back into the format you want.