I want to combine elements from two lists, my program looks like this
(ns datamodel
(:use
[net.cgrand.enlive-html :as en-html ])
(:require
[clojure.zip :as z]
[clojure.data.zip.xml :only (attr text xml->) :as xz]
[clojure.xml :as xml ]
[clojure.data.zip.xml :as zf]
[clojure.java.io :as io]
))
(def data-url "http://api.eventful.com/rest/events/search?app_key=4H4Vff4PdrTGp3vV&keywords=music&location=Belgrade&date=Future")
(defn map-tags-contents [url & tags]
(map #(hash-map % (keyword (last tags)))
(mapcat (comp :content z/node)
(apply xz/xml->
(-> url xml/parse z/xml-zip)
(for [t tags]
(zf/tag= t)
)))))
(def titles (map-tags-contents data-url :events :event :title))
(def descriptions (map-tags-contents data-url :events :event :description))
(defn create-map [](for [el1 titles
el2 descriptions] (into {} (conj el1 el2 ))))
But when I call create-map resulting maps in list are duplicated. I see that I got Cartesian join, because I didn't say the way elements will be combined. And I want first element from first map and first from second map to be combined, second element from first map and second from second map, etc...
Element-wise combination
(map list [1 2 3] [:a :b :c]) ;=> ((1 :a) (2 :b) (3 :c))
Cartesian product
(for [x [1 2 3], y [:a :b :c]] (list x y))
;=> ((1 :a) (1 :b) (1 :c) (2 :a) (2 :b) (2 :c) (3 :a) (3 :b) (3 :c))
So the fn should look like this
(defn create-map [](map conj titles descriptions ))
Thank to #A. Webb
Related
There is a map containing sequences. The sequences contain items.
I want to remove a given item from any sequence that contains it.
The solution I found does what it should, but I wonder if there is a better
or more elegant way to achieve the same.
my current solution:
(defn remove-item-from-map-value [my-map item]
(apply merge (for [[k v] my-map] {k (remove #(= item %) v)})))
The test describe the expected behaviour:
(require '[clojure.test :as t])
(def my-map {:keyOne ["itemOne"]
:keyTwo ["itemTwo" "itemThree"]
:keyThree ["itemFour" "itemFive" "itemSix"]})
(defn remove-item-from-map-value [my-map item]
(apply merge (for [[k v] my-map] {k (remove #(= item %) v)})))
(t/is (= (remove-item-from-map-value my-map "unknown-item") my-map))
(t/is (= (remove-item-from-map-value my-map "itemFive") {:keyOne ["itemOne"]
:keyTwo ["itemTwo" "itemThree"]
:keyThree ["itemFour" "itemSix"]}))
(t/is (= (remove-item-from-map-value my-map "itemThree") {:keyOne ["itemOne"]
:keyTwo ["itemTwo"]
:keyThree ["itemFour" "itemFive" "itemSix"]}))
(t/is (= (remove-item-from-map-value my-map "itemOne") {:keyOne []
:keyTwo ["itemTwo" "itemThree"]
:keyThree ["itemFour" "itemFive" "itemSix"]}))
I'm fairly new to clojure and am interested in different solutions.
So any input is welcome.
I throw in the specter
version for good measure. It keeps the vectors inside the map
and it's really compact.
(setval [MAP-VALS ALL #{"itemFive"}] NONE my-map)
Example
user=> (use 'com.rpl.specter)
nil
user=> (def my-map {:keyOne ["itemOne"]
#_=> :keyTwo ["itemTwo" "itemThree"]
#_=> :keyThree ["itemFour" "itemFive" "itemSix"]})
#_=>
#'user/my-map
user=> (setval [MAP-VALS ALL #{"itemFive"}] NONE my-map)
{:keyOne ["itemOne"],
:keyThree ["itemFour" "itemSix"],
:keyTwo ["itemTwo" "itemThree"]}
user=> (setval [MAP-VALS ALL #{"unknown"}] NONE my-map)
{:keyOne ["itemOne"],
:keyThree ["itemFour" "itemFive" "itemSix"],
:keyTwo ["itemTwo" "itemThree"]}
i would go with something like this:
user> (defn remove-item [my-map item]
(into {}
(map (fn [[k v]] [k (remove #{item} v)]))
my-map))
#'user/remove-item
user> (remove-item my-map "itemFour")
;;=> {:keyOne ("itemOne"),
;; :keyTwo ("itemTwo" "itemThree"),
;; :keyThree ("itemFive" "itemSix")}
you could also make up a handy function map-val performing mapping on map values:
(defn map-val [f data]
(reduce-kv
(fn [acc k v] (assoc acc k (f v)))
{} data))
or shortly like this:
(defn map-val [f data]
(reduce #(update % %2 f) data (keys data)))
user> (map-val inc {:a 1 :b 2})
;;=> {:a 2, :b 3}
(defn remove-item [my-map item]
(map-val (partial remove #{item}) my-map))
user> (remove-item my-map "itemFour")
;;=> {:keyOne ("itemOne"),
;; :keyTwo ("itemTwo" "itemThree"),
;; :keyThree ("itemFive" "itemSix")}
I think your solution is mostly okay, but I would try to avoid the apply merge part, as you can easily recreate a map from a sequence with into. Also, you could also use map instead of for which I think is a little bit more idiomatic in this case as you don't use any of the list comprehension features of for.
(defn remove-item-from-map-value [m item]
(->> m
(map (fn [[k vs]]
{k (remove #(= item %) vs)}))
(into {})))
Another solution much like #leetwinski:
(defn remove-item [m i]
(zipmap (keys m)
(map (fn [v] (remove #(= % i) v))
(vals m))))
Here's a one-liner which does this in an elegant way. The perfect function for me to use in this scenario is clojure.walk/prewalk. What this fn does is it traverse all of the sub-forms of the form that you pass to it and it transforms them with the provided fn:
(defn remove-item-from-map-value [data item]
(clojure.walk/prewalk #(if (map-entry? %) [(first %) (remove #{item} (second %))] %) data))
What the remove-item-from-map-value fn will do is it will check if current form is a map entry and if so, it will remove specified key from its value (second element of the map entry, which is a vector containing a key and a value, respectively).
The best this about this approach is that is is completely extendable: you could decide to do different things for different types of forms, you can also handle nested forms, etc.
It took me some time to master this fn but once I got it I found it extremely useful!
I am fairly new to Clojure and would help help with some code. I have a function which takes a vector and i would like to loop through the vector and get the value at an index 'i' and the value of 'i' itself. 'i' is the value which is incremented in the loop.
I have checked 'for' at the clojure docs at for and wrote the following code.
(for [i some-vector]
(print (get-intersec i (.length some-vector) loop-count)))
The loop-count variable is supposed to be the loop count.
I have also checked loop but it does not seem like a feasible solution. Can someone help me with a clojure function i can use or help me write a macro or function that can do that.
Thank you.
Ps: To solve my problem, i use my own counter but would like a better solution.
First, keep in mind that for is for list comprehension, that is, creating new sequences. For looping through a sequence for some side effect, like printing, you probably want to use doseq.
To include a numeric count with each element as you loop through, you can use map-indexed:
(def xs [:a :b :c :d])
(doseq [[n elem] (map-indexed #(vector %1 %2) xs)]
(println n "->" elem))
Output:
0 -> :a
1 -> :b
2 -> :c
3 -> :d
If you find yourself doing this a lot, like I did, you can create a macro:
(defmacro doseq-indexed [[[item idx] coll] & forms]
`(doseq [[~idx ~item] (map-indexed #(vector %1 %2) ~coll)]
~#forms))
And use it like this:
> (doseq-indexed [[n elem] xs] (println n "->" elem))
0 -> :a
1 -> :b
2 -> :c
3 -> :d
Don't forget dotimes for simple stuff like this:
(let [data [:a :b :c :d]]
(dotimes [i (count data)]
(println i " -> " (data i))
; or (nth data i)
; or (get data i)
))
with result
0 -> :a
1 -> :b
2 -> :c
3 -> :d
Using loop/recur would look like this:
(let [data [:a :b :c :d]]
(loop [i 0
items data]
(let [curr (first items)]
(when curr
(println i "->" curr)
(recur (inc i) (rest items))))))
Update:
If you need this a lot, I already wrote a function that will add the index value to the beginning of each entry in a sequence:
(ns tst.demo.core
(:use tupelo.test)
(:require [tupelo.core :as t]) )
(dotest
(let [data [:a :b :c :d]]
(t/spy-pretty :indexed-data
(t/indexed data))))
with result
:indexed-data =>
([0 :a]
[1 :b]
[2 :c]
[3 :d])
The general signature is:
(indexed & colls)
Given one or more collections, returns a sequence of indexed tuples
from the collections like:
(indexed xs ys zs) -> [ [0 x0 y0 z0]
[1 x1 y1 z1]
[2 x2 y2 z2]
... ]
If your not set on using for, you could use map-indexed e.g.
(map-indexed (fn [i v]
(get-intersect v (.length some-vector) i))
some-vector))
I don't know what get-intersect is and assume .length is java interop? Anyway, map-indexed expects a function of 2 arguments, the 1st is the index and the second is the value.
Getting data from the database as a list of maps (LazySeq) leaves me in need of transforming it into a map of maps.
I tried to 'assoc' and 'merge', but that didn't bring the desired result because of the nesting.
This is the form of my data:
(def data (list {:structure 1 :cat "A" :item "item1" :val 0.1}
{:structure 1 :cat "A" :item "item2" :val 0.2}
{:structure 1 :cat "B" :item "item3" :val 0.4}
{:structure 2 :cat "A" :item "item1" :val 0.3}
{:structure 2 :cat "B" :item "item3" :val 0.5}))
I would like to get it in the form
=> {1 {"A" {"item1" 0.1}
"item2" 0.2}}
{"B" {"item3" 0.4}}
2 {"A" {"item1" 0.3}}
{"B" {"item3" 0.5}}}
I tried
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply merge-with into))
This gives
{1 {"A" {"item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
By merging I lose some entries, but I can't think of any other way. Is there a simple way? I was even about to try to use specter.
Any thoughts would be appreciated.
If I'm dealing with nested maps, first stop is usually to think about update-in or assoc-in - these take a sequence of the nested keys. For a problem like this where the data is very regular, it's straightforward.
(assoc-in {} [1 "A" "item1"] 0.1)
;; =>
{1 {"A" {"item1" 0.1}}}
To consume a sequence into something else, reduce is the idiomatic choice. The reducing function is right on the edge of the complexity level I'd consider an anonymous fn for, so I'll pull it out instead for clarity.
(defn- add-val [acc line]
(assoc-in acc [(:structure line) (:cat line) (:item line)] (:val line)))
(reduce add-val {} data)
;; =>
{1 {"A" {"item1" 0.1, "item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
Which I think was the effect you were looking for.
Roads less travelled:
As your sequence is coming from a database, I wouldn't worry about using a transient collection to speed the aggregation up. Also, now I think about it, dealing with nested transient maps is a pain anyway.
update-in would be handy if you wanted to add up any values with the same key, for example, but the implication of your question is that structure/cat/item tuples are unique and so you just need the grouping.
juxt could be used to generate the key structure - i.e.
((juxt :structure :cat :item) (first data))
[1 "A" "item1"]
but it's not clear to me that there's any way to use this to make the add-val fn more readable.
You may continue to use your existing code. Only the final merge has to change:
(defn deep-merge [& xs]
(if (every? map? xs)
(apply merge-with deep-merge xs)
(apply merge xs)))
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply deep-merge))
;; =>
{1
{"A" {"item1" 0.1, "item2" 0.2},
"B" {"item3" 0.4}},
2
{"A" {"item1" 0.3},
"B" {"item3" 0.5}}}
Explanation: your original (apply merge-with into) only merge one level down. deep-merge from above will recurse into all nested maps to do the merge.
Addendum: #pete23 - one use of juxt I can think of is to make the function reusable. For example, we can extract arbitrary fields with juxt, then convert them to nested maps (with yet another function ->nested) and finally do a deep-merge:
(->> data
(map (juxt :structure :cat :item :val))
(map ->nested)
(apply deep-merge))
where ->nested can be implemented like:
(defn ->nested [[k & [v & r :as t]]]
{k (if (seq r) (->nested t) v)})
(->nested [1 "A" "item1" 0.1])
;; => {1 {"A" {"item1" 0.1}}}
One sample application (sum val by category):
(let [ks [:cat :val]]
(->> data
(map (apply juxt ks))
(map ->nested)
(apply (partial deep-merge-with +))))
;; => {"A" 0.6000000000000001, "B" 0.9}
Note deep-merge-with is left as an exercise for our readers :)
(defn map-values [f m]
(into {} (map (fn [[k v]] [k (f v)])) m))
(defn- transform-structures [ss]
(map-values (fn [cs]
(into {} (map (juxt :item :val) cs))) (group-by :cat ss)))
(defn transform [data]
(map-values transform-structures (group-by :structure data)))
then
(transform data)
Is there a convenient way in ClojureScript to pretty print a nested hash-map in the way that the whole tree-structure becomes immediately visible.
For instance a map like this
(def my-map {:a {:b 1 :c 9} :b {:d 8 :e {:f 2 :g 3 :h 4}} :c 10})
should be printed like this:
{:a {:b 1
:c 9}
:b {:d 8
:e {:f 2
:g 3
:h 4}}
:c 10}
EDIT: There might also be vectors in the map. The usecase is just to inspect larger data structures during development.
There is no built-in way to do it. You might come close to what you want by using cljs.pprint and setting cljs.pprint/*print-right-margin* to a low value.
I would recommend to take a look at a small library shodan which provides a very useful inspect function:
(require '[shodan.inspection :refer [inspect]])
(inspect {:aaaaaa 1
:bbbbbb {:ccc 2
:dddddd [1 2 3 4 5]}})
It won't print anything in your CLJS REPL but will provide a handy view in your browser's console:
You can collapse and expand nested datastructures - it basically does what you asked for.
As a personal challenge I wrote the following code:
(enable-console-print!)
(def atomic? (complement coll?))
(def padding #(apply str (repeat % " ")))
(def tabulate #(apply str (repeat % "\t")))
(def strcat #(->> (apply concat %&) (apply str)))
(defn my-max-key [x] (if (empty? x) [""] (apply (partial max-key count) x)))
(defn longest-key [m] (->> m keys (filter atomic?) (map str) my-max-key))
(def length (comp count str))
(def not-map? (complement map?))
(def nested? #(some coll? %))
(def join #(apply str (interpose % %2)))
(def join-lines (partial join "\n"))
(defn has-atomic? [coll] (some atomic? coll))
(defn diff-key-lengths [key1 key2] (- (length key1) (length key2)))
(defn convert
([thing] (convert -1 thing))
([depth thing]
(defn convert-items []
(defn convert-seq []
(conj []
(map (partial convert (inc depth)) thing)
""))
(defn string-horizontally [[key value]]
(str (tabulate (inc depth))
key
(padding (diff-key-lengths (longest-key thing) key))
" → "
value))
(defn string-vertically [[key value]]
(str (convert (inc depth) key) "\n"
(convert (+ 2 depth) "↓") "\n"
(convert (inc depth) value) "\n"))
(defn convert-kv [[key value]]
(if (nested? [key value])
(string-vertically [key value])
(string-horizontally [key value])))
(cond (atomic? thing)
[(str (tabulate depth) thing)]
(not-map? thing)
(convert-seq)
(map? thing)
(map convert-kv thing)))
(->> (convert-items) flatten join-lines)))
(def sample-input [["the first thing in this nested vector"]
{{"this is a key in a nested map"
"that points to me!!!"}
{"and that entire map points to this map!!!"
"cool!!!"
"but it gets cooler cause..."
"the value's line up!!!"}}])
(->> sample-input convert println)
The terminal output is (psst... the values in a map do line up but I don't think that chrome uses a monospaced font!):
Very simple + silly question:
Does clojure provide multi maps? I currently have something like this:
(defn wrap [func]
(fn [mp x]
(let [k (func x)]
(assoc mp k
(match (get mp k)
nil [x]
v (cons v x))))))
(defn create-mm [func lst]
(reduce (wrap func) {} lst))
Which ends up creating a map, where for each key, we have a vector of all elements with that key. However, it seems like multi maps is a very basic data structure, and I wonder if clojure has it built-in.
Thanks
I don't think this is really necessary as a distinct type, as Clojure's flexibility allow you to quickly make your own by just using maps and sets. See here:
http://paste.lisp.org/display/89840
Edit (I should have just pasted this in since it's so small)
Example Code (Courtesy Stuart Sierra)
(ns #^{:doc "A multimap is a map that permits multiple values for each
key. In Clojure we can represent a multimap as a map with sets as
values."}
multimap
(:use [clojure.set :only (union)]))
(defn add
"Adds key-value pairs the multimap."
([mm k v]
(assoc mm k (conj (get mm k #{}) v)))
([mm k v & kvs]
(apply add (add mm k v) kvs)))
(defn del
"Removes key-value pairs from the multimap."
([mm k v]
(let [mmv (disj (get mm k) v)]
(if (seq mmv)
(assoc mm k mmv)
(dissoc mm k))))
([mm k v & kvs]
(apply del (del mm k v) kvs)))
(defn mm-merge
"Merges the multimaps, taking the union of values."
[& mms]
(apply (partial merge-with union) mms))
(comment
(def mm (add {} :foo 1 :foo 2 :foo 3))
;; mm == {:foo #{1 2 3}}
(mm-merge mm (add {} :foo 4 :bar 2))
;;=> {:bar #{2}, :foo #{1 2 3 4}}
(del mm :foo 2)
;;=> {:foo #{1 3}}
)
Extra test for the case pointed out in the comments:
(comment
(-> {} (add :a 1) (del :a 1) (contains? :a))
;;=> false
)