I have a map obtained from running a sql query.
(defn print-info
[rs]
(doseq [req rs]
(let [rs1 req]
(println "rs1" rs1))))
(defn display-info
[uname]
(sql/with-connection
db
(sql/with-query-results rs
["Select * from user"]
(print-info rs))))
Now i have copied the result set into another map. Is there any way to add another record to this existing map and return this new map to another function ?
If you use the non-deprecated query syntax which was explained in your other question, you can use the :row-fn to manipulate each record in a resultset. A default row is a map, so if you use only functions that return a map (like in this case select-keys), you can just use assoc to add a new key-value pair to the map.
(query db ["select * from user"]
:row-fn #(assoc (select-keys % [:name]) :type :user))
Generally speaking when I want to make a fix to a record I'll use merge, which merges any number of maps in a left-to-right direction. Thus:
user=> (merge {:foo 1 :bar 2 :ban 3} {:bar 20} {:ban 300})
{:foo 1, :bar 20, :ban 300}
Although the assoc approach mentioned above also works, I find merge generally more useful and more flexible.
This applies to anything represented as a map, of course, not just records!
As an aside I'd strongly recommend using [Korma|http://sqlkorma.com/] for your database queries.
Related
I really like using contains? because it's so terse and readable. I want to see if a set contains maps that have the same key and value pairs of an example that also had other key value pairs. I'm pretty sure contains? won't work here. Is there an alternative? Maybe I'll have to write one (I'm finally getting into the mindset!). For example, if I had
(def some-set #{{:foo "bar" :beep "boop"}{:foo "bar"} {:foo "bar" :hi "there"}})
what would be a quick way to know if it had any maps that matched {:foo "bar" :one "two"} on :foo "bar"?
Edited: Remembering that a map is a collection of key-value vectors, here is an implementation for the predicate submap?:
(defn submap?
"Returns true if subm is a submap of m, false otherwise."
[subm m]
(every? (fn [[k v]] (= (get m k ::not-found) v)) subm))
This predicate can be used to filter any collection:
(filter #(submap? {:a 1 :b 2} %) [{:a 1} {:a 1 :b 2 :c 3}])
=> ({:a 1, :b 2, :c 3})
Original answer
This solution works but is slower than my updated answer, due to the construction of (set m) for large m
(defn submap?
"Returns true if subm is a submap of m, false otherwise."
[subm m]
(let [kvs (set m)]
(every? kvs subm)))
A generic way would be to write a predicate, that checks if a map
contains another map. This can be done using select-keys to only get
a map with certain keys; using the keys from the map to compare and
then just comparing the result will give you that.
(def maps #{{:foo "bar" :beep "boop"} {:foo "bar"} {:foo "bar" :hi "there"} {:foo "baz"}})
(defn submap?
[submap m]
(= (select-keys m (keys submap)) submap))
(println
(filter (partial submap? {:foo "bar"}) maps))
; → ({:foo bar, :beep boop} {:foo bar, :hi there} {:foo bar})
Yet this is just a simple sequential search. This does not (and AFAIR
there is nothing in core to help) utilize your maps being in a set.
Also note, that the order of the result is undefined since the order of
sets is too.
You can find many predicates of this nature and related helper functions in the Tupelo library, in particular:
submap?
submatch?
wild-match?
wild-submatch?
These are especially helpful in writing unit tests. For example, you may only care about certain fields like :body when testing a webserver response, and you want to ignore other fields like the IP address or a timestamp.
The unit tests show the code in action.
I need to apply additional logic (like mapping, conditionals, aggregating) to entities I get from Datomic. I had hard time translating it to Datomic query (I'm not sure if it's even possible in my case), which is why I used datomic's raw index access instead, so the most work and logic is done in Clojure.
It worked fine until I got to ~500K entries and the whole approach is getting very slow.
The relevant code:
(defn e->entry
"Map e into entry"
[e]
{:id (:entry/uuid e)
;; each flat field increases mapping time (seems linearly)
:date (:entry/date e)
:summ (:entry/summ e)
;; although when using nested fields, mapping time rises significantly
:groups (map #(-> % :dimension/group :group/name)
(:entry/dimensions e))})
;; query code:
(->> (d/datoms db :aevt :entry/uuid)
(map #(->> %
:e
(d/entity db)
e->entry))))
;; TODO: other actions on mapped entries ...
It takes about 30 seconds to run query code just to map entities and the more fields I need in my query, the more it takes.
Is this an expected behavior? Is there a way I can speed things up or am I missing something and this is bad approach?
To fully answer this question would require more information, please feel free to ask on the forum or open a support ticket.
I ended up with following optimizations, in case someone will need it:
(defn eid->entry
"Mapping via :eavt index"
[db eid]
(->> (d/datoms db :eavt eid) ; access all datoms by eid once
(seq)
(reduce (fn [m dtm]
(let [attr-key (d/ident db (:a dtm))
v (:v dtm)]
(assoc m attr-key v))))))
;; new query code
(->> (d/datoms db :aevt :entry/uuid)
(pmap #(->> %
:e
(eid->entry db))))
I used pmap instead of map and resorted to :eavt index to get all attributes and values of entity instead of accessing fields directly with d/entity
I'm running two select statements against Cassandra, so instead of having a join I need to join them in code. Being relatively new to Clojure, I'm having a hard time doing this without resorting to really ugly nested loops. Furthermore, if table-b is missing a matching entry from table-a, it should add default table-b values.
The two selects each result in a list of maps (each "row" is one map). The id key is a UUID, not string.
Here's how the selects look if I def something with the same structure.
(def table-a (list {:id "105421db-eca4-4500-9a2c-08f1e09a35ca" :col-b "b-one"}
{:id "768af3f3-3981-4e3f-a93d-9758cd53a056" :col-b "b-two"}))
(def table-b (list {:id "105421db-eca4-4500-9a2c-08f1e09a35ca" :col-c "c-one"}))
I want the end result to be this:
({:id "105421db-eca4-4500-9a2c-08f1e09a35ca" :col-b "b-one" :col-c "c-one"}
{:id "768af3f3-3981-4e3f-a93d-9758cd53a056" :col-b "b-two" :col-c "default-value"})
Thanks for any help.
This can be done by splitting it into groups with the same key, merging all the like-keyed maps and then filling in the default values:
user> (->> (concat table-a table-b) ;; stat with all the data
(sort-by :id) ;; split it into groups
(partition-by :id) ;; by id
(map (partial apply merge)) ;; merge each group into a single map.
(map #(assoc % ;; fill in the missing default values.
:col-c (or (:col-c %) "default value")
:col-b (or (:col-b %) "default value"))))
({:col-c "c-one",
:col-b "b-one",
:id "105421db-eca4-4500-9a2c-08f1e09a35ca"}
{:col-c "default value",
:col-b "b-two",
:id "768af3f3-3981-4e3f-a93d-9758cd53a056"})
Using the thread-last macro ->> makes this a lot easier for me to read, though that is just my opinion. There is also likely a more elegant way to supply the default keys.
Following up from this question: Idiomatic clojure map lookup by keyword
Map access using clojure can be done in many ways.
(def m {:a 1}
(get m :a) ;; => 1
(:a m) ;; => 1
(m :a) ;; => 1
I know I use mainly the second form, and sometimes the third, rarely the first. what are the advantages (speed/composability) of using each?
get is useful when the map could be nil or not-a-map, and the key could be something non-callable (i.e. not a keyword)
(def m nil)
(def k "some-key")
(m k) => NullPointerException
(k m) => ClassCastException java.lang.String cannot be cast to clojure.lang.IFn
(get m k) => nil
(get m :foo :default) => :default
From the clojure web page we see that
Maps implement IFn, for invoke() of one argument (a key) with an
optional second argument (a default value), i.e. maps are functions of
their keys. nil keys and values are ok.
Sometimes it is rewarding to take a look under the hoods of Clojure. If you look up what invoke looks like in a map, you see this:
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/APersistentMap.java#L196
It apparently calls the valAt method of a map.
If you look at what the get function does when called with a map, this is a call to clojure.lang.RT.get, and this really boils down to the same call to valAt for a map (maps implement ILookUp because they are Associatives):
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L634.
The same is true for a map called with a key and a not-found-value. So, what is the advantage? Since both ways boil down to pretty much the same, performance wise I would say nothing. It's just syntactic convenience.
You can pass get to partial etc. to build up HOFs for messing with your data, though it doesn't come up often.
user=> (def data {"a" 1 :b 2})
#'user/data
user=> (map (partial get data) (keys data))
(1 2)
I use the third form a lot when the data has strings as keys
I don't think there is a speed difference, and even if that would be the case, that would be an implementation detail.
Personally I prefer the second option (:a m) because it sometimes makes code a bit easier on the eye. For example, I often have to iterate through a sequence of maps:
(def foo '({:a 1} {:a 2} {:a 3}))
If I want to filter all values of :a I can now use:
(map :a foo)
Instead of
(map #(get % :a) foo)
or
(map #(% :a) foo)
Of course this is a matter of personal taste.
To add to the list, get is also useful when using the threading macro -> and you need to access via a key that is not a keyword
(let [m {"a" :a}]
(-> m
(get "a")))
One advantage of using the keyword first approach is it is the most concise way of accessing the value with a forgiving behavior in the case the map is nil.
First, I assume each structure-specific sequences would have different ways to remove an item: Vectors could be by index, List could be remove first or last, Set should be passing of the actual item to remove, etc.
Second, I assume there are some methods for removal that are structure agnostic; they work on seq interface.
Since sequences are immutable in Clojure, I suspect what you're actually doing is making a cheap copy of the original, only without the original item. This means list comprehension could be used for removal, but I suspect it would be unnecessarily verbose.
Please give some idiomatic examples of the different ways to remove items from Clojure sequences.
There is no single interface for removing things from all of Clojure's data structure types, possibly because of the different performance characteristics.
(disj #{:foo :bar} :foo) ; => #{:bar}
(dissoc {:foo 1 :bar 2} :foo) ; => {:bar 2}
(pop [:bar :foo]) ; => [:bar]
(pop (list :foo :bar)) ; => (:bar)
These also work (returning a seq):
(remove #{:foo} #{:foo :bar}) ; => (:bar)
(remove #{:foo} [:foo :bar]) ; => (:bar)
(remove #{:foo} (list :foo :bar)) ; => (:bar)
This doesn't work for hash-maps because when you iterate over a map, you get key/value pairs. But this works:
(remove (fn [[k v]] (#{:foo} k)) {:foo 1 :bar 2}) ; => ([:bar 2])
Look at the Clojure reference for sequences. filter and remove are what you seek.
As an extension of Brian Carper's answer. It depends on what you will be doing with the result. If you are passing the result to something that wants to work on the entire set of data (ie to print it) It is idiomatic to make a seq and use filter or remove to solve the problem lazily. If on the other hand you are modifying the data structure to save for various later uses then creating a seq on it would loose its favorable update characteristics so in this case its better to use the update function specific to that data structure.