I'm new to the world of clojure and I have a doubt.
I have a nested map such as
(def accounts (hash-map :XYZ (hash-map :balance (hash-map 171000 0 :171018 500 :171025 200)
:statement (hash-map :171018 [{:desc "purchase" :amount 200}
{:desc "deposit" :amount 700}]
:171025 [{:desc "purchase" :amount 300}]))
And I want to update the statements, so I wrote a simple function:
(defn add-statement
[account date desc amount]
(def accounts (update-in accounts [account :statement date] conj {:desc desc :amount amount}))
But I have a feeling that I'm doing that the wrong way...
You would need to change accounts to be mutable if you want to update them. The usual way to do this is by making accounts an atom. Then your function might look like this:
(defn add-statement! [account date desc amount]
(swap! accounts update-in [account :statement date]
(fn [line-items]
(conj (or line-items []) {:desc desc :amount amount}))))
This will add a statement line-item at a new or an existing date. The update-in is the same as yours, except that a line-item for a new date will be put into a vector rather than a list. (conj keeps the type, but it has to know the type: (conj nil :a) gives (:a)).
To turn accounts into an atom:
(def accounts (atom (hash-map ...)))
I notice your balances are not correct anyway. But if you are updating them be sure to do so in the same swap! function.
To answer your question, "Is (def m (update-in m ...)) a good practice?". Definitely not inside a defn. If you are thinking to put a def inside a defn use a let instead. Outside of a defn it would be fine to have a def that updates another def that has a different name, so (def m2 (update-in m1 ...)).
Related
I have many JSON objects, and I am trying to filter those objects by the date. These objects are being parsed from several JSON files using Cheshire.core, meaning that the JSON objects are in a collection. The date is being passed in in the following format "YYYY-MM-DD" (eg. 2015-01-10). I have tried using the filter and contains? functions to do this, but I am having no luck so far. How can I filter these JSON objects by my chosen date?
Current Clojure code:
(def filter-by-date?
(fn [orders-data date-chosen]
(contains? (get (get orders-data :date) :date) date-chosen)))
(prn (filter (filter-by-date? orders-data "2017-12-25")))
Example JSON object:
{
"id":"05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
"date":{
"date":"2015-01-10T19:11:41.000Z"
},
"total":{
"GBP":57.45
}
}
JSON after parsing with Cheshire:
[({:id "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-01-10T19:11:41.000Z"},
:total {:GBP 57.45}}) ({:id "325bd04-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-02-23T10:15:14.000Z"},
:total {:GBP 32.90}})]
First, I'm going to assume you've parsed the JSON first into something like this:
(def parsed-JSON {:id "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-01-10T19:11:41.000Z"},
:total {:GBP 57.45}})
The main problem is the fact that the date as stored in the JSON contains time information, so you aren't going to be able to check it directly using equality.
You can get around this by using clojure.string/starts-with? to check for prefixes. I'm using s/ here as an alias for clojure.string:
(defn filter-by-date [date jsons]
(filter #(s/starts-with? (get-in % [:date :date]) date)
jsons))
You were close, but I made a few changes:
You can't use contains? like that. From the docs of contains?: Returns true if key is present in the given collection, otherwise returns false. It can't be used to check for substrings; it's used to test for the presence of a key in a collection.
Use -in postfix versions to access nested structures instead of using multiple calls. I'm using (get-in ...) here instead of (get (get ...)).
You're using (def ... (fn [])) which makes things more complicated than they need to be. This is essentially what defn does, although defn also adds some more stuff as well.
To address the new information, you can just flatten the nested sequences containing the JSONs first:
(->> nested-json-colls ; The data at the bottom of the question
(flatten)
(filter-by-date "2015-01-10"))
#!/usr/bin/env boot
(defn deps [new-deps]
(merge-env! :dependencies new-deps))
(deps '[[org.clojure/clojure "1.9.0"]
[cheshire "5.8.0"]])
(require '[cheshire.core :as json]
'[clojure.string :as str])
(def orders-data-str
"[{
\"id\":\"987654\",
\"date\":{
\"date\":\"2015-01-10T19:11:41.000Z\"
},
\"total\":{
\"GBP\":57.45
}
},
{
\"id\":\"123456\",
\"date\":{
\"date\":\"2016-01-10T19:11:41.000Z\"
},
\"total\":{
\"GBP\":23.15
}
}]")
(def orders (json/parse-string orders-data-str true))
(def ret (filter #(clojure.string/includes? (get-in % [:date :date]) "2015-01-") orders))
(println ret) ; ({:id 987654, :date {:date 2015-01-10T19:11:41.000Z}, :total {:GBP 57.45}})
You can convert the date string to Date object using any DateTime library like joda-time and then do a proper filter if required.
clj-time has functions for parsing strings and comparing date-time objects. So you could do something like:
(ns filter-by-time-example
(:require [clj-time.coerce :as tc]
[clj-time.core :as t]))
(def objs [{"id" nil
"date" {"date" "2015-01-12T19:11:41.000Z"}
"total" nil}
{"id" "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f"
"date" {"date" "2015-01-10T19:11:41.000Z"}
"total" {"GBP" :57.45}}
{"id" nil
"date" {"date" "2015-01-11T19:11:41.000Z"}
"total" nil}])
(defn filter-by-day
[objs y m d]
(let [start (t/date-time y m d)
end (t/plus start (t/days 1))]
(filter #(->> (get-in % ["date" "date"])
tc/from-string
(t/within? start end)) objs)))
(clojure.pprint/pprint (filter-by-day objs 2015 1 10)) ;; Returns second obj
If you're going to repeatedly do this (e.g. for multiple days) you could parse all dates in your collection into date-time objects with
(map #(update-in % ["date" "date"] tc/from-string) objs)
and then just work with that collection to avoid repeating the parsing step.
(ns filter-by-time-example
(:require [clj-time.format :as f]
[clj-time.core :as t]
[cheshire.core :as cheshire]))
(->> json-coll
(map (fn [json] (cheshire/parse-string json true)))
(map (fn [record] (assoc record :dt-date (f/format (get-in record [:date :date])))))
(filter (fn [record] (t/after? (tf/format "2017-12-25") (:dt-date record))))
(map (fn [record] (dissoc record :dt-date))))
Maybe something like this? You might need to change the filter for your usecase but as :dt-time is now a jodo.DateTime you can leverage all the clj-time predicates.
New to clojure. Trying to solve the following problem with a java background. I need to transform table to a hash-map that maps products to all the cities that sell the product. So the output should be.
{"Pencil": ("Oshawa" "Toronto")
"Bread": ("Ottawa" "Oshawa" "Toronto")}
(def table [
{:product "Pencil"
:city "Toronto"
:year "2010"
:sales "2653.00"}
{:product "Pencil"
:city "Oshawa"
:year "2010"
:sales "525.00"}
{:product "Bread"
:city "Toronto"
:year "2010"
:sales "136,264.00"}
{:product "Bread"
:city "Oshawa"
:year "nil"
:sales "242,634.00"}
{:product "Bread"
:city "Ottawa"
:year "2011"
:sales "426,164.00"}])
This is what I have so far. I write this code into the repl.
(let [product-cities {}]
(for [row table]
(if (= (contains? product-cities (keyword (row :product))) true)
(println "YAMON") ;;To do after. Add city to product if statement is true
(into product-cities {(keyword (row :product)) (str (row :city))}))))
However, the outcome is the following:
({:Pencil "Toronto"}
{:Pencil "Oshawa"}
{:Bread "Toronto"}
{:Bread "Oshawa"}
{:Bread "Ottawa"})
My if statement keeps returning false. I see that there are semi-circle brackets around the many hash-maps. I can't figure out why it's not returning one hashmap and why there are many hashmap? Thanks
EDIT:
QUESTION 2:
Transform table to a hash-map that maps products to the city that has the highest sale. For example, the output should look like:
{"Pencil": "Toronto"
"Bread": "Ottawa"}
I think a different strategy is needed than building up a value but here's what I'm thinking:
(reduce (fn [product-cities {:keys [product city sales]}]
(update-in product-cities [product] (fnil conj []) {(keyword city) sales}))
{}
table)
This produces the following output:
{"Bread"
[{:Toronto "136,264.00"}
{:Oshawa "242,634.00"}
{:Ottawa "426,164.00"}],
"Pencil"
[{:Toronto "2653.00"}
{:Oshawa "525.00"}]}
I could then use the reduce function again but only add the city with max sales. I don't think this is the most efficient way.
You seem to have some misconceptions about how clojure works. Coming from java it can be hard to know how to do stuff, just because it's so different. This small problem serves nicely as an introduction to how to build up a value, and I'll try to explain each part of the solution.
Immutability
It's a common pattern in java to define a variable that will hold the final result, then loop through something while adding to that variable.
That's what you're trying to do with your product-cities local. When you define a local with let in clojure it never changes, so to build up a value you need another pattern.
Adding something to a map
First let's take a look at how to "add something to a map". In clojure what you actually do is make a new map with the thing added. The old map doesn't change. We still sometimes phrase it as adding to a map, but that's just shorthand for "make a new map with the thing added".
assoc takes a map, a key and a value and returns a new map with the value added at the key. If there's already a value there it will be overwritten. We want multiple things for each key, so it's not the right thing in this case.
update is similar, but it takes a map, a key, a function and optionally arguments to that function. It will call the function with the value that's already at key as the first argument and (if present) the arguments supplied. The returned map will have the return value of the function as the new value at key. Some examples might make this clearer.
;; The function - is called with the old value of :foo and the argument supplied
;; (- 10 3)
(update {:foo 10} :foo - 3) ;=> {:foo 7}
If there's nothing already at key, the function will be called with nil as the first argument. That's what nil means, nothing.
(update {} :foo + 5) ;=> Null pointer exception. Same as (+ nil 5)
Null pointers are no good. There's a trick for avoiding them. fnil is a higher order function that takes a function and arguments. It returns a new function that will substitute a nil argument for the arguments supplied.
;; The nil here is substituted with 0, so no NPE
((fnil + 0) nil 5) ;=> 5
;; If the arg is not nil, the zero is not used.
((fnil + 0) 5 5) ;=> 10
;; So now we can update the value at :foo regardless of whether it's already there.
(update {} :foo (fnil + 0) 5) ;=> {:foo 5}
conj adds something to a collection. If that collection is a vector, it adds it at the end.
(conj [] :foo) ;=> :foo
(conj [:foo] :bar) ;=> [:foo :bar]
To add things to the map we can combine these:
(update {} "product" (fnil conj []) "city") ;=> {"product ["city"]"}
(update {"product" ["city"]} "product" (fnil conj []) "another city")
;;=> {"product" ["city" "another city"]}
Building up a value
We need to do some looping somehow. A for in clojure is a list comprehension however, and not a for loop. It will return a sequence of things, so it's not the right thing to use when you want to build up a value.
One way to do it is with loop.
With a loop you define binding names paired with their initial value. The bindings in loop can be thought of as "the things that are going to change during the loop".
One difference between loop and traditional loops is that to break out of the loop you simply don't do anything, and you need to specifically use recur to keep looping.
;; the bindings are pairs of names and initial values
(loop [product-cities {} ; This is the accumulator that we're gonna build up.
rows table] ; and these are the rows, we're gonna go through them one by one.
;; Here's the base case, if there are no rows left we return the accumulator.
(if (empty? rows)
product-cities
;; If there are rows left, we need to add to the accumulator.
(let [row (first rows)
city (:city row)
product (:product row)
new-accumulator (update product-cities product (fnil conj []) city)]
;; recur takes as many arguments as the pairs we defined
;; and "jumps" back to the loop
(recur new-accumulator (rest rows)))))
;;=> {"Pencil" ["Toronto" "Oshawa"], "Bread" ["Toronto" "Oshawa" "Ottawa"]}
This can be made nicer with some destructuring.
(loop [product-cities {}
[{:keys [city product] :as row} & rows] table]
(if (nil? row)
product-cities
(recur (update product-cities product (fnil conj []) city) rows)))
;;=> {"Pencil" ["Toronto" "Oshawa"], "Bread" ["Toronto" "Oshawa" "Ottawa"]}
loop is not much used in normal clojure though. It's too general and usually you want something more specific. Like in this case, you want to build up a value while looping through every thing in a sequence of things. That's what reduce does.
Reduce takes three arguments, a function, an initial value and a collection. The function, called a "reducing function" takes two arguments; the accumulated value so far and an item. It is called once for each item in the collection.
So the final implementation becomes:
(reduce (fn [product-cities {:keys [product city]}]
(update product-cities product (fnil conj []) city))
{}
table)
Edit:
About your comment on the other answer. update was added in clojure 1.7.0 so you're presumable on an older version. You can use update-in instead (though you should consider upgrading). It's called in exactly the same way except the key is in a vector.
(reduce (fn [product-cities {:keys [product city]}]
(update-in product-cities [product] (fnil conj []) city))
{}
table)
You need to go through the table one by one and accumulate (conj) the cities under the product:
(reduce
(fn [acc {:keys [product city]}]
(update acc product conj city))
{}
table)
;; => {"Pencil" ("Oshawa" "Toronto"), "Bread" ("Ottawa" "Oshawa" "Toronto")}
The use of the updating function (conj in this case) can be a bit tricky, so here is an alternative formation of the update:
(update acc product (fn [cities]
(conj cities city)))
Instead of {} I started out with:
{"Pencil" []
"Bread" []}
That might make it easier to see that for each entry in the table the update is updating the product key ("Pencil" or "Bread") by putting the latest city on the end (that's what conj does) of the sequence. When it was working I just replaced with {}, using the fact that update will insert a new key if one is not there.
I think of for as being generative. It takes a sequence as input and at each step generates a new thing - hence you end up with a sequence of new things. There is no updating of product-cities possible with generation.
reduce is more useful for what you want to do as you get an 'accumulator' that can be slightly modified at each step. Actually you are creating a new 'accumulator' each time, by modifying the one that is passed in, so in reality you are not modifying anything at all: Clojure being a functional language, its all about creating new things.
I have an atom called app-state that holds a map. It looks like this:
{:skills [{:id 1 :text "hi"} {:id 2 :text "yeah"}]}
What is the idiomatic way to remove the element inside the vector with :id = 2 ? The result would look like:
{:skills [{:id 1 :text "hi"}]}
...
So far, I have this:
(defn new-list [id]
(remove #(= id (:id %)) (get #app-state :skills)))
swap! app-state assoc :skills (new-list 2)
It works, but I feel like this isn't quite right. I think it could be something like:
swap! app-state update-in [:skills] remove #(= id (:id %))
But this doesn't seem to work.
Any help is much appreciated!
Try this:
(defn new-list [app-state-map id]
(assoc app-state-map :skills (into [] (remove #(= id (:id %)) (:skills app-state-map)))))
(swap! app-state new-list 2)
swap! will pass the current value of the atom to the function you supply it. There's no need to dereference it yourself in the function.
See the docs on swap! for more details.
(swap! state update :skills (partial remove (comp #{2} :id)))
(def skills {:skills [{:id 1 :text "hi"} {:id 2 :text "yeah"}]})
(defn remove-skill [id]
(update skills :skills (fn [sks] (vec (remove #(= id (:id %)) sks)))))
You would then be able to call say (remove-skill 1) and see that only the other one (skill with :id of 2) is left.
I like your way better. And this would need to be adapted for use against an atom.
You can use filter to do this. Here is a function that takes an id and the map and let's you filter out anything that doesn't match your criteria. Of course, you can make the #() reader macro check for equality rather than inequality depending on your needs.
user=> (def foo {:skills [{:id 1 :text "hi"} {:id 2 :text "yeah"}]})
#'user/foo
user=> (defn bar [id sklz] (filter #(not= (:id %) id) (:skills sklz)))
#'user/bar
user=> (bar 1 foo)
({:id 2, :text "yeah"})
user=> (bar 2 foo)
({:id 1, :text "hi"})
I tried to use directly Clojure's hashmap with MapDB and ran into weird behaviour. I checked Clojure and MapDB sources and couldn't understand the problem.
First everything looks fine:
lein try org.mapdb/mapdb "1.0.6"
; defining a db for the first time
(import [org.mapdb DB DBMaker])
(defonce db (-> (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
.closeOnJvmShutdown
.compressionEnable
.make))
(defonce fruits (.getTreeMap db "fruits-store"))
(do (.put fruits :banana {:qty 2}) (.commit db))
(get fruits :banana)
=> {:qty 2}
(:qty (get fruits :banana))
=> 2
(first (keys (get fruits :banana)))
=> :qty
(= :qty (first (keys (get fruits :banana))))
=> true
CTRL-D
=> Bye for now!
Then I try to access the data again:
lein try org.mapdb/mapdb "1.0.6"
; loading previsously created db
(import [org.mapdb DB DBMaker])
(defonce db (-> (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
.closeOnJvmShutdown
.compressionEnable
.make))
(defonce fruits (.getTreeMap db "fruits-store"))
(get fruits :banana)
=> {:qty 2}
(:qty (get fruits :banana))
=> nil
(first (keys (get fruits :banana)))
=> :qty
(= :qty (first (keys (get fruits :banana))))
=> false
(class (first (keys (get fruits :banana))))
=> clojure.lang.Keyword
How come the same keyword be different with respect to = ?
Is there some weird reference problem happening ?
The problem is caused by the way equality of keywords works. Looking at the
implementation of the = function we see that since keywords are not
clojure.lang.Number or clojure.lang.IPersistentCollection their equality is
determined in terms of the Object.equals method. Skimming the source of
clojure.lang.Keyword we learn that keywords don't not override
Object.equals and therefore two keywords are equal iff they are the same
object.
The default serializer of MapDB is org.mapdb.SerializerPojo, a subclass of
org.mapdb.SerializerBase. In its documentation we can read that
it's a
Serializer which uses ‘header byte’ to serialize/deserialize most of classes
from ‘java.lang’ and ‘java.util’ packages.
Unfortunately, it doesn't work that well with clojure.lang classes; It doesn't
preserve identity of keywords, thus breaking equality.
In order to fix it let's attempt to write our own serializer using the
EDN format—alternatively, you could consider, say, Nippy—and use
it in our MapDB.
(require '[clojure.edn :as edn])
(deftype EDNSeralizer []
;; See docs of org.mapdb.Serializer for semantics.
org.mapdb.Serializer
(fixedSize [_]
-1)
(serialize [_ out obj]
(.writeUTF out (pr-str obj)))
(deserialize [_ in available]
(edn/read-string (.readUTF in)))
;; MapDB expects serializers to be serializable.
java.io.Serializable)
(def edn-serializer (EDNSeralizer.))
(import [org.mapdb DB DBMaker])
(def db (.. (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
closeOnJvmShutdown
compressionEnable
make))
(def more-fruits (.. db
(createTreeMap "more-fruits")
(valueSerializer (EDNSeralizer.))
(makeOrGet)))
(.put more-fruits :banana {:qty 2})
(.commit db)
Once the more-fruits tree map is reopened in a JVM with EDNSeralizer defined
the :qty object stored inside will be the same object as any other :qty
instance. As a result equality checks will work properly.
If I'm using a Clojure atom to keep count of something (votes for example), I can do this:
(def votes (atom {}))
(defn vote! [candidate]
(swap! votes update-in [candidate] (fnil inc 0)))
(vote! "Abraham Lincoln")
(vote! "Abraham Lincoln")
(vote! "Winston Churchill")
votes ;=> {"Abraham Lincoln" 2, "Winston Churchill" 2}
Here, update-in neatly transforms the value at the given key, without having to look it up first.
How can I accomplish the same in Datomic? I could do something like this...
(defn vote! [db candidate]
(let [[e v] (first (q '[:find ?e ?v
:in $ ?candidate
:where [[?e :name ?candidate]
[?e :votes ?v]] db candidate)
(transact! conn [{:db/id e :votes (inc v)}]))
But it seems a bit cumbersome, having to run the query, return the value and then transact with the new value. Is there a more idiomatic way of doing this (like a swap! or update-in)?
To me it seems most idiomatic to record the fact of the vote, then make a count query when you want to know the total. Depending on the needs of your application, this approach supports things like checking for double votes, vote change/retraction, counts in the last 24 hours, ...