I'm new in Clojure Programming. Have been playing with it for a week now. I read the book "Clojure Programming" and after reading again and again and try to dissect this function in REPL but don't know how exactly this function works:
(defn reduce-by [key-fn f init coll]
(reduce (fn [summaries x]
(let [k (key-fn x)]
(assoc summaries k (f (summaries k init) x))))
{} coll))
I am still cannot understand the assoc part:
(assoc summaries k (f (summaries k init) x))))
Especially in the (summaries k init). It doesn't look like a function because summaries is defined as a map.
The function is intended to be used as follows
(def orders
[{:product "Clock", :customer "Wile Coyote", :qty 6, :total 300}
{:product "Dynamite", :customer "Wile Coyote", :qty 20, :total 5000}
{:product "Shotgun", :customer "Elmer Fudd", :qty 2, :total 800}
{:product "Shells", :customer "Elmer Fudd", :qty 4, :total 100}
{:product "Hole", :customer "Wile Coyote", :qty 1, :total 1000}
{:product "Anvil", :customer "Elmer Fudd", :qty 2, :total 300}
{:product "Anvil", :customer "Wile Coyote", :qty 6, :total 900}])
(reduce-by :customer #(+ %1 (:total %2)) 0 orders)
and it will yield a seq like below
;= {"Elmer Fudd" 1200, "Wile Coyote" 7200}
I appreciate to anyone who can explain it to me.
Thanks
Okay, looks like I figured it out.
Because maps are also functions, (summaries k init) will return the value of init if the map summaries doesn't contain key k.
Silly me to skim and forget.
Related
The problem context relates to stock trading. I'm trying to update the holdings for a particular stock, when a sale is made. Simplified excerpt
;; #holdings - an atom
{ "STOCK1" {:trades [Trade#{:id 100 :qty 50}, Trade#{ :id 140 :qty 50}]}
"STOCK2" ... }
Now given a sale trade of Trade{:id 200 :stock "STOCK1", :qty 75}, I'm expecting the holdings to reflect
{ "STOCK1" {:trades [Trade#{:id 100 :qty 0}, Trade#{ :id 140 :qty 25}]} }
;; or better drop the records with zero qty.
{ "STOCK1" {:trades [Trade#{ :id 140 :qty 25}]} }
The functional answer eludes me.. All I can see is a doseq loop with atoms to hold state (like sale-qty which may be satisfied by 1 or n trades) - but it feels like C in Clojure.
Is there a more clojure-aligned solution to this? Map doesnt look like a fit because every record processing needs to update an external state (pending sale-qty 75 -> 25 -> 0)
Disclaimer: Clojure Newbie, who wants to learn.
(require '[com.rpl.specter :as s])
(let [stocks {"STOCK1" {:trades [{:trade/id 100 :trade/qty 50}, {:trade/id 140 :trade/qty 50}]}}
sale-trade {:trade/id 200 :trade/stock "STOCK1" :trade/qty 75}
trade-path [(s/keypath (:trade/stock sale-trade) :trades) s/ALL]
qty-path (conj trade-path :trade/qty)
[new-qty _] (reduce (fn [[new-amounts leftover] v]
(let [due-amount (min v leftover)]
[(conj new-amounts (- v due-amount)) (- leftover due-amount)]))
[[] (:trade/qty sale-trade)]
(s/select qty-path stocks))]
(->> stocks
(s/setval (s/subselect qty-path) new-qty)
(s/setval [trade-path #(zero? (:trade/qty %))] s/NONE)))
=> {"STOCK1" {:trades [#:trade{:id 140, :qty 25}]}}
Whenever you want to go over a sequence/collection in Clojure, while passing some additional state around think of reduce Reduce is like a Swiss army knife, for example map and filter can both be implemented with reduce. But how can you store multiple states in a reducing function? You simply use a map as the accumulator.
Let me distill your problem a bit. Let's create a function that only deals with one problem.
(defn substract-from
"Given a seq of numbers `values`, substract the number `value` from each number
in `values` until whole `value` is substracted. Returns a map with 2 keys, :result contains
a vector of substracted values and :rem holds a remainder."
[values value]
(reduce (fn [{:keys [rem] :as result} n]
(if (zero? rem)
(update result :result conj n)
(let [sub (min rem n)
res (- n sub)
rem (Math/abs (- sub rem))]
(-> result
(update :result conj res)
(assoc :rem rem)))))
{:rem value :result []}
values))
;; when value is smaller than the sum of all values, remainder is 0
(substract-from [100 200 300 400] 500)
;; => {:rem 0, :result [0 0 100 400]}
;; when value is larger than the sum of all values, remainder is > 0
(substract-from [100 200 300 400] 1200)
;; => {:rem 200, :result [0 0 0 0]}
Now we can use this function to sell stocks. Note that map can accept multiple collections/sequences as arguments.
(def stocks
(atom { "STOCK1" {:trades [{:id 100 :qty 50} { :id 140 :qty 50}]}}))
(defn sell [stocks {:keys [id stock qty]}]
(let [trades (get-in stocks [stock :trades])
qtys (map :qty trades)
new-qtys (:result (substract-from qtys qty))]
(map (fn [trade qty]
(assoc trade :qty qty))
trades
new-qtys)))
(sell #stocks {:id 300 :qty 75 :stock "STOCK1"})
;; => ({:id 100, :qty 0} {:id 140, :qty 25})
i would probably start with finding out which part of essential functionality is absent from the core library. In your case it is the function to map over the collection while keeping some changing state.
It could look this way:
(defn map-state [f state data]
(when-let [[x & xs] (seq data)]
(lazy-seq
(let [[new-state new-x] (f state x)]
(cons new-x (map-state f new-state xs))))))
small example of how it could work in context like yours:
(def running-subtract (partial map-state
#(let [qty (min %1 %2)]
[(- %1 qty) (- %2 qty)])))
#'user/running-subtract
user> (running-subtract 10 (range 7))
;;=> (0 0 0 0 0 5 6)
so, you can use it to subtract the state from your trades:
(defn running-decrease-trades [trades amount]
(map-state (fn [amount trade]
(let [sub (min (:qty trade) amount)]
[(- amount sub) (update trade :qty - sub)]))
amount
trades))
and transforming your data with this function would be as easy as the following:
(defn handle-trade [data {:keys [stock qty]}]
(update-in data [stock :trades] running-decrease-trades qty))
user> (handle-trade
{"STOCK1" {:trades [{:id 100, :qty 50} {:id 140, :qty 50}]}}
{:stock "STOCK1" :qty 75})
{"STOCK1" {:trades ({:id 100, :qty 0} {:id 140, :qty 25})}}
Although i like specter very much, i would say it is an overkill for this one.
Unlike imperative programming, where you often modify values in place, in functional programming you instead create new values that contain the modifications. So you will have to create a new version of your map (using update-in) that contains a modified vector with your trades. Something like this:
(def conj-positive-trade ((filter (comp pos? :qty)) conj))
(defn sell [trades sale]
(update-in trades
[(:stock sale) :trades]
#(first
(reduce (fn [[dst remaining] {:keys [qty id]}]
(let [diff (- qty remaining)]
[(conj-positive-trade dst {:id id :qty diff})
(max 0 (- diff))]))
[[] (:qty sale)]
%))))
Here, conj-positive-trade is a function that only conjoins positive trades to a vector.
Here is how to use the sell function:
(sell {"STOCK1" {:trades [{:id 100 :qty 50} {:id 140 :qty 50} {:id 150 :qty 70}]}}
{:id 200 :stock "STOCK1", :qty 75})
;; => {"STOCK1" {:trades [{:id 140, :qty 25} {:id 150, :qty 70}]}}
As an alternative solution that wouldn't use specter (which is great, but requires buy-in). I would keep two atoms, one that is a raw listing of all trades (a vector of maps that you just conj to, so for instance {:trade-id 1 :name "AAPL" :price 100 :qty 20}]), and another that is a map of maps indexed by stock name grouped-result. You'd go from one to the other by group-by or filter so if you added a trade in "AAPL" you can update the quantity as such:
(swap! grouped-result update-in ["AAPL"] (-> #listing (filter #(= (:name %) "AAPL")) (map :qty) (reduce +)))
When it comes to the trade-id you keep it's a bit more complicated as when you factor in PnL there can be FIFO or LIFO considerations - but again you can use reductions or reduced to stop where you want.
Given a vector:
(def vec [{:key 1, :value 10, :other "bla"}, {:key 2, :value 13, :other "bla"}, {:key 1, :value 7, :other "bla"}])
I'd like to iterate over each element and update :value with the sum of all :values to that point, so I would have:
[{:key 1, :value 10, :other "bla"}, {:key 2, :value 23, :other "bla"}, {:key 1, :value 30, :other "bla"}])
I've found this for printing the result, but I've tried to change the prn command to update-in, assoc-in in the code below (extracted from the link above) but I didn't work quite well.
(reduce (fn [total {:keys [key value]}]
(let [total (+ total value)]
(prn key total)
total))
0 vec)
I'm new to Clojure, how can I make it work?
If you want to get the running totals then the simplest way is to use reductions:
(reductions (fn [acc ele] (+ acc (:value ele)))
0
[{:key 1, :value 10, :other "bla"}, {:key 2, :value 13, :other "bla"}, {:key 1, :value 7, :other "bla"}])
;; => (0 10 23 30)
As you can see the function you pass to reductions has the same signature as the function you pass to a reduce. It is like you are asking for a reduce to be done every time a new element is reached. Another way of thinking about it is that every time a new accumulator is calculated it is kept, unlike with reduce where the caller only gets to see the result of the last calculation.
And so this is the code that would directly answer your question:
(->> [{:key 1, :value 10, :other "bla"}, {:key 2, :value 13, :other "bla"}, {:key 1, :value 7, :other "bla"}]
(reductions #(update %2 :value + (:value %1))
{:value 0})
next
vec)
;; => [{:key 1, :value 10, :other "bla"} {:key 2, :value 23, :other "bla"} {:key 1, :value 30, :other "bla"}]
You can accumulate the :values thus:
(reductions + (map :value v))
=> (10 23 30)
(I renamed the vector v to avoid tripping over clojure.core/vec.)
Then you can use mapv over assoc:
(let [value-sums (reductions + (map :value v))]
(mapv #(assoc %1 :value %2) v value-sums))
=> [{:key 1, :value 10, :other "bla"} {:key 2, :value 23, :other "bla"} {:key 1, :value 30, :other "bla"}]
I would like to update values in hashes, but I'm not sure how this can be done efficiently
I tried using a loop approach, but keeping the previous record's value also in account seems like a big challenge.
This is what I am trying to do,
Considering the records are sorted based on created_at in descending order, For example,
[{:id 1, :created_at "2016-08-30 11:07:00"}{:id 2, :created_at "2016-08-30 11:05:00"}...]
]
; Basically in humanised form.
Could anyone share some ideas to achieve this? Thanks in advance.
Simplified example:
(def data [{:value 10} {:value 8} {:value 3}])
(conj
(mapv
(fn [[m1 m2]] (assoc m1 :difference (- (:value m1) (:value m2))))
(partition 2 1 data))
(last data))
;;=> [{:value 10, :difference 2} {:value 8, :difference 5} {:value 3}]
what you need, is to iterate over all the pairs of consecutive records, keeping the first of them, adding the difference to it.
first some utility functions for dates handling:
(defn parse-date [date-str]
(.parse (java.text.SimpleDateFormat. "yyyy-MM-dd HH:mm:ss") date-str))
(defn dates-diff [date-str1 date-str2]
(- (.getTime (parse-date date-str1))
(.getTime (parse-date date-str2))))
then the mapping part:
user> (def data [{:id 1, :created_at "2016-08-30 11:07:00"}
{:id 2, :created_at "2016-08-30 11:05:00"}
{:id 3, :created_at "2016-08-30 10:25:00"}])
user> (map (fn [[rec1 rec2]]
(assoc rec1 :difference
(dates-diff (:created_at rec1)
(:created_at rec2))))
(partition 2 1 data))
({:id 1, :created_at "2016-08-30 11:07:00", :difference 120000}
{:id2, :created_at "2016-08-30 11:05:00", :difference 2400000})
notice that it doesn't contain the last item, since it was never the first item of a pair. So you would have to add it manually:
user> (conj (mapv (fn [[rec1 rec2]]
(assoc rec1 :difference
(dates-diff (:created_at rec1)
(:created_at rec2))))
(partition 2 1 data))
(assoc (last data) :difference ""))
[{:id 1, :created_at "2016-08-30 11:07:00", :difference 120000}
{:id 2, :created_at "2016-08-30 11:05:00", :difference 2400000}
{:id 3, :created_at "2016-08-30 10:25:00", :difference ""}]
now it's ok. The only difference with your desired variant, is that the diff is in millis, rather than formatted string. To do that you can add the formatting to the dates-diff function.
I have a collection of maps
(def a '({:id 9345 :value 3 :type "orange"}
{:id 2945 :value 2 :type "orange"}
{:id 145 :value 3 :type "orange"}
{:id 2745 :value 6 :type "apple"}
{:id 2345 :value 6 :type "apple"}))
I want to group this first by value, followed by type.
My output should look like:
{
:orange [{
:value 3,
:id [9345, 145]
}, {
:value 2,
:id [2935]
}],
:apple [{
:value 6,
:id [2745, 2345]
}]
}
How would I do this in Clojure? Appreciate your answers.
Thanks!
Edit:
Here is what I had so far:
(defn by-type-key [data]
(group-by #(get % "type") data))
(reduce-kv
(fn [m k v] (assoc m k (reduce-kv
(fn [sm sk sv] (assoc sm sk (into [] (map #(:id %) sv))))
{}
(group-by :value (map #(dissoc % :type) v)))))
{}
(by-type-key a))
Output:
=> {"orange" {3 [9345 145], 2 [2945]}, "apple" {6 [2745 2345], 3 [125]}}
I just couldnt figure out how to proceed next...
Your requirements are a bit inconsistent (or rather irregular) - you use :type values as keywords in the result, but the rest of the keywords are carried through. Maybe that's what you must do to satisfy some external formats - otherwise you need to either use the same approach as with :type through, or add a new keyword to the result, like :group or :rows and keep the original keywords intact. I will assume the former approach for the moment (but see below, I will get to the shape as you want it,) so the final shape of data is like
{:orange
{:3 [9345 145],
:2 [2945]},
:apple
{:6 [2745 2345]}
}
There is more than one way of getting there, here's the gist of one:
(group-by (juxt :type :value) a)
The result:
{["orange" 3] [{:id 9345, :value 3, :type "orange"} {:id 145, :value 3, :type "orange"}],
["orange" 2] [{:id 2945, :value 2, :type "orange"}],
["apple" 6] [{:id 2745, :value 6, :type "apple"} {:id 2345, :value 6, :type "apple"}]}
Now all rows in your collection are grouped by the keys you need. From this, you can go and get the shape you want, say to get to the shape above you can do
(reduce
(fn [m [k v]]
(let [ks (map (comp keyword str) k)]
(assoc-in m ks
(map :id v))))
{}
(group-by (juxt :type :value) a))
The basic idea is to get the rows grouped by the key sequence (and that's what group-by and juxt do,) and then combine reduce and assoc-in or update-in to beat the result into place.
To get exactly the shape you described:
(reduce
(fn [m [k v]]
(let [type (keyword (first k))
value (second k)
ids (map :id v)]
(update-in m [type]
#(conj % {:value value :id ids}))))
{}
(group-by (juxt :type :value) a))
It's a bit noisy, and it might be harder to see the forest for the trees - that's why I simplified the shape, to highlight the main idea. The more regular your shapes are, the shorter and more regular your functions become - so if you have control over it, try to make it simpler for you.
I would do the transform in two stages (using reduce):
the first to collect the values
the second for formating
The following code solves your problem:
(def a '({:id 9345 :value 3 :type "orange"}
{:id 2945 :value 2 :type "orange"}
{:id 145 :value 3 :type "orange"}
{:id 2745 :value 6 :type "apple"}
{:id 2345 :value 6 :type "apple"}))
(defn standardise [m]
(->> m
;; first stage
(reduce (fn [out {:keys [type value id]}]
(update-in out [type value] (fnil #(conj % id) [])))
{})
;; second stage
(reduce-kv (fn [out k v]
(assoc out (keyword k)
(reduce-kv (fn [out value id]
(conj out {:value value
:id id}))
[]
v)))
{})))
(standardise a)
;; => {:orange [{:value 3, :id [9345 145]}
;; {:value 2, :id [2945]}],
;; :apple [{:value 6, :id [2745 2345]}]}
the output of the first stage is:
(reduce (fn [out {:keys [type value id]}]
(update-in out [type value] (fnil #(conj % id) [])))
{}
a)
;;=> {"orange" {3 [9345 145], 2 [2945]}, "apple" {6 [2745 2345]}}
You may wish to use the built-in function group-by. See http://clojuredocs.org/clojure.core/group-by
This question already has answers here:
Turn a hash map inside out in Clojure
(3 answers)
Closed 8 years ago.
In clojure, I have a map that contains for each day, and each fruit, the number of fruits eaten. I would like to "reverse the hierarchy" of the map and to return the same data but with the fruits at the top of the hierarchy.
I will explain by an example:
(map-reverse-hierarchy {:monday {:banana 2 :apple 3}
:tuesday {:banana 5 :orange 2}})
; => {:orange {:tuesday 2},
; :banana {:tuesday 5, :monday 2},
; :apple {:monday 3}}
You could use a list comprehension and some destructuring, like
user=> (->> (for [[day consum] data
#_=> [fruit amount] consum]
#_=> {fruit {day amount}})
#_=> (apply merge-with conj))
{:orange {:tuesday 2}, :banana {:tuesday 5, :monday 2}, :apple {:monday 3}}
user=>
or using a function + mapcat instead:
(defn flip [[day consum]]
(map (fn [[fruit amount]] {fruit {day amount}}) consum))
(apply merge-with conj (mapcat flip data))
My solution first transposes the pieces of the nested maps and then merges them all.
The pieces are transposed from {k1 {k2 v}}to {k2 {k1 v}} and then merged by apply merge-with conj
(defn map-reverse-hierarchy [mm]
(apply merge-with conj
(for [[k1 m] mm [k2 v] m] {k2 {k1 v}})))
Maybe:
(defn map-reverse-hierarchy [m]
(let [foo (fn [a lst]
(map #(do [(first %) {a (second %)}]) lst))
bbb (map (fn [[a b]] (into {} (foo a b))) m)]
(if (seq bbb)
(apply merge-with merge bbb)
{})))
(map-reverse-hierarchy {:monday {:banana 2 :apple 3}
:tuesday {:banana 5 :orange 2}})
;{:banana {:monday 2, :tuesday 5}, :apple {:monday 3}, :orange {:tuesday 2}}
I think you'll need some custom function. Use clojure.set/map-invert[1] to swap keys and values in hash-map
[1] http://clojure.github.io/clojure/clojure.set-api.html#clojure.set/map-invert
brute-force solution:
(defn x []
(let [i {:monday {:banana 2 :apple 3}
:tuesday {:banana 5 :orange 2}}]
(reduce-kv (fn [h day fruits]
(reduce-kv (fn [h fruit n]
(update-in h [fruit day] #(+ (or % 0) n))) h fruits)) {} i)))
user> (pprint (x))
{:orange {:tuesday 2},
:banana {:tuesday 5, :monday 2},
:apple {:monday 3}}