I am trying to pull the maximum and minimum results from an SQL query into Clojure so that I can perform a mathematical analysis on them but I am unsure as to why I am getting an error.
I have used the max and min functions in the code to try and determine these results although I keep getting two errors back that seem to relate to the way I am using the :counter keyword.
The data returned in a map from the SQL query looks like this:
{:date1 "20131007", :data "object1", :counter 1000}
{:date1 "20131007", :data "object2", :counter 50}
{:date1 "20131007", :data "object3", :counter 230}
When I use this code:
minvalue(min(map(keyword :counter)data2))
maxvalue(max(map(keyword :counter)data2))
valrange(- maxvalue minvalue)
valpc(* (/ valrange 100) 10)
x(- maxvalue valpc)
I would obviously want minvalue to be set as 50 and maxvalue to be set to 1000 although I am getting this error:
java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.Number
If I remove the map function from the code and run it again, I get this error:
java.lang.ClassCastException: clojure.lang.Keyword cannot be cast to java.lang.Number
Any help is appreciated on this as I'm completely stuck (and its probably pretty ovbious that I'm a novice at Clojure)! Thanks
A few changes from your code:
min and max take a variable number of parameters, rather than a collection. Use apply to, well apply the contents of a collection as parameters instead.
Clojure naming conventions use - between words
Valid calls have to be in a form (basically a list where the first item is callable)
No need to use "keyword", keyword is for when you have a string and need an associated keyword. For dragging data out of maps keywords act as functions themselves e.g (:a {:a 1 :b 2}) returns 1.
Factored out the commonality in your first two lines
(def data [{:date1 "20131007", :data "object1", :counter 1000}
{:date1 "20131007", :data "object2", :counter 50}
{:date1 "20131007", :data "object3", :counter 230}])
(def counters (map :counter data)) ; => (100 50 230)
(def min-value (apply min counters)) ; => 50
(def max-value (apply max counters)) ; => 1000
(def val-range (- max-value min-value)) ; => 950
(def val-pc (* (/ val-range 100) 10)) ; => 95
(def x (- max-value val-pc)) ; => 905
user=> data
[{:data "object1", :date1 "20131007", :counter 1000} {:data "object2", :date1 "20131007", :counter 50} {:data "object3", :date1 "20131007", :counter 230}]
user=> (apply max-key :counter data)
{:data "object1", :date1 "20131007", :counter 1000}
user=> (apply min-key :counter data)
{:data "object2", :date1 "20131007", :counter 50}
Related
The problem context relates to stock trading. I'm trying to update the holdings for a particular stock, when a sale is made. Simplified excerpt
;; #holdings - an atom
{ "STOCK1" {:trades [Trade#{:id 100 :qty 50}, Trade#{ :id 140 :qty 50}]}
"STOCK2" ... }
Now given a sale trade of Trade{:id 200 :stock "STOCK1", :qty 75}, I'm expecting the holdings to reflect
{ "STOCK1" {:trades [Trade#{:id 100 :qty 0}, Trade#{ :id 140 :qty 25}]} }
;; or better drop the records with zero qty.
{ "STOCK1" {:trades [Trade#{ :id 140 :qty 25}]} }
The functional answer eludes me.. All I can see is a doseq loop with atoms to hold state (like sale-qty which may be satisfied by 1 or n trades) - but it feels like C in Clojure.
Is there a more clojure-aligned solution to this? Map doesnt look like a fit because every record processing needs to update an external state (pending sale-qty 75 -> 25 -> 0)
Disclaimer: Clojure Newbie, who wants to learn.
(require '[com.rpl.specter :as s])
(let [stocks {"STOCK1" {:trades [{:trade/id 100 :trade/qty 50}, {:trade/id 140 :trade/qty 50}]}}
sale-trade {:trade/id 200 :trade/stock "STOCK1" :trade/qty 75}
trade-path [(s/keypath (:trade/stock sale-trade) :trades) s/ALL]
qty-path (conj trade-path :trade/qty)
[new-qty _] (reduce (fn [[new-amounts leftover] v]
(let [due-amount (min v leftover)]
[(conj new-amounts (- v due-amount)) (- leftover due-amount)]))
[[] (:trade/qty sale-trade)]
(s/select qty-path stocks))]
(->> stocks
(s/setval (s/subselect qty-path) new-qty)
(s/setval [trade-path #(zero? (:trade/qty %))] s/NONE)))
=> {"STOCK1" {:trades [#:trade{:id 140, :qty 25}]}}
Whenever you want to go over a sequence/collection in Clojure, while passing some additional state around think of reduce Reduce is like a Swiss army knife, for example map and filter can both be implemented with reduce. But how can you store multiple states in a reducing function? You simply use a map as the accumulator.
Let me distill your problem a bit. Let's create a function that only deals with one problem.
(defn substract-from
"Given a seq of numbers `values`, substract the number `value` from each number
in `values` until whole `value` is substracted. Returns a map with 2 keys, :result contains
a vector of substracted values and :rem holds a remainder."
[values value]
(reduce (fn [{:keys [rem] :as result} n]
(if (zero? rem)
(update result :result conj n)
(let [sub (min rem n)
res (- n sub)
rem (Math/abs (- sub rem))]
(-> result
(update :result conj res)
(assoc :rem rem)))))
{:rem value :result []}
values))
;; when value is smaller than the sum of all values, remainder is 0
(substract-from [100 200 300 400] 500)
;; => {:rem 0, :result [0 0 100 400]}
;; when value is larger than the sum of all values, remainder is > 0
(substract-from [100 200 300 400] 1200)
;; => {:rem 200, :result [0 0 0 0]}
Now we can use this function to sell stocks. Note that map can accept multiple collections/sequences as arguments.
(def stocks
(atom { "STOCK1" {:trades [{:id 100 :qty 50} { :id 140 :qty 50}]}}))
(defn sell [stocks {:keys [id stock qty]}]
(let [trades (get-in stocks [stock :trades])
qtys (map :qty trades)
new-qtys (:result (substract-from qtys qty))]
(map (fn [trade qty]
(assoc trade :qty qty))
trades
new-qtys)))
(sell #stocks {:id 300 :qty 75 :stock "STOCK1"})
;; => ({:id 100, :qty 0} {:id 140, :qty 25})
i would probably start with finding out which part of essential functionality is absent from the core library. In your case it is the function to map over the collection while keeping some changing state.
It could look this way:
(defn map-state [f state data]
(when-let [[x & xs] (seq data)]
(lazy-seq
(let [[new-state new-x] (f state x)]
(cons new-x (map-state f new-state xs))))))
small example of how it could work in context like yours:
(def running-subtract (partial map-state
#(let [qty (min %1 %2)]
[(- %1 qty) (- %2 qty)])))
#'user/running-subtract
user> (running-subtract 10 (range 7))
;;=> (0 0 0 0 0 5 6)
so, you can use it to subtract the state from your trades:
(defn running-decrease-trades [trades amount]
(map-state (fn [amount trade]
(let [sub (min (:qty trade) amount)]
[(- amount sub) (update trade :qty - sub)]))
amount
trades))
and transforming your data with this function would be as easy as the following:
(defn handle-trade [data {:keys [stock qty]}]
(update-in data [stock :trades] running-decrease-trades qty))
user> (handle-trade
{"STOCK1" {:trades [{:id 100, :qty 50} {:id 140, :qty 50}]}}
{:stock "STOCK1" :qty 75})
{"STOCK1" {:trades ({:id 100, :qty 0} {:id 140, :qty 25})}}
Although i like specter very much, i would say it is an overkill for this one.
Unlike imperative programming, where you often modify values in place, in functional programming you instead create new values that contain the modifications. So you will have to create a new version of your map (using update-in) that contains a modified vector with your trades. Something like this:
(def conj-positive-trade ((filter (comp pos? :qty)) conj))
(defn sell [trades sale]
(update-in trades
[(:stock sale) :trades]
#(first
(reduce (fn [[dst remaining] {:keys [qty id]}]
(let [diff (- qty remaining)]
[(conj-positive-trade dst {:id id :qty diff})
(max 0 (- diff))]))
[[] (:qty sale)]
%))))
Here, conj-positive-trade is a function that only conjoins positive trades to a vector.
Here is how to use the sell function:
(sell {"STOCK1" {:trades [{:id 100 :qty 50} {:id 140 :qty 50} {:id 150 :qty 70}]}}
{:id 200 :stock "STOCK1", :qty 75})
;; => {"STOCK1" {:trades [{:id 140, :qty 25} {:id 150, :qty 70}]}}
As an alternative solution that wouldn't use specter (which is great, but requires buy-in). I would keep two atoms, one that is a raw listing of all trades (a vector of maps that you just conj to, so for instance {:trade-id 1 :name "AAPL" :price 100 :qty 20}]), and another that is a map of maps indexed by stock name grouped-result. You'd go from one to the other by group-by or filter so if you added a trade in "AAPL" you can update the quantity as such:
(swap! grouped-result update-in ["AAPL"] (-> #listing (filter #(= (:name %) "AAPL")) (map :qty) (reduce +)))
When it comes to the trade-id you keep it's a bit more complicated as when you factor in PnL there can be FIFO or LIFO considerations - but again you can use reductions or reduced to stop where you want.
I want to memoize a function return that a function makes a http request to an API.
I'm unable to do it.
(defn _get_userid
[id cid]
(p1.nms2/get_uerid id cid))
(def get_userid
(memo/ttl _get_userid
{}
:ttl/threshold p1.constant/ttl-millisecs))
Given your 2nd parameter is like a context (for logging), you can use a dynamic var so you don't need to pass it as an extra argument to your memoized function.
(def ^:dynamic *cid* nil)
(def get-userid
(memoize
(fn [id]
{:input id
:context *cid*
:output (inc id)})))
(binding [*cid* "what"]
(get-userid 1))
;; => {:input 1, :context "what", :output 2}
(binding [*cid* "when"]
(get-userid 1))
;; => {:input 1, :context "what", :output 2}
(binding [*cid* "why"]
(get-userid 2))
;; => {:input 2, :context "why", :output 3}
My problem is next, i have list of maps, for example:
({:id 1 :request-count 10 ..<another key-value pair>..}
{:id 2 :request-count 15 ..<another key-value pair>..}
...)
Need create map with records in which, key is value of 'id' and value is value of 'request-count', for each map from prev example, like:
{1 10
2 15
...}
I know how to do this. My question is - standard library have function for achieve this? Or maybe i can achiev this with combination few function, without 'reduce'?
Use the juxt function to generate a sequence of pairs, and then toss them into a map:
(into {} (map (juxt :id :request-count) data))
Example:
user=> (def data [{:id 1 :request-count 10 :abc 1}
#_=> {:id 2 :request-count 15 :def 2}
#_=> {:id 3 :request-count 20 :ghi 3}])
#'user/data
user=> (into {} (map (juxt :id :request-count) data))
{1 10, 2 15, 3 20}
Be aware that if there is more than one map in data with the same :id, then the last one encountered by map will be the one that survives in the output map.
I would do it like so:
(def data
[{:id 1 :request-count 10}
{:id 2 :request-count 15}] )
(defn make-id-req-map [map-seq]
(vec (for [curr-map map-seq]
(let [{:keys [id request-count]} curr-map]
{id request-count}))))
With result:
(make-id-req-map data) => [{1 10} {2 15}]
Note: while you could combine the map destructuring into the for statement, I usually like to label the intermediate values as described in Martin Fowler's refactoring "Introduce Explaining Variable".
I have a collection of maps
(def a '({:id 9345 :value 3 :type "orange"}
{:id 2945 :value 2 :type "orange"}
{:id 145 :value 3 :type "orange"}
{:id 2745 :value 6 :type "apple"}
{:id 2345 :value 6 :type "apple"}))
I want to group this first by value, followed by type.
My output should look like:
{
:orange [{
:value 3,
:id [9345, 145]
}, {
:value 2,
:id [2935]
}],
:apple [{
:value 6,
:id [2745, 2345]
}]
}
How would I do this in Clojure? Appreciate your answers.
Thanks!
Edit:
Here is what I had so far:
(defn by-type-key [data]
(group-by #(get % "type") data))
(reduce-kv
(fn [m k v] (assoc m k (reduce-kv
(fn [sm sk sv] (assoc sm sk (into [] (map #(:id %) sv))))
{}
(group-by :value (map #(dissoc % :type) v)))))
{}
(by-type-key a))
Output:
=> {"orange" {3 [9345 145], 2 [2945]}, "apple" {6 [2745 2345], 3 [125]}}
I just couldnt figure out how to proceed next...
Your requirements are a bit inconsistent (or rather irregular) - you use :type values as keywords in the result, but the rest of the keywords are carried through. Maybe that's what you must do to satisfy some external formats - otherwise you need to either use the same approach as with :type through, or add a new keyword to the result, like :group or :rows and keep the original keywords intact. I will assume the former approach for the moment (but see below, I will get to the shape as you want it,) so the final shape of data is like
{:orange
{:3 [9345 145],
:2 [2945]},
:apple
{:6 [2745 2345]}
}
There is more than one way of getting there, here's the gist of one:
(group-by (juxt :type :value) a)
The result:
{["orange" 3] [{:id 9345, :value 3, :type "orange"} {:id 145, :value 3, :type "orange"}],
["orange" 2] [{:id 2945, :value 2, :type "orange"}],
["apple" 6] [{:id 2745, :value 6, :type "apple"} {:id 2345, :value 6, :type "apple"}]}
Now all rows in your collection are grouped by the keys you need. From this, you can go and get the shape you want, say to get to the shape above you can do
(reduce
(fn [m [k v]]
(let [ks (map (comp keyword str) k)]
(assoc-in m ks
(map :id v))))
{}
(group-by (juxt :type :value) a))
The basic idea is to get the rows grouped by the key sequence (and that's what group-by and juxt do,) and then combine reduce and assoc-in or update-in to beat the result into place.
To get exactly the shape you described:
(reduce
(fn [m [k v]]
(let [type (keyword (first k))
value (second k)
ids (map :id v)]
(update-in m [type]
#(conj % {:value value :id ids}))))
{}
(group-by (juxt :type :value) a))
It's a bit noisy, and it might be harder to see the forest for the trees - that's why I simplified the shape, to highlight the main idea. The more regular your shapes are, the shorter and more regular your functions become - so if you have control over it, try to make it simpler for you.
I would do the transform in two stages (using reduce):
the first to collect the values
the second for formating
The following code solves your problem:
(def a '({:id 9345 :value 3 :type "orange"}
{:id 2945 :value 2 :type "orange"}
{:id 145 :value 3 :type "orange"}
{:id 2745 :value 6 :type "apple"}
{:id 2345 :value 6 :type "apple"}))
(defn standardise [m]
(->> m
;; first stage
(reduce (fn [out {:keys [type value id]}]
(update-in out [type value] (fnil #(conj % id) [])))
{})
;; second stage
(reduce-kv (fn [out k v]
(assoc out (keyword k)
(reduce-kv (fn [out value id]
(conj out {:value value
:id id}))
[]
v)))
{})))
(standardise a)
;; => {:orange [{:value 3, :id [9345 145]}
;; {:value 2, :id [2945]}],
;; :apple [{:value 6, :id [2745 2345]}]}
the output of the first stage is:
(reduce (fn [out {:keys [type value id]}]
(update-in out [type value] (fnil #(conj % id) [])))
{}
a)
;;=> {"orange" {3 [9345 145], 2 [2945]}, "apple" {6 [2745 2345]}}
You may wish to use the built-in function group-by. See http://clojuredocs.org/clojure.core/group-by
I have a vector of entities (maps), I have filtered this vector to those entities with a particular key. Then I apply a function to update some values in those entities, and now have a subset that I'd like to merge with the original superset.
world [{a} {b} {c} {d} {e}]
a [{b} {d} {e}] ; (filter matches? world)
b [{b'} {d'} {e'}] ; (map func a)
new-world [{a} {b'} {c} {d'} {e'}] ; (merge world b)
How can I merge my update b with the original world?
My map structure is:
{:id identity
:attrs {:property1 {:param1 value
:param2 value}
:property2 {}}
There can be a variable number of properties. Properties can have 0 ({}) or more params. The only part of the map that is changing are the values, and only some of them.
I have to peer into each map to identify it, and then merge accordingly? What is an idiomatic way to do this?
Clojure's map function can take multiple sequences, but won't compare my subset b to the whole superset world, only b number of world entities, and stop once b is exhausted.
> (map #(str %1 " " %2) [1 2 3 4 5] [6 7 8])
("1 6" "2 7" "3 8")
Have I chosen the wrong structure for my data in the first place? Would I be better off without a vector and just one map?
{:entid1
{:property1 {:param1 value :param2 value}
:property2 {}}
:entid2
{:property1 {:param1 value :param2 value}
:property2 {:param1 value}
:property3 {:param1 value :param2 value :param3 value}}}
This question looks similar but does not merge my vectors correctly.
Real world implementation
My actual code is part of a game I'm writing to familiarise myself with Clojure (ClojureScript in this instance).
The game state (world) is as follows:
[{:id :player,
:attrs
{:gravity {:weight 10},
:jump {:height 60, :falling false, :ground true},
:renderable {:width 37, :height 35},
:position {:x 60, :y 565},
:walk {:step 4, :facing :right},
:input {},
:solid {:blocked false}}}
{:id wall1,
:attrs
{:solid {},
:position {:x 0, :y 0},
:renderable {:width 20, :height 600}}}
{:id wall2,
:attrs
{:solid {},
:position {:x 780, :y 0},
:renderable {:width 20, :height 600}}}
{:id platform3,
:attrs
{:solid {},
:position {:x 20, :y 430},
:renderable {:width 600, :height 20}}}]
I update the world on each animation frame, then re-render the canvas.
(defn game-loop []
(ui/request-frame game-loop)
(-> world
system/update!
ui/render))
system/update! is:
(defn update! [world]
(let [new-world (->> #world
(phys/move #player/input-cmd))]
(player/clear-cmd)
(reset! world new-world)
#world))
My plan was to have a chain of systems updating the world in the thread last macro. I am in the process of writing phys/move.
This means that systems have to update the world, instead of just returning their effect on the world (vector of changes).
I'm contemplating if it's more manageable to have the system/update! function manage the effects on the world (applying the updates). So systems only return their list of changes. Something like:
(defn update! [world]
(let [updates []
game #world]
(conj updates (phys/move #player/input-cmd game))
(conj updates (camera/track :player game))
; etc.
(reset! world
(map #(update-world updates %) world))
#world))
Repeated (conj updates (func world)) feels clunky though. This is possibly more work than just merging updates (or returning an modified entity) in a system function.
How to elegantly pass state changes between my system functions (phys/move, camera/track), and subsystem functions (walk, jump)?
My stab at applying Joaquin's map only approach to my move system:
(ns game.phys)
(def step 4)
(def height 50)
(defn walk?
"is this a walk command?"
[cmd]
(#{:left :right} cmd))
(defn walks?
"update? equivalent"
[entity]
(contains? (:attrs entity) :position))
(defn walk
[cmd entity]
(if (walks? entity)
(let [x (get-in entity [:attrs :position :x]
op (if (= cmd :left) - +)
update {:attrs {:position {:x (op x step)}
:walk {:facing cmd}}}]
(merge entity update))
entity))
; cmd is a deref'ed atom that holds player input commands
; e.g. :left :right: :jump
(defn move
[cmd world]
(cond
(walk? cmd) (map #(walk input-cmd %) world)
(jump? cmd) (map #(jump height %) world)
:else world))
It is simpler than all that (having a vector of maps is something pretty common, and it may fit your problem properly). Instead of doing a filter and then a map, just do a map:
(defn update? [entity] ...)
(defn update-entity [entity] ...)
(defn update-world [world]
(let [update #(if (matches? %) (update-entity %) %)]
(map update world)))
This pattern of updating something or just leaving it as it is is pretty common, and it is idiomatic to make the functions that update something return the updated thing, or the old thing if it did nothing, so at the end it would be something like:
(defn update-entity?
"Predicate that returns if an entity needs updating"
[entity] ...)
(defn update-entity
"Updates an entity if it is needed.
Otherwise returns the unchanged entity"
[entity]
(if (update-entity? entity)
(assoc entity :something :changed)
entity))
(defn update-world [world]
(map update-entity world))
You can see some examples of this behavior in the game logic of this snake game:
https://github.com/joakin/cnake/blob/master/src/cnake/game.cljs#L54-L66
https://github.com/joakin/cnake/blob/master/src/cnake/game.cljs#L102-L114