I'm trying to create a stateful transducer, join-averages (gist here).
Running the let block shows me that I'm correctly joining values. But the result output still doesn't have the joined value.
...
c' {:tick-list {:uuid 1, :last-trade-price 11.1}, :ema-list {:uuid 1, :last-trade-price-exponential 10}, :sma-list {:uuid 1, :last-trade-price-average 10.1}}
record {:uuid 1, :last-trade-price 11.1}
c' {:tick-list {:uuid 2, :last-trade-price 11.2}, :ema-list {:uuid 2, :last-trade-price-exponential 11}, :sma-list {:uuid 2, :last-trade-price-average 10.2}}
record {:uuid 2, :last-trade-price 11.2}
...
c' {:tick-list {:uuid 3, :last-trade-price 11.3}, :ema-list {:uuid 3, :last-trade-price-exponential 12}, :sma-list {:uuid 3, :last-trade-price-average 10.3}}
record {:uuid 1, :last-trade-price-exponential 10}
record {:uuid 2, :last-trade-price-exponential 11}
record {:uuid 3, :last-trade-price 11.3}
record {:uuid 3, :last-trade-price-exponential 12}
Any ideas as to why the join-averages stateful transducer, not returning the correct result?
CODE
(:require [clojure.core.async :refer [chan sliding-buffer <! go-loop pipeline onto-chan] :as async]
[clojure.set :refer [subset?]]
[clojure.tools.logging :as log])
(defn has-all-lists? [averages-map]
(subset? #{:tick-list :sma-list :ema-list} (->> averages-map keys (into #{}))))
(defn join-averages []
(let [state (atom {})]
(fn [rf]
(fn
([] (rf))
([accumulator] (rf accumulator))
([accumulator input]
(let [uuid (:uuid input)
entry (cond
(:last-trade-price-exponential input) {:ema-list input}
(:last-trade-price-average input) {:sma-list input}
(:last-trade-price input) {:tick-list input})]
(if-let [current (get #state uuid)]
(let [_ (swap! state update-in [uuid] merge entry)
c' (get #state uuid)]
(log/info "c'" c')
(log/info "accumulator" accumulator)
(log/info "state" (with-out-str (clojure.pprint/pprint #state)))
(if (has-all-lists? c')
c'
(rf accumulator input)))
(do (swap! state merge {uuid entry})
(rf accumulator input)))))))))
(comment
(let [ema-list [{:uuid "1" :last-trade-price-exponential 10}
{:uuid "2" :last-trade-price-exponential 11}
{:uuid "3" :last-trade-price-exponential 12}]
sma-list [{:uuid "1" :last-trade-price-average 10.1}
{:uuid "2" :last-trade-price-average 10.2}
{:uuid "3" :last-trade-price-average 10.3}]
tick-list [{:uuid "1" :last-trade-price 11.1}
{:uuid "2" :last-trade-price 11.2}
{:uuid "3" :last-trade-price 11.3}]
ec (chan (sliding-buffer 100))
sc (chan (sliding-buffer 100))
tc (chan (sliding-buffer 100))
_ (onto-chan ec ema-list)
_ (onto-chan sc sma-list)
_ (onto-chan tc tick-list)
merged-ch (async/merge [tc sc ec])
output-ch (chan (sliding-buffer 100) (join-averages))]
(async/pipeline 1 output-ch (join-averages) merged-ch)
(go-loop [r (<! output-ch)]
(when-not (nil? r)
(log/info "record" r)
(recur (<! output-ch))))))
You didn't give us what your result should look like so I'll have to guess.
Also, to see how to implement a stateful transducer I usually just look at distinct. You should init your state after the transducer was initialized. This should work:
(defn has-all-lists? [averages-map]
(set/subset? #{:tick-list :sma-list :ema-list} (->> averages-map keys (into #{}))))
(defn join-averages []
(fn [rf]
(let [state (atom {})]
(fn
([] (rf))
([accumulator] (rf accumulator))
([accumulator input]
(let [uuid (:uuid input)
entry (condp #(%1 %2) input
:last-trade-price-exponential {:ema-list input}
:last-trade-price-average {:sma-list input}
:last-trade-price {:tick-list input})]
(let [nv (swap! state update-in [uuid] merge entry)
c' (get nv uuid)]
(if (has-all-lists? c')
(rf accumulator c')
accumulator))))))))
(let [ema-list [{:uuid "1" :last-trade-price-exponential 10}
{:uuid "2" :last-trade-price-exponential 11}
{:uuid "3" :last-trade-price-exponential 12}]
sma-list [{:uuid "1" :last-trade-price-average 10.1}
{:uuid "2" :last-trade-price-average 10.2}
{:uuid "3" :last-trade-price-average 10.3}]
tick-list [{:uuid "1" :last-trade-price 11.1}
{:uuid "2" :last-trade-price 11.2}
{:uuid "3" :last-trade-price 11.3}]]
(into []
(join-averages)
(concat ema-list sma-list tick-list)))
Related
I'm kind of newbie with Clojure, all is new but pretty fun too. So I have this data:
{:test {:title "Some Title"}, :questions [
{:id 1, :full-question {:question "Foo question", :id 1, :answers [{:id 7, :question_id 1, :answer "Foobar answer"}, {:id 8, :question_id 1, :answer "Foobar answer two"}]}},
{:id 5, :full-question {:question "Foo question", :id 5, :answers [{:id 12, :question_id 5, :answer "Foobar answer"}]}},
{:id 9, :full-question {:question "Foo question", :id 9, :answers [{:id 14, :question_id 9, :answer "Foobar answer"}, {:id 20, :question_id 9, :answer "Foobar answer two"}]}}
]}
A "classic" Test->Question->Answer kind of data structure. And I have this new info:
(def new-answer {:id 33, :answer "Another foobar answer", :question-id 9 })
I need to update the first structure to add "new-answer" into the "answers" for the :id number 9 in the :questions vector.
I tried with the update-in function but I don't know what to tell the correspondent :id in the maps inside the two vectors. I mean, I don't know how to build the "path" where I want to make the change.
also, there is a nice library for that kind of structural editing, called specter
your case could be solved like this:
(require '[com.rpl.specter :refer [ALL AFTER-ELEM setval]])
(defn add-answer [data {question-id :question-id :as new-answer}]
(setval [:questions ALL #(== question-id (:id %)) :full-question :answers AFTER-ELEM]
new-answer data))
user> (add-answer data {:id 33, :answer "Another foobar answer", :question-id 9 })
;;=> {:test {:title "Some Title"},
;; :questions
;; [
;; ;; ... all other ids
;; {:id 9,
;; :full-question
;; {:question "Foo question",
;; :id 9,
;; :answers
;; [{:id 14, :question_id 9, :answer "Foobar answer"}
;; {:id 20, :question_id 9, :answer "Foobar answer two"}
;; {:id 33, :answer "Another foobar answer", :question-id 9}]}}]}
You have the right idea with update-in. You can first calculate the index in your questions vector, then create the path :questions, "question-index", :full-question, :answers. Then you may conj in your new answer:
(def data {...})
(defn index-by-id
[v id]
(first (filter #(= (:id (v %)) id) (range (count v)))))
(defn add-answer
[answer]
(let [q-index (index-by-id (:questions data) (:question-id answer))]
(update-in data [:questions q-index :full-question :answers]
conj answer)))
Using clojure, you have the update-in function found here and the assoc found here. You can use the code suggested by Alex for update-in. assoc is fairly similar,
(defn change [ma a-map id]
(assoc (:questions ma)
(if-let [xq (first (filter int? (map-indexed (fn [idx mp] (if (= (:id mp) id) idx nil)) (:questions ma))))]
xq
(inc (count ma)))
a-map))
You can update your map as
(change o-map n-map idx) ;;param-1 is map to change,
;;param-2 is new-answer,
;;idx is the :id to change.
You can also refer to assoc-in found here which also associates a value in a nested associative structure.
Hope this helps.
How i can count mobile and web access discarding a nil values from a list of maps? the output should be anything like this " Statistic mobile = 1 web = 2", but all is imutable on other languagens a simple i++ resolve but how is in clojure. thanks.
def data [{:name "app1" :type "mobile" }
{:name "site1" :type "web" }
{:name "site1" :type "web" }
{:name "boot" :type nil }]
(frequencies (map :type data))
gives
{"mobile" 1, "web" 2, nil 1}
user=> (for [[k v] (group-by :type data) :when k] [k (count v)])
(["mobile" 1] ["web" 2])
Please look at the following code:
(def data {:color ["R", "B", "G"] :name "Hello" :up "down"})
(defn collapse-vector-kvp [k v]
(map #(hash-map k %) v))
(defn collapse-map [m]
(map #(let
[x %]
(if (vector? (val x))
(collapse-vector-kvp (key x) (val x))
(hash-map (key x) (val x))
)) m))
(collapse-map data)
=> ({:name "Hello"} ({:color "R"} {:color "B"} {:color "G"}) {:up "down"})
What I would like to do is create a single list, rather than have the 'color' entries be in a list within the list. Is this easily achievable?
user=> (def data2 '({:name "Hello"} ({:color "R"} {:color "B"} {:color "G"}) {:up "down"}))
#'user/data2
user=> (flatten data2)
({:name "Hello"} {:color "R"} {:color "B"} {:color "G"} {:up "down"})
Another version of collapse-map:
(defn collapse-map [m]
(let [sep-m (group-by (comp vector? val) m)]
(concat (map (fn [[k v]] {k v})
(sep-m false))
(apply concat (map (fn [[k v]]
(collapse-vector-kvp k v))
(sep-m true))))))
(def test-data {:color ["R" "B" "G"]
:name "Hello"
:k ["v1" "v2" "v3"]
:up "down"})
(collapse-map test-data)
=> ({:name "Hello"}
{:up "down"}
{:color "R"}
{:color "B"}
{:color "G"}
{:k "v1"}
{:k "v2"}
{:k "v3"})
I need to merge a collection of arrays based on id.
Example data:
EDIT: (changed to match Clojure data structures)
[{:id 1, :region :NA, :name :Test1, :OS :W}
{:id 1, :region :EU, :name :Test2, :OS :W}
{:id 2, :region :AS, :name :test3, :OS :L}
{:id 2, :region :AS, :name :test4, :OS :M}]
Becomes:
EDIT: (changed to match Clojure data structures)
[{:id 1, :region [:NA :EU], :name [:Test1 :Test2] ,:OS [:W]}
{:id 2, :region [:AS] :name [:test3 :Test4], :OS [:L :M]}]
| is the delimiter (changeable)
If possible, also would like alphabetical order as well.
(def data
[{:id 1, :region :NA, :name :Test1, :OS :W}
{:id 1, :region :EU, :name :Test2, :OS :W}
{:id 2, :region :AS, :name :test3, :OS :L}
{:id 2, :region :AS, :name :test4, :OS :M}])
(defn key-join
"join of map by key , value is distinct."
[map-list]
(let [keys (keys (first map-list))]
(into {} (for [k keys] [k (vec (set (map #(% k) map-list)))]))))
(defn group-reduce [key map-list]
(let [gdata (group-by key map-list)]
(into [] (for [[k m] gdata] (let [m (key-join m)](assoc m key ((key m) 0)))))))
user=> (group-reduce :id data)
[{:name [:Test2 :Test1], :OS [:W], :region [:EU :NA], :id 1} {:name [:test3 :test4], :OS [:L :M], :region [:AS], :id 2}]
You can use some combination of functions from clojure.set (if you change the outermost vector to set). Specifically clojure.set/index looks promising.
You can use the merge-with function as shown below in the example.
Firstly, we define some helper functions
(defn collect [& xs]
(apply vector (-> xs distinct sort)))
The collect function makes sure that the items in xs are unique and sorted and finally returns them in a vector.
(defn merge-keys [k xs]
(map #(apply merge-with collect %) (vals (group-by k xs))))
merge-keys first groups the hash-maps in xs by a primary key (in your case :id), takes each list of grouped items and merges the values of the keys using the collect function from above.
(def xs [{:id 1, :region :NA, :name :Test1, :OS :W}
{:id 1, :region :EU, :name :Test2, :OS :W}
{:id 2, :region :AS, :name :test3, :OS :L}
{:id 2, :region :AS, :name :test4, :OS :M}])
(merge-keys :id xs)
=> ({:id [1],
:region [:EU :NA],
:name [:Test1 :Test2],
:OS [:W]}
{:id [2],
:region [:AS],
:name [:test3 :test4],
:OS [:L :M]})
Note however that even the :id key now has vector associated with it. You can easily un-vector it by either introducing an if statement in collect which associates a single value with the key instead of a vector...
(defn collect [& xs]
(let [cs (apply vector (-> xs distinct sort))]
(if (= 1 (count cs)) (first cs) cs)))
...or take the result from merge-keys and do
(map #(update-in % [:id] first) result)
which will only un-vector the :id map entry
What would be a best way to impose a condition on the nested fields of complex nested structure like...
{
:aa {:a "a_val",:b "b_val"},
:qq {:abc
{
:x1 {:x "abc",:u "ee"},
:x2 {:y "abc",:i "ee"},
:x3 {:x "abc",:i "ee"}
}
},
:ww {:xyz {
:y1 {:x "abc",:u "ee"},
:y2 {:y "abc",:i "0"},
:y3 {:x "abc",:i "ee"}
}
}
}
I want to check whether the "i" part exist and has value "0" in each of aa,qq and ww and depending upon that exclude(or perform any operation) on aa,qq and ww. For example if "ww" has "i"="0" at that position then get a map like below
{
:ww {:xyz {
:y1 {:x "abc",:u "ee"},
:y2 {:y "abc",:i "0"},
:y3 {:x "abc",:i "ee"}
}
}
}
user> (defn vvals [m] (when (map? m) (vals m)))
'user/vvals
user> (filter #(some #{"0"} (for [v (vvals (val %)), v (vvals v)] (:i v))) xx)
([:ww {:xyz {:y3 {:x "abc", :i "ee"}, :y2 {:y "abc", :i "0"}, :y1 {:x "abc", :u "ee"}}}])