If i have 10 buffered subscription channels each with a transducer that takes 5sec to execute. Are the transducers going to be executed concurrent or all on the same thread (assuming multithreaded context)?
it is quite easy to check, logging the thread, the transducer is being executed by:
(def log-chan (chan))
(go-loop []
(println (<! log-chan))
(recur))
(def channels
(repeatedly
10
#(chan 10
(map (fn [item]
(let [thread (Thread/currentThread)]
(go (>! log-chan thread)))
item)))))
(doseq [c channels]
(go (>! c :item)))
output:
#object[java.lang.Thread 0x77a39fa0 Thread[async-dispatch-27,5,main]]
#object[java.lang.Thread 0x7d5bf4d0 Thread[async-dispatch-31,5,main]]
#object[java.lang.Thread 0x53ecb32b Thread[async-dispatch-29,5,main]]
#object[java.lang.Thread 0x2b74f3ac Thread[async-dispatch-25,5,main]]
#object[java.lang.Thread 0x6eb50f9e Thread[async-dispatch-26,5,main]]
#object[java.lang.Thread 0x30701edb Thread[async-dispatch-30,5,main]]
#object[java.lang.Thread 0x1a370b69 Thread[async-dispatch-36,5,main]]
#object[java.lang.Thread 0x3d9884a2 Thread[async-dispatch-24,5,main]]
#object[java.lang.Thread 0x208941d0 Thread[async-dispatch-23,5,main]]
#object[java.lang.Thread 0x2c77aeb Thread[async-dispatch-28,5,main]]
so you can see, there are different threads.
But it really depends on a go block, inside which you send data to your channel, so if you change this:
(doseq [c channels]
(go (>! c :item)))
to this:
(go (doseq [c channels]
(>! c :item)))
you get this output:
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
#object[java.lang.Thread 0x16856295 Thread[async-dispatch-6,5,main]]
so all channels' transducers would use the same thread
They will be executed in parallel, so on different real threads at the same time. Your machine might not have 10 cores however, so there will be some parking and blocking. You can look up the technical details here in the section 'Parking and Blocking'.
Related
How go macro transforms the code
(let [c1 (chan)
c2 (chan)]
(go (while true
(let [[v ch] (alts! [c1 c2])]
(println "Read" v "from" ch))))
(go (>! c1 "hi"))
(go (>! c2 "there")))
It is a state machine at the back. You can read more in The State Machines of core.async
I have a map as seen bellow. I am getting the information from a datomic database. Now I want to transform the data structure here:
(def my-map [{:db/id #object[Object 56536887900242005],
:height 630,
:distance 1474.1,
:coordinates [-26.65622109697031 30.48401767312403],
:location #:location{:id 1}}
{:db/id #object[Object 56536887900242006],
:height 22075,
:distance 1503.2,
:coordinates [-26.65622109697031 30.48401767312403],
:location #:location{:id 2}}
{:db/id #object[Object 56536887900242007],
:height 24248,
:distance 1695.6,
:coordinates [-26.662030943549 30.25648873549992],
:location #:location{:id 3}})
to look like this
{1 {:height 630, :distance 1474.1,:coordinates [-26.65622109697031 30.48401767312403]}
2 {:height 22075, :distance 1503.2,:coordinates [-26.65622109697031 30.48401767312403]}
3 {:height 24248, :distance 1695.6,:coordinates [-26.65622109697031 30.48401767312403]}}
I want to pull the 1 from #:location{:id 1} which I will then assoc with
{:height 22075, :distance 1503.2,:coordinates [-26.65622109697031 30.48401767312403]}
Bellow I have code that return the above, but I do not know how to assoc it into the :id and I also do not know how to get the id seeing that the data has a #
(map #(dissoc % :db/id :location ) my-map)
I use plumbing.core in every project anyway, and if you do as well, then this solution might appeal to you.
(grouped-map
(fn-> :location :location/id)
(fn-> (dissoc :location :db/id))
my-map)
You can write like this:
(into {}
(map
#(hash-map
(get-in % [:location :location/id])
(dissoc % :db/id :location))
my-map))
Sometimes using for can help make the structure of your data more apparent:
(->> (for [record my-map]
[(-> record :location :location/id)
(dissoc record :db/id :location)])
(into {}))
The following function will do the trick:
(defn transform [coll]
(->> coll
(map #(hash-map
(-> % :location :location/id)
(dissoc % :db/id :location)))
(into {})))
I am new to clojure and am trying to refactor some code that I've written that looks like this
(defn transform
[entity]
(let [new-obj (doto (SomeObj.)
(.setField1 (:field-1 entity)))
new-obj))
I have many objects that will need this implemented but would like to create a macro that accepts an entity, prototype Ex: (SomeObj.), and a map where the keys are the field names of the prototype and the values are vectors of keywords to get the correct field from the entity. For each key I would need to call .set + keyName using the argument from (get-in map [value as a vector]).
My hope is for each new entity I can create a config of the mappings and only write code for special cases. Is this possible using a macro?
Or is there a more idiomatic way to do this in clojure.
yes, you could easily do it with a macro like this:
(defmacro map-to [type mappings entity]
`(doto (new ~type)
~#(map (fn [[field entity-field]]
`(~(symbol (str ".set" (clojure.string/capitalize field)))
(~entity-field ~entity)))
mappings)))
this would generate exactly the code you need:
(map-to java.util.Date {date :dt minutes :mm hours :h}
{:dt 10 :mm 22 :h 12})
would be expanded into the following:
(doto
(new java.util.Date)
(.setDate (:dt {:dt 10, :mm 22, :h 12}))
(.setMinutes (:mm {:dt 10, :mm 22, :h 12}))
(.setHours (:h {:dt 10, :mm 22, :h 12})))
a few things to notice here:
1) you don't need to introduce a new variable new-obj, since doto returns the object being operated on.
2) your mappings should be passed as a literal map, because otherwise you cannot get the keys to pass to . special form.
3) you can see that the entity map is being repeated. You can fix this by introducing another binding inside the macro:
(defmacro map-to [type mappings entity]
(let [ent (gensym "entity")]
`(let [~ent ~entity]
(doto (new ~type)
~#(map (fn [[field entity-field]]
`(~(symbol (str ".set" (clojure.string/capitalize field)))
(~entity-field ~ent)))
mappings)))))
so now it expands like this:
(let [entity20047 {:dt 10, :mm 22, :h 12}]
(doto
(new java.util.Date)
(.setDate (:dt entity20047))
(.setMinutes (:mm entity20047))
(.setHours (:h entity20047))))
in repl:
user> (map-to java.util.Date {date :dt minutes :mm hours :h}
{:dt 10 :mm 22 :h 12})
;;=> #inst "2016-09-10T09:22:48.867-00:00"
user> (let [ent {:dt 10 :mm 22 :h 12}]
(map-to java.util.Date {date :dt minutes :mm hours :h} ent))
;;=> #inst "2016-09-10T09:22:48.899-00:00"
(the value is three hours earlier due to my time zone (gmt+3))
update
to get your desired behaviour (with get-in) you can just slightly modify this macro:
(defmacro map-to [type mappings entity]
(let [ent (gensym "entity")]
`(let [~ent ~entity]
(doto (new ~type)
~#(map (fn [[field entity-field]]
`(~(symbol (str ".set" (clojure.string/capitalize field)))
(get-in ~ent ~entity-field)))
mappings)))))
in repl:
user> (map-to java.util.Date {date [:date :dt]
minutes [:time :mm]
hours [:time :h]}
{:date {:dt 10} :time {:mm 22 :h 12}})
;;=> #inst "2016-09-10T09:22:41.935-00:00"
expands to:
(let [entity20094 {:date {:dt 10}, :time {:mm 22, :h 12}}]
(doto
(new java.util.Date)
(.setDate (get-in entity20094 [:date :dt]))
(.setMinutes (get-in entity20094 [:time :mm]))
(.setHours (get-in entity20094 [:time :h]))))
now you can make one more macro to automate the creation of mapping functions:
first of all you need a function to produce maker name from class object:
(defn make-name [c]
(->> c
.getName
(#(clojure.string/split % #"\."))
(clojure.string/join "-")
(str "create-")
symbol))
user> (make-name java.util.Date)
;;=> create-java-util-Date
now the macro to define functions to create instances from entities:
(defmacro defmapper [type mappings]
`(defn ~(make-name type) [entity#]
(map-to ~type ~mappings entity#)))
this one would create functions, that are, given an entity, convert it to the class instance. It's just an ordinary function:
(defmapper java.util.Date {date [:date :dt]
minutes [:time :mm]
hours [:time :h]})
expands to:
(defn create-java-util-Date [entity__20122__auto__]
(map-to
java.util.Date
{date [:date :dt], minutes [:time :mm], hours [:time :h]}
entity__20122__auto__))
in repl:
user> (map create-java-util-Date
[{:date {:dt 10} :time {:mm 22 :h 12}}
{:date {:dt 11} :time {:mm 22 :h 12}}
{:date {:dt 12} :time {:mm 22 :h 12}}])
;;(#inst "2016-09-10T09:22:18.974-00:00"
;; #inst "2016-09-11T09:22:18.974-00:00"
;; #inst "2016-09-12T09:22:18.974-00:00")
How to evaluate a function correctly every minute using at-at and chime?
Here are my tests:
(require '[overtone.at-at :refer :all]
'[chime :refer [chime-at]]
'[clj-time.periodic :refer [periodic-seq]]
'[clj-time.core :as t])
;; 1. Use of future
(defonce data1 (atom {:num 1}))
(defonce updater
(future
(while true
(swap! data1 update-in [:num] inc)
(Thread/sleep 60000))))
;; 2. Using at-at
(defonce data2 (atom {:num 1}))
(def my-pool (mk-pool))
(every 60000 #(swap! data2 update-in [:num] inc) my-pool)
;; 3. Using chime
(defonce data3 (atom {:num 1}))
(chime-at (periodic-seq (t/now) (-> 60 t/seconds))
(fn [] (swap! data3 update-in [:num] inc))
{:error-handler (fn [e] (str e))})
After 5 minutes:
#data1
;;=> {:num 5}
#data2
;;=> {:num 8}
#data3
;;=> {:num 1}
Why is at-at counting to fast?
Why is chimenot counting at all?
Thank you!
Not sure what's up with at-at.
As for Chime, chime-at calls the callback function with the time of the current chime, so you'll need to amend your callback to something like
(fn [time] (swap! data3 update-in [:num] inc))
With (fn [] …) you'll get an ArityException at each chime and your :error-handler swallows those. (Chime's default handler prints a stack trace; NB. depending on your setup that stack trace may or may not be visible in your REPL window – for example with a fairly typical Emacs/CIDER setup you might have to switch to an *nrepl-server* buffer to see it.)
(Incidentally, in 1.7 alphas you can use update :num instead of update-in [:num].)
This works as expected:
java -jar clojure-1.4.0.jar -e "(do (require 'clojure.repl) (.setDynamic #'clojure.repl/print-doc) (with-bindings {#'clojure.repl/print-doc str} (eval '(clojure.repl/doc println))))"
Output:
"{:ns #<Namespace clojure.core>, :name println, :arglists ([& more]), :added \"1.0\", :static true, :doc \"Same as print followed by (newline)\", :line 3325, :file \"clojure/core.clj\"}"
But the same does not work in the REPL:
java -jar clojure-1.4.0.jar -e "(do (require 'clojure.repl) (.setDynamic #'clojure.repl/print-doc) (clojure.main/repl :init (fn [] {#'clojure.repl/print-doc str}))))"
Output of (doc println):
user=> (doc println)
-------------------------
clojure.core/println
([& more])
Same as print followed by (newline)
nil
user=>
I don't know what I'm doing wrong.
Found the answer after diving into the counterclockwise and nrepl code:
java -jar clojure-1.4.0.jar -e "(do (require 'clojure.repl) (.setDynamic #'clojure.repl/print-doc) (with-bindings {#'clojure.repl/print-doc str} (clojure.main/repl)))))"
The output is the same as above:
"{:ns #<Namespace clojure.core>, :name println, :arglists ([& more]), :added \"1.0\", :static true, :doc \"Same as print followed by (newline)\", :line 3325, :file \"clojure/core.clj\"}"
The trick is to use with-bindings before calling repl:
(with-bindings {#'clojure.repl/print-doc str}
(repl))