I want to calculate the rate of events, for example, key press events, and display it. I also want to start rate calculation only after first event happened. Another requirement is that rate calculation should stop after certain amount of key press events. My current approach to that is
(defn keypress-chan
"Returns a channel that receives keys"
[]
(let [out (chan) handler (fn [e] (put! out (.-key e)))]
(js/document.addEventListener "keypress" handler false)
out))
(defn tick
"Pushes to `out` 1000 ms later"
[out]
(go
(<! (timeout 1000))
(>! out 1)))
(defn get-rate
"Calculates keypress per minute rate"
[ticks pressed]
(quot (* pressed 60) ticks))
(defn complete?
[count]
(>= count 100))
(defn main-loop []
(let [key-chan (keypress-chan) ticker-chan (chan)]
(go-loop [pressed 0 ticks 0]
(let [[value ch] (alts! [key-chan ticker-chan])]
(if (= ch key-chan)
(do
;; Start timer to calculate keypress rate
(if (zero? pressed)
(tick ticker-chan))
;; Do something with `value`, i. e. the key being pressed
(when-not (complete?)
(recur (inc pressed) ticks)))
(let [speed (get-rate ticks pressed)]
(swap! app-state assoc :rate rate)
(tick ticker-chan)
(recur pressed (inc ticks))))))))
So, this works, but the code in main-loop seems ugly to me. I've been thinking of having a "trigger" that fires only when first value appears in the key-chan and starts the timer. Then another trigger stops the timer when necessary. But I'm new to Clojure and don't know well how to work with async, so I couldn't create anything better than code above.
How this loop could be improved?
Can I move timer to a function and start the timer by the first value in the key-chan?
How can I stop that timer later?
I came up with the following solution.
(ns ratecalc.core
(:require [cljs.core.async :refer [chan <! go-loop]])
(:import [goog.date DateTime]))
(defn calc-rate [input-chan do-fun max-count]
(go-loop [count 0
start-time nil]
(<! input-chan)
(let [inc-count (inc count)
now (.getTime (DateTime.))
start-time* (if-not start-time now start-time)
rate-per-ms (/ inc-count (- now start-time*))]
(do-fun rate-per-ms)
(if (= count max-count)
rate-per-ms
(recur inc-count start-time*)))))
The calc-rate function calculates the number of messages on the channel per millisecond. You can pass it a function to do some side effect like printing the current average. The last argument limits it calculation to a maximum number of messages. Here is an example of how to use it:
(calc-rate (keypress-chan) println 10)
Related
I have a channel where I am putting values into inside a doseq loop.
This code reads from a list of isbns and for each isbn, does an amazon search to return contents of a book, and then calls another function to get the title and rank
(def book_channel (chan 10))
make sure you use clojure.core.async/into rather than clojure.core/into. Here is an example of a round trip from collection to channel and back to collection:
user> (require '[clojure.core.async :as async :refer [<! <!! >!! >! chan go]])
nil
user> (def book-chan (async/to-chan [:book1 :book2 :book3]))
#'user/book-chan
user> (<!! (clojure.core.async/into [] book-chan))
[:book1 :book2 :book3]
clojure.core.async/into returns a channel that will have exactly one item written to it. That one item will be written once it's input channel closes. This keeps the whole thing asynchronous and it does require that the code putting things into the book-channel close the chan to signal that all the books are there.
You need to do some type of coordination to determine when all of your work is finished. You can pull that coordination out into the main thread fairly easily:
(def book_channel (chan 10))
(defn concurrency_test
[list_of_isbns]
(doseq [isbn list_of_isbns]
(go (>! book_channel
(get_title_and_rank_for_one_isbn
(amazon_search isbn)))))
(prn (loop [results []]
(if (= (count results) (count list_of_isbns))
results
(recur (conj results (<!! book_channel)))))))
Here, I used a loop that keeps waiting for results and adding them to the vector until we have as many results as we do isbns. You'll want to make sure that get_title_and_rank_for_one_isbn always generates a result that can be put on a channel, otherwise the loop will wait forever.
You should close! the book_channel after you finish pushing stuff into it. Per async/into documentation - "ch must close before into produces a result."
(let [book> (chan)]
(go
(doseq [e (range 8)]
(>! book> e))
(close! book>))
(<!! (async/into [] book>)))
Alternatively, you can use async/onto-chan which will close the channel for you:
(let [book> (chan)]
(async/onto-chan book> (range 8))
(<!! (async/into [] book>)))
Is there an elegant way to stop a running go block?
(without introducing a flag and polluting the code with checks/branches)
(ns example
(:require-macros [cljs.core.async.macros :refer [go]])
(:require [cljs.core.async :refer [<! timeout]]))
(defn some-long-task []
(go
(println "entering")
; some complex long-running task (e.g. fetching something via network)
(<! (timeout 1000))
(<! (timeout 1000))
(<! (timeout 1000))
(<! (timeout 1000))
(println "leaving")))
; run the task
(def task (some-long-task))
; later, realize we no longer need the result and want to cancel it
; (stop! task)
Sorry, this is not possible with core.async today. What you get back from creating a go block is a normal channel what the result of the block will be put on, though this does not give you any handle to the actual block itself.
As stated in Arthur's answer, you cannot terminate a go block immediately, but you since your example indicates a multi-phased task (using sub-tasks), an approach like this might work:
(defn task-processor
"Takes an initial state value and number of tasks (fns). Puts tasks
on a work queue channel and then executes them in a go-loop, with
each task passed the current state. A task's return value is used as
input for next task. When all tasks are processed or queue has been
closed, places current result/state onto a result channel. To allow
nil values, result is wrapped in a map:
{:value state :complete? true/false}
This fn returns a map of {:queue queue-chan :result result-chan}"
[init & tasks]
(assert (pos? (count tasks)))
(let [queue (chan)
result (chan)]
(async/onto-chan queue tasks)
(go-loop [state init, i 0]
(if-let [task (<! queue)]
(recur (task state) (inc i))
(do (prn "task queue finished/terminated")
(>! result {:value state :complete? (== i (count tasks))}))))
{:queue queue
:result result}))
(defn dummy-task [x] (prn :task x) (Thread/sleep 1000) (inc x))
;; kick of tasks
(def proc (apply task-processor 0 (repeat 100 dummy-task)))
;; result handler
(go
(let [res (<! (:result proc))]
(prn :final-result res)))
;; to stop the queue after current task is complete
;; in this example it might take up to an additional second
;; for the terminated result to be delivered
(close! (:queue proc))
You may want to use future and future-cancel for such task.
(def f (future (while (not (Thread/interrupted)) (your-function ... ))))
(future-cancel f)
Why do cancelled Clojure futures continue using CPU?
I'm using core.async to do something in parallel, and then using alts!! wait on certain amount of result with timeout.
(ns c
(:require [clojure.core.async :as a]))
(defn async-call-on-vector [v]
(mapv (fn [n]
(a/go (a/<! (a/timeout n)) ; simulate long time work
n))
v))
(defn wait-result-with-timeout [chans num-to-get timeout]
(let [chans-count (count chans)
num-to-get (min num-to-get
chans-count)]
(if (empty? chans)
[]
(let [timeout (a/timeout timeout)]
(loop [result []
met 0]
(if (or (= (count result) num-to-get)
(= met chans-count)) ; all chan has been consumed
result
(let [[v c] (a/alts!! (conj chans timeout))]
(if (= c timeout)
result
(case v
nil (do (println "got nil") (recur result met)) ; close! on that channel
(recur (conj result v) (inc met)))))))))))
and then invoke like:
user=> (-> [1 200 300 400 500] c/async-call-on-vector (c/wait-result-with-timeout 2 30))
this expression will prints out a lot of got nil. It seems channel returned by go block will close that channel after result has been returned. And this will causes alts!! return nil on this case. but this is very CPU unfriendly, it just like busy waiting. Is there a way to avoid this?
I solved this by define a macro like go, but return a channel that will not closed on result returned. Is this a right way to solve it?
I'm using core.async to do something in parallel, and then using alts!! wait on certain amount of result with timeout.
It looks like you want to collect all of the values that will be delivered by some channels, until all of those channels are closed, or until a timeout occurs. One way to do that is to merge those channels onto a single channel, and then use alts! within a go-loop to collect the values into a vector:
(defn wait-result-with-timeout [chans timeout]
(let [all-chans (a/merge chans)
t-out (a/timeout timeout)]
(a/go-loop [vs []]
(let [[v _] (a/alts! [all-chans t-out])]
;; v will be nil if either every channel in
;; `chans` is closed, or if `t-out` fires.
(if (nil? v)
vs
(recur (conj vs v)))))))
It seems channel returned by go block will close that channel after result has been returned.
You are correct, that is the documented behavior of a go block.
I solved this by define a macro like go, but return a channel that will not closed on result returned. Is this a right way to solve it?
Probably not, although it's not for me to say whether it's right or wrong for your particular use case. Generally speaking, channels should close if they are done delivering values, to indicate the semantics of being done delivering values. For example, the above code uses the closing of all-chans to indicate that there is no more work to wait on.
What's best way in clojure to implement something like an actor or agent (asynchronously updated, uncoordinated reference) that does the following?
gets sent messages/data
executes some function on that data to obtain new state; something like (fn [state new-msgs] ...)
continues to receive messages/data during that update
once done with that update, runs the same update function against all messages that have been sent in the interim
An agent doesn't seem quite right here. One must simultaneously send function and data to agents, which doesn't leave room for a function which operates on all data that has come in during the last update. The goal implicitly requires a decoupling of function and data.
The actor model seems generally better suited in that there is a decoupling of function and data. However, all actor frameworks I'm aware of seem to assume each message sent will be processed separately. It's not clear how one would turn this on it's head without adding extra machinery. I know Pulsar's actors accept a :lifecycle-handle function which can be used to make actors do "special tricks" but there isn't a lot of documentation around this so it's unclear whether the functionality would be helpful.
I do have a solution to this problem using agents, core.async channels, and watch functions, but it's a bit messy, and I'm hoping there is a better solution. I'll post it as a solution in case others find it helpful, but I'd like to see what other's come up with.
Here's the solution I came up with using agents, core.async channels, and watch functions. Again, it's a bit messy, but it does what I need it to for now. Here it is, in broad strokes:
(require '[clojure.core.async :as async :refer [>!! <!! >! <! chan go]])
; We'll call this thing a queued-agent
(defprotocol IQueuedAgent
(enqueue [this message])
(ping [this]))
(defrecord QueuedAgent [agent queue]
IQueuedAgent
(enqueue [_ message]
(go (>! queue message)))
(ping [_]
(send agent identity)))
; Need a function for draining a core async channel of all messages
(defn drain! [c]
(let [cc (chan 1)]
(go (>! cc ::queue-empty))
(letfn
; This fn does all the hard work, but closes over cc to avoid reconstruction
[(drainer! [c]
(let [[v _] (<!! (go (async/alts! [c cc] :priority true)))]
(if (= v ::queue-empty)
(lazy-seq [])
(lazy-seq (cons v (drainer! c))))))]
(drainer! c))))
; Constructor function
(defn queued-agent [& {:keys [buffer update-fn init-fn error-handler-builder] :or {:buffer 100}}]
(let [q (chan buffer)
a (agent (if init-fn (init-fn) {}))
error-handler-fn (error-handler-builder q a)]
; Set up the queue, and watcher which runs the update function when there is new data
(add-watch
a
:update-conv
(fn [k r o n]
(let [queued (drain! q)]
(when-not (empty? queued)
(send a update-fn queued error-handler-fn)))))
(QueuedAgent. a q)))
; Now we can use these like this
(def a (queued-agent
:init-fn (fn [] {:some "initial value"})
:update-fn (fn [a queued-data error-handler-fn]
(println "Receiving data" queued-data)
; Simulate some work/load on data
(Thread/sleep 2000)
(println "Done with work; ready to queue more up!"))
; This is a little warty at the moment, but closing over the queue and agent lets you requeue work on
; failure so you can try again.
:error-handler-builder
(fn [q a] (println "do something with errors"))))
(defn -main []
(doseq [i (range 10)]
(enqueue a (str "data" i))
(Thread/sleep 500) ; simulate things happening
; This part stinks... have to manually let the queued agent know that we've queued some things up for it
(ping a)))
As you'll notice, having to ping the queued-agent here every time new data is added is pretty warty. It definitely feels like things are being twisted out of typical usage.
Agents are the inverse of what you want here - they are a value that gets sent updating functions. This easiest with a queue and a Thread. For convenience I am using future to construct the thread.
user> (def q (java.util.concurrent.LinkedBlockingDeque.))
#'user/q
user> (defn accumulate
[summary input]
(let [{vowels true consonents false}
(group-by #(contains? (set "aeiouAEIOU") %) input)]
(-> summary
(update-in [:vowels] + (count vowels))
(update-in [:consonents] + (count consonents)))))
#'user/accumulate
user> (def worker
(future (loop [summary {:vowels 0 :consonents 0} in-string (.take q)]
(if (not in-string)
summary
(recur (accumulate summary in-string)
(.take q))))))
#'user/worker
user> (.add q "hello")
true
user> (.add q "goodbye")
true
user> (.add q false)
true
user> #worker
{:vowels 5, :consonents 7}
I came up with something closer to an actor, inspired by Tim Baldridge's cast on actors (Episode 16). I think this addresses the problem much more cleanly.
(defmacro take-all! [c]
`(loop [acc# []]
(let [[v# ~c] (alts! [~c] :default nil)]
(if (not= ~c :default)
(recur (conj acc# v#))
acc#))))
(defn eager-actor [f]
(let [msgbox (chan 1024)]
(go (loop [f f]
(let [first-msg (<! msgbox) ; do this so we park efficiently, and only
; run when there are actually messages
msgs (take-all! msgbox)
msgs (concat [first-msg] msgs)]
(recur (f msgs)))))
msgbox))
(let [a (eager-actor (fn f [ms]
(Thread/sleep 1000) ; simulate work
(println "doing something with" ms)
f))]
(doseq [i (range 20)]
(Thread/sleep 300)
(put! a i)))
;; =>
;; doing something with (0)
;; doing something with (1 2 3)
;; doing something with (4 5 6)
;; doing something with (7 8 9 10)
;; doing something with (11 12 13)
I'm looking for a very simple way to call a function periodically in Clojure.
JavaScript's setInterval has the kind of API I'd like. If I reimagined it in Clojure, it'd look something like this:
(def job (set-interval my-callback 1000))
; some time later...
(clear-interval job)
For my purposes I don't mind if this creates a new thread, runs in a thread pool or something else. It's not critical that the timing is exact either. In fact, the period provided (in milliseconds) can just be a delay between the end of one call completing and the commencement of the next.
If you want very simple
(defn set-interval [callback ms]
(future (while true (do (Thread/sleep ms) (callback)))))
(def job (set-interval #(println "hello") 1000))
=>hello
hello
...
(future-cancel job)
=>true
Good-bye.
There's also quite a few scheduling libraries for Clojure:
(from simple to very advanced)
at-at
chime (core.async integration)
monotony
quartzite
Straight from the examples of the github homepage of at-at:
(use 'overtone.at-at)
(def my-pool (mk-pool))
(let [schedule (every 1000 #(println "I am cool!") my-pool)]
(do stuff while schedule runs)
(stop schedule))
Use (every 1000 #(println "I am cool!") my-pool :fixed-delay true) if you want a delay of a second between end of task and start of next, instead of between two starts.
This is how I would do the core.async version with stop channel.
(defn set-interval
[f time-in-ms]
(let [stop (chan)]
(go-loop []
(alt!
(timeout time-in-ms) (do (<! (thread (f)))
(recur))
stop :stop))
stop))
And the usage
(def job (set-interval #(println "Howdy") 2000))
; Howdy
; Howdy
(close! job)
The simplest approach would be to just have a loop in a separate thread.
(defn periodically
[f interval]
(doto (Thread.
#(try
(while (not (.isInterrupted (Thread/currentThread)))
(Thread/sleep interval)
(f))
(catch InterruptedException _)))
(.start)))
You can cancel execution using Thread.interrupt():
(def t (periodically #(println "Hello!") 1000))
;; prints "Hello!" every second
(.interrupt t)
You could even just use future to wrap the loop and future-cancel to stop it.
I took a stab at coding this up, with a slightly modified interface than specified in the original question. Here's what I came up with.
(defn periodically [fn millis]
"Calls fn every millis. Returns a function that stops the loop."
(let [p (promise)]
(future
(while
(= (deref p millis "timeout") "timeout")
(fn)))
#(deliver p "cancel")))
Feedback welcomed.
Another option would be to use java.util.Timer's scheduleAtFixedRate method
edit - multiplex tasks on a single timer, and stop a single task rather than the entire timer
(defn ->timer [] (java.util.Timer.))
(defn fixed-rate
([f per] (fixed-rate f (->timer) 0 per))
([f timer per] (fixed-rate f timer 0 per))
([f timer dlay per]
(let [tt (proxy [java.util.TimerTask] [] (run [] (f)))]
(.scheduleAtFixedRate timer tt dlay per)
#(.cancel tt))))
;; Example
(let [t (->timer)
job1 (fixed-rate #(println "A") t 1000)
job2 (fixed-rate #(println "B") t 2000)
job3 (fixed-rate #(println "C") t 3000)]
(Thread/sleep 10000)
(job3) ;; stop printing C
(Thread/sleep 10000)
(job2) ;; stop printing B
(Thread/sleep 10000)
(job1))
Using core.async
(ns your-namespace
(:require [clojure.core.async :as async :refer [<! timeout chan go]])
)
(def milisecs-to-wait 1000)
(defn what-you-want-to-do []
(println "working"))
(def the-condition (atom true))
(defn evaluate-condition []
#the-condition)
(defn stop-periodic-function []
(reset! the-condition false )
)
(go
(while (evaluate-condition)
(<! (timeout milisecs-to-wait))
(what-you-want-to-do)))