Im playing around with core.async and find it very fun to work with. However I fail at understanding what are the different use cases for mult and broadcast. Are they both needed or will one be replaced by the other?
So far the only difference i have found is that it is easier to tap in and untag with mult. Not sure how to unsubscribe from a broadcast, is this the only difference?
Below I have an example showing how to solve a problem using both methods.
;; Using mult with tap
(def in (chan))
(def multiple (mult in))
(def out-1 (chan))
(tap multiple out-1)
(def out-2 (chan))
(tap multiple out-2)
(go (>! in "PutIN"))
(go (prn "From out-1: " (<! out-1)))
(go (prn "From out-2: " (<! out-2)))
//
;; Using broadcast
(def bout-1 (chan))
(def bout-2 (chan))
(def broadcast-in (broadcast bout-1 bout-2))
(go (>! broadcast-in "PutINBroadcast"))
(go (prn "From bout-1: " (<! bout-1)))
(go (prn "From bout-2: " (<! bout-2)))
This is the note to the clojure.core.async.lab namespace with broadcast.
core.async HIGHLY EXPERIMENTAL feature exploration
Caveats:
Everything defined in this namespace is experimental, and subject to change or deletion without warning.
Many features provided by this namespace are highly coupled to implementation details of core.async. Potential features which operate
at higher levels of abstraction are suitable for inclusion in the
examples.
Features provided by this namespace MAY be promoted to clojure.core.async at a later point in time, but there is no guarantee
any of them will.
Nobody has worked on it for a long time now, it has never been ported to ClojureScript. I'd expect it to be removed from core.async in the near future. mult is the later development.
Related
I am trying to figure out core.async in my REPL and am completely confused as to how my usage of (go-loop ...) doesn't manage to qualify as a "go block" for the purpose of async/>!
My go-loop is like...
(async/go-loop [page (range 3)]
(if (empty? page)
(async/close! ch)
(dorun (map (fn [row]
(println row)
(async/>! ch row)) page)))
(recur (range (dec (count page)))))
But the REPL is all upset...
=>
#object[clojure.core.async.impl.channels.ManyToManyChannel
0x23465937
"clojure.core.async.impl.channels.ManyToManyChannel#23465937"]
0
Exception in thread "async-dispatch-12" java.lang.AssertionError: Assert failed: >! used not in (go ...) block
nil
...
Why isn't the scope of that (go-loop ...) sufficient for the (async/>! row) call?
Should I even be using a go-loop here?
>! and other parking calls can't be used inside of functions nested inside of a go unfortunately.
go turns the code you give it into a state machine and looks for parking calls. It doesn't however look inside of nested functions.
From Clojure.Asyncs Github best practice page:
Unsupported constructs and other limitations in go blocks
The go macro stops translating at function creation boundaries. This means the following code will fail to compile, or may just throw a runtime error stating that <! was used outside of a go block:
(go (let [my-fn (fn [] (<! c))]
(my-fn)))
This is one thing to remember since many Clojure constructs create functions inside macros. The following are examples of code that will not work as one would expect:
(go (map <! some-chan))
(go (for [x xs]
(<! x)))
Is there a reasonable way to have multiple def statements happen with destructing the same way that let does it? For Example:
(let [[rtgs pcts] (->> (sort-by second row)
(apply map vector))]
.....)
What I want is something like:
(defs [rtgs pcts] (->> (sort-by second row)
(apply map vector)))
This comes up a lot in the REPL, notebooks and when debugging. Seriously feels like a missing feature so I'd like guidance on one of:
This exists already and I'm missing it
This is a bad idea because... (variable capture?, un-idiomatic?, Rich said so?)
It's just un-needed and I must be suffering from withdrawals from an evil language. (same as: don't mess up our language with your macros)
A super short experiment give me something like:
(defmacro def2 [[name1 name2] form]
`(let [[ret1# ret2#] ~form]
(do (def ~name1 ret1#)
(def ~name2 ret2#))))
And this works as in:
(def2 [three five] ((juxt dec inc) 4))
three ;; => 3
five ;; => 5
Of course and "industrial strength" version of that macro might be:
checking that number of names matches the number of inputs. (return from form)
recursive call to handle more names (can I do that in a macro like this?)
While I agree with Josh that you probably shouldn't have this running in production, I don't see any harm in having it as a convenience at the repl (in fact I think I'll copy this into my debug-repl kitchen-sink library).
I enjoy writing macros (although they're usually not needed) so I whipped up an implementation. It accepts any binding form, like in let.
(I wrote this specs-first, but if you're on clojure < 1.9.0-alpha17, you can just remove the spec stuff and it'll work the same.)
(ns macro-fun
(:require
[clojure.spec.alpha :as s]
[clojure.core.specs.alpha :as core-specs]))
(s/fdef syms-in-binding
:args (s/cat :b ::core-specs/binding-form)
:ret (s/coll-of simple-symbol? :kind vector?))
(defn syms-in-binding
"Returns a vector of all symbols in a binding form."
[b]
(letfn [(step [acc coll]
(reduce (fn [acc x]
(cond (coll? x) (step acc x)
(symbol? x) (conj acc x)
:else acc))
acc, coll))]
(if (symbol? b) [b] (step [] b))))
(s/fdef defs
:args (s/cat :binding ::core-specs/binding-form, :body any?))
(defmacro defs
"Like def, but can take a binding form instead of a symbol to
destructure the results of the body.
Doesn't support docstrings or other metadata."
[binding body]
`(let [~binding ~body]
~#(for [sym (syms-in-binding binding)]
`(def ~sym ~sym))))
;; Usage
(defs {:keys [foo bar]} {:foo 42 :bar 36})
foo ;=> 42
bar ;=> 36
(defs [a b [c d]] [1 2 [3 4]])
[a b c d] ;=> [1 2 3 4]
(defs baz 42)
baz ;=> 42
About your REPL-driven development comment:
I don't have any experience with Ipython, but I'll give a brief explanation of my REPL workflow and you can maybe comment about any comparisons/contrasts with Ipython.
I never use my repl like a terminal, inputting a command and waiting for a reply. My editor supports (emacs, but any clojure editor should do) putting the cursor at the end of any s-expression and sending that to the repl, "printing" the result after the cursor.
I usually have a comment block in the file where I start working, just typing whatever and evaluating it. Then, when I'm reasonably happy with a result, I pull it out of the "repl-area" and into the "real-code".
(ns stuff.core)
;; Real code is here.
;; I make sure that this part always basically works,
;; ie. doesn't blow up when I evaluate the whole file
(defn foo-fn [x]
,,,)
(comment
;; Random experiments.
;; I usually delete this when I'm done with a coding session,
;; but I copy some forms into tests.
;; Sometimes I leave it for posterity though,
;; if I think it explains something well.
(def some-data [,,,])
;; Trying out foo-fn, maybe copy this into a test when I'm done.
(foo-fn some-data)
;; Half-finished other stuff.
(defn bar-fn [x] ,,,)
(keys 42) ; I wonder what happens if...
)
You can see an example of this in the clojure core source code.
The number of defs that any piece of clojure will have will vary per project, but I'd say that in general, defs are not often the result of some computation, let alone the result of a computation that needs to be destructured. More often defs are the starting point for some later computation that will depend on this value.
Usually functions are better for computing a value; and if the computation is expensive, then you can memoize the function. If you feel you really need this functionality, then by all means, use your macro -- that's one of the sellings points of clojure, namely, extensibility! But in general, if you feel you need this construct, consider the possibility that you're relying too much on global state.
Just to give some real examples, I just referenced my main project at work, which is probably 2K-3K lines of clojure, in about 20 namespaces. We have about 20 defs, most of which are marked private and among them, none are actually computing anything. We have things like:
(def path-prefix "/some-path")
(def zk-conn (atom nil))
(def success? #{200})
(def compile* (clojure.core.memoize/ttl compiler {} ...)))
(def ^:private nashorn-factory (NashornScriptEngineFactory.))
(def ^:private read-json (comp json/read-str ... ))
Defining functions (using comp and memoize), enumerations, state via atom -- but no real computation.
So I'd say, based on your bullet points above, this falls somewhere between 2 and 3: it's definitely not a common use case that's needed (you're the first person I've ever heard who wants this, so it's uncommon to me anyway); and the reason it's uncommon is because of what I said above, i.e., it may be a code smell that indicates reliance on too much global state, and hence, would not be very idiomatic.
One litmus test I have for much of my code is: if I pull this function out of this namespace and paste it into another, does it still work? Removing dependencies on external vars allows for easier testing and more modular code. Sometimes we need it though, so see what your requirements are and proceed accordingly. Best of luck!
Consider the following snippet:
(require '[clojure.core.async :refer :all])
(def my-chan (chan (buffer 10)))
(go (while true
(>! my-chan (rand))))
This basically provides a buffered channel, which always contains some 10 random numbers. When the channel is consumed, the buffer is filled again.
Is there an abstraction for this in core.async? As there are transducers for manipulating the consumption of channels, there might be something for the production of them as well:
For sequences one would go for something like this:
(def my-seq
(map (fn [_] (rand)) (range)))
or, just:
(def my-seq (repeatedly rand))
Which of course is not buffered, but it might give an idea of what I'm looking for.
Transducers don't manipulate the consumption of channels -- they affect the values, but they don't affect the consumption of the data on the channel.
You seem to be asking of a way to abstract the creation of a channel, and then get values off of it as a sequence. Here are some ideas, though I'm not convinced that core.async really offers anything above normal clojure.core functionality in this case.
Abstraction is done here the way it usually is done -- with functions. This will call f and put its result on the channel. The implication here is of course that f will be side-effecting, and impure, otherwise it would be quite a boring channel to consume from, with every value being identical.
(defn chan-factory
[f buf]
(let [c (chan buf)]
(go-loop []
(>! c (f))
(recur))
c))
If you then wanted to create a lazy sequence from this, you could do:
(defn chan-to-seq [c]
(lazy-seq
(cons (<!! c) (chan-to-seq c))))
Define your seq:
(def rand-from-chan (chan-to-seq (chan-factory rand 10)))
(take 5 rand-from-chan)
=>
(0.6873518531956767
0.6940302424998631
0.07293052906941855
0.7264083273536271
0.4670275072317531)
However, you can accomplish this same thing by doing:
(def rand-nums (repeatedly rand))
So, while what you're doing is a great thought experiment, it may be more helpful to find some concrete use cases, and then maybe you will receive more specific ideas. Good luck!
Consider the following snippet:
(let [chs (repeatedly 10 chan)]
(doseq [c chs]
(>!! c "hello"))
(doseq [c chs]
(println (<!! c))))
Executing this will hang forever. Why is that?
If I do (go (>! c "hello")) instead, it works just fine.
To make an asynchronous put, use clojure.core.async/put!
(let [chs (repeatedly 10 chan)]
(doseq [c chs]
(put! c "hello"))
(doseq [c chs]
(println (<!! c))))
This works in this example as <!! will always unblock because of all necessary puts happening asynchronously. Notice the following things:
Blocking serves as a synchronization constraint between different processes
>!! and <!! block the main-thread. go routines run on the main thread but their code is modified via macroexpansion so that execution control is inverted and they can be parked/executed successively ordered by the laws of core.async channel blocking/buffering logic. This technique is commonly referred to as IOC (inversion of control) state machine.
ClojureScript has only one thread. Consequently its implementation of core.async doesn't even contain >!!/<!!. If you write code intended to be ClojureScript compatible, only take from channels within go-routines or dispatch values from them in higher order functions passed to take! and always do puts either in go-routines or use put!.
Is (go (>! ch v)) equivalent to (put! ch v)?
Yes but it's not the same. put! is an API wrapper around the channels implementation of the core.async.impl.protocols/WritePort put! method. Macroexpansion of (go (>! ch v)) ends up in the same method call happening but wraps it in lots of generated state-machine code to possibly park the putting operation and pause execution of the go-routine until a consumer is ready to take from ch (try to (macroexpand `(go (>! ch v))) yourself). Spawning a go-block to only do one asynchronous putting operation is kind of a waste and performs worse than calling put! right away. go spawns and returns an extra channel that you can take its bodies result from. This offers you to await completion of its execution which you didn't intend to do in your example (aiming at an asynchronous operation).
That channel has no buffer, and >!! is blocking. Refer to the examples for this exact case. They spawn a second thread for this reason - to prevent blocking the main thread. Using a goroutine, as in your question, operates on a similar principle.
You can also create the channel with some buffer space - (chan 1)
I am working on my first (non-trivial) Clojure program I don't really feel comfortable with how I am declaring all my mutable state globally. For example:
(def next-blocks (atom []))
(def num-next-blocks 1)
(def is-game-over (atom false))
(def user-name (atom (str)))
(def hs-xml (atom nil))
Since I use C a lot at work I came up with the idea of using common C-style encapsulation techniques. It typically involves struct object that passed as a first argument to any "member functions" that operate on it. See udev for example.
Applying this to Clojure would result in functions that look like this (untested):
(defstruct gamestate)
(defn game-new []
(struct-map gamestate
:level (atom 0)
:score (atom 0)
;etc...
))
(def game-get-score [game]
#(game :score))
(defn game-set-score [game new-score]
(reset! (game :score) new-score))
(defn game-get-level [game]
#(game :level))
(defn game-inc-level [game]
(swap! (game :level) inc))
; etc...
I think it would definitely be a step forward to the global defines that I'm using currently.
So is this the recommended way to go? Or is there a more standard Clojure way?
Update
I'm currently using Clojure 1.1.0.
The basic idea of functional programming is that you have very few global variables but mostly local ones (see Rich Hickey's article on state and identity). Writing a game in this paradigm may be challenging, I'd recommend this post on functional programming for games (the examples are in Erlang, though).
I don't know what kind of game you want to implement. Here is a code snippet improv of what I would do with local variables.
(defn play-game [name score blocks]
(let [level (start-mission (:level score) blocks)]
(if level
(assoc score :level level)
score)))
(defn the-whole-game []
(let [name (ask-username)
score (or (load-score name) {:level 0, :score 0}]
(when-let [new-score (play-game name score [])]
(save-score name new-score))))
You may want to check out another Tetris clone in clojure, it uses opengl, though.
In my Clojure game I'm using a single atom containing a map to store all my mutable game state. This includes certain elements of the user interface state.
This is currently defined as follows:
(def state
(atom
{:game (gamefactory/make-game)
:scroll [0 0]
:mouseover [0 0]
:command-state nil
:commands (clojure.lang.PersistentQueue/EMPTY)
:animations {}
:player-id nil}))
This model is working very well for me. You can easily access the state elements directly e.g. (:game #state), or alternatively define accessor functions.
You can use a map to emulate a C style struct. You can also use (if you are using v1.2) you can use deftype/defrecord.
(defn get-game-score [game]
(:score game))
(defn set-game-store [game new-score]
(assoc game :score new-score))
I would recommend using a Map esp. since those can be easily used in multimethods.
The biggest thing to try to keep in mind that is that you should not think about using variables in clojure the same way you do in C and atoms are not the same as variables.