I am trying to figure out core.async in my REPL and am completely confused as to how my usage of (go-loop ...) doesn't manage to qualify as a "go block" for the purpose of async/>!
My go-loop is like...
(async/go-loop [page (range 3)]
(if (empty? page)
(async/close! ch)
(dorun (map (fn [row]
(println row)
(async/>! ch row)) page)))
(recur (range (dec (count page)))))
But the REPL is all upset...
=>
#object[clojure.core.async.impl.channels.ManyToManyChannel
0x23465937
"clojure.core.async.impl.channels.ManyToManyChannel#23465937"]
0
Exception in thread "async-dispatch-12" java.lang.AssertionError: Assert failed: >! used not in (go ...) block
nil
...
Why isn't the scope of that (go-loop ...) sufficient for the (async/>! row) call?
Should I even be using a go-loop here?
>! and other parking calls can't be used inside of functions nested inside of a go unfortunately.
go turns the code you give it into a state machine and looks for parking calls. It doesn't however look inside of nested functions.
From Clojure.Asyncs Github best practice page:
Unsupported constructs and other limitations in go blocks
The go macro stops translating at function creation boundaries. This means the following code will fail to compile, or may just throw a runtime error stating that <! was used outside of a go block:
(go (let [my-fn (fn [] (<! c))]
(my-fn)))
This is one thing to remember since many Clojure constructs create functions inside macros. The following are examples of code that will not work as one would expect:
(go (map <! some-chan))
(go (for [x xs]
(<! x)))
Related
Is there a reasonable way to have multiple def statements happen with destructing the same way that let does it? For Example:
(let [[rtgs pcts] (->> (sort-by second row)
(apply map vector))]
.....)
What I want is something like:
(defs [rtgs pcts] (->> (sort-by second row)
(apply map vector)))
This comes up a lot in the REPL, notebooks and when debugging. Seriously feels like a missing feature so I'd like guidance on one of:
This exists already and I'm missing it
This is a bad idea because... (variable capture?, un-idiomatic?, Rich said so?)
It's just un-needed and I must be suffering from withdrawals from an evil language. (same as: don't mess up our language with your macros)
A super short experiment give me something like:
(defmacro def2 [[name1 name2] form]
`(let [[ret1# ret2#] ~form]
(do (def ~name1 ret1#)
(def ~name2 ret2#))))
And this works as in:
(def2 [three five] ((juxt dec inc) 4))
three ;; => 3
five ;; => 5
Of course and "industrial strength" version of that macro might be:
checking that number of names matches the number of inputs. (return from form)
recursive call to handle more names (can I do that in a macro like this?)
While I agree with Josh that you probably shouldn't have this running in production, I don't see any harm in having it as a convenience at the repl (in fact I think I'll copy this into my debug-repl kitchen-sink library).
I enjoy writing macros (although they're usually not needed) so I whipped up an implementation. It accepts any binding form, like in let.
(I wrote this specs-first, but if you're on clojure < 1.9.0-alpha17, you can just remove the spec stuff and it'll work the same.)
(ns macro-fun
(:require
[clojure.spec.alpha :as s]
[clojure.core.specs.alpha :as core-specs]))
(s/fdef syms-in-binding
:args (s/cat :b ::core-specs/binding-form)
:ret (s/coll-of simple-symbol? :kind vector?))
(defn syms-in-binding
"Returns a vector of all symbols in a binding form."
[b]
(letfn [(step [acc coll]
(reduce (fn [acc x]
(cond (coll? x) (step acc x)
(symbol? x) (conj acc x)
:else acc))
acc, coll))]
(if (symbol? b) [b] (step [] b))))
(s/fdef defs
:args (s/cat :binding ::core-specs/binding-form, :body any?))
(defmacro defs
"Like def, but can take a binding form instead of a symbol to
destructure the results of the body.
Doesn't support docstrings or other metadata."
[binding body]
`(let [~binding ~body]
~#(for [sym (syms-in-binding binding)]
`(def ~sym ~sym))))
;; Usage
(defs {:keys [foo bar]} {:foo 42 :bar 36})
foo ;=> 42
bar ;=> 36
(defs [a b [c d]] [1 2 [3 4]])
[a b c d] ;=> [1 2 3 4]
(defs baz 42)
baz ;=> 42
About your REPL-driven development comment:
I don't have any experience with Ipython, but I'll give a brief explanation of my REPL workflow and you can maybe comment about any comparisons/contrasts with Ipython.
I never use my repl like a terminal, inputting a command and waiting for a reply. My editor supports (emacs, but any clojure editor should do) putting the cursor at the end of any s-expression and sending that to the repl, "printing" the result after the cursor.
I usually have a comment block in the file where I start working, just typing whatever and evaluating it. Then, when I'm reasonably happy with a result, I pull it out of the "repl-area" and into the "real-code".
(ns stuff.core)
;; Real code is here.
;; I make sure that this part always basically works,
;; ie. doesn't blow up when I evaluate the whole file
(defn foo-fn [x]
,,,)
(comment
;; Random experiments.
;; I usually delete this when I'm done with a coding session,
;; but I copy some forms into tests.
;; Sometimes I leave it for posterity though,
;; if I think it explains something well.
(def some-data [,,,])
;; Trying out foo-fn, maybe copy this into a test when I'm done.
(foo-fn some-data)
;; Half-finished other stuff.
(defn bar-fn [x] ,,,)
(keys 42) ; I wonder what happens if...
)
You can see an example of this in the clojure core source code.
The number of defs that any piece of clojure will have will vary per project, but I'd say that in general, defs are not often the result of some computation, let alone the result of a computation that needs to be destructured. More often defs are the starting point for some later computation that will depend on this value.
Usually functions are better for computing a value; and if the computation is expensive, then you can memoize the function. If you feel you really need this functionality, then by all means, use your macro -- that's one of the sellings points of clojure, namely, extensibility! But in general, if you feel you need this construct, consider the possibility that you're relying too much on global state.
Just to give some real examples, I just referenced my main project at work, which is probably 2K-3K lines of clojure, in about 20 namespaces. We have about 20 defs, most of which are marked private and among them, none are actually computing anything. We have things like:
(def path-prefix "/some-path")
(def zk-conn (atom nil))
(def success? #{200})
(def compile* (clojure.core.memoize/ttl compiler {} ...)))
(def ^:private nashorn-factory (NashornScriptEngineFactory.))
(def ^:private read-json (comp json/read-str ... ))
Defining functions (using comp and memoize), enumerations, state via atom -- but no real computation.
So I'd say, based on your bullet points above, this falls somewhere between 2 and 3: it's definitely not a common use case that's needed (you're the first person I've ever heard who wants this, so it's uncommon to me anyway); and the reason it's uncommon is because of what I said above, i.e., it may be a code smell that indicates reliance on too much global state, and hence, would not be very idiomatic.
One litmus test I have for much of my code is: if I pull this function out of this namespace and paste it into another, does it still work? Removing dependencies on external vars allows for easier testing and more modular code. Sometimes we need it though, so see what your requirements are and proceed accordingly. Best of luck!
Consider the following snippet:
(require '[clojure.core.async :refer :all])
(def my-chan (chan (buffer 10)))
(go (while true
(>! my-chan (rand))))
This basically provides a buffered channel, which always contains some 10 random numbers. When the channel is consumed, the buffer is filled again.
Is there an abstraction for this in core.async? As there are transducers for manipulating the consumption of channels, there might be something for the production of them as well:
For sequences one would go for something like this:
(def my-seq
(map (fn [_] (rand)) (range)))
or, just:
(def my-seq (repeatedly rand))
Which of course is not buffered, but it might give an idea of what I'm looking for.
Transducers don't manipulate the consumption of channels -- they affect the values, but they don't affect the consumption of the data on the channel.
You seem to be asking of a way to abstract the creation of a channel, and then get values off of it as a sequence. Here are some ideas, though I'm not convinced that core.async really offers anything above normal clojure.core functionality in this case.
Abstraction is done here the way it usually is done -- with functions. This will call f and put its result on the channel. The implication here is of course that f will be side-effecting, and impure, otherwise it would be quite a boring channel to consume from, with every value being identical.
(defn chan-factory
[f buf]
(let [c (chan buf)]
(go-loop []
(>! c (f))
(recur))
c))
If you then wanted to create a lazy sequence from this, you could do:
(defn chan-to-seq [c]
(lazy-seq
(cons (<!! c) (chan-to-seq c))))
Define your seq:
(def rand-from-chan (chan-to-seq (chan-factory rand 10)))
(take 5 rand-from-chan)
=>
(0.6873518531956767
0.6940302424998631
0.07293052906941855
0.7264083273536271
0.4670275072317531)
However, you can accomplish this same thing by doing:
(def rand-nums (repeatedly rand))
So, while what you're doing is a great thought experiment, it may be more helpful to find some concrete use cases, and then maybe you will receive more specific ideas. Good luck!
I'm using this basic macro as a building block for other timing macros:
(defmacro time-pure
"Evaluates expr and returns the time it took.
Modified the native time macro to return the time taken."
[expr]
`(let [start# (current-nano-timestamp)
ret# ~expr]
(/ (double (- (current-nano-timestamp) start#)) 1000000.0)))
I've tested this, and used it in other macros, so I know it works fine.
My problem can be described by the following snippet:
(defmacro time-each [& exprs]
`(mapv #(time-pure %) '~exprs))
I would expect this to give each expression to time-each, where it executes and times it; returning the result. When I test it however, it finishes instantly:
(time-each
(Thread/sleep 500)
(Thread/sleep 1000))
[0.036571 0.0]
I'm confused by this since I know that (time-pure (Thread/sleep 1000)) will take around a second to return, and all this macro does it delegate to time-pure.
What's causing this? I really have no idea how to properly debug a macro. I used macro-expand-1 to inspect the generated code:
(clojure.pprint/pprint
(macroexpand-1
'(time-each
(Thread/sleep 500)
(Thread/sleep 1000))))
(clojure.core/mapv
(fn*
[p1__1451__1452__auto__]
(helpers.general-helpers/time-pure p1__1451__1452__auto__))
'((Thread/sleep 500) (Thread/sleep 1000)))
But nothing really stands out to me.
What's going on here?
(Note, this is a dupe of a question I posted a few minutes ago. I realized the case I was showing was convoluted, so I made up a better example.)
Solution
I think what happens here is that your mapv executes after compile time, meaning at run time, this is why it cant just "paste" the code list as an sexp. I think a better approach would be to keep the mapv out of the syntax quoting:
(defmacro each-time-mine [& exprs]
(mapv #(time-pure %) exprs))
(each-time-mine
(Thread/sleep 500)
(Thread/sleep 1000))
[501.580465 1001.196752]
Original
Although eval is usually frowned upon, it seems to solve the problem in this situation:
(defmacro time-pure
[expr]
`(let [start# (current-nano-timestamp)
ret# (eval ~expr)]
(/ (double (- (current-nano-timestamp) start#)) 1000000.0)))
(defmacro time-each [& exprs]
`(mapv #(time-pure %) '~exprs))
(time-each
(Thread/sleep 500)
(Thread/sleep 1000))
[501.249249 1001.242522]
What happens is that mapv sees the sexps as lists, and at the moment time-pure wants to execute them, it simply assigns ret# the value of the list. So that list needs to be evaled.
There might be better ways to achieve this though.
(defmacro time-each [& exprs]
`(list ~#(for [expr exprs]
`(time-pure ~expr))))
You need to be careful not to evaluate an expression outside of a time-pure context, as is done in Arthur's answer, which evaluates each expression and then calls time-pure on the (very fast) operation of "looking at the result". Rather, wrap time-pure around each expression before evaluating it.
"I'm still trying to don't wrap my head around macros."
It takes a while. :) One thing that helped me was macro-expanding:
(macroexpand-1 '(time-each (+ 2 2 )(+ 3 3)))
That produces:
(clojure.core/mapv
(fn* [p1__24013__24014__auto__]
(rieclj.core/time-pure p1__24013__24014__auto__))
(quote ((+ 2 2) (+ 3 3))))
The alarming thing there is that time-pure is being passed a quoted list, so it is all just symbols at run time, not macroexpansion time.
Then another thing that helped me was that the beauty of Lispy macros is that they are normal functions that just run at a different time (while the reader is processing the source). But because they are normal functions we can probe them with print statements just as we probe normal code.
I have modified time-pure to print out its macroexpansion-time argument expr and, in the generate code, to have it print out the evaluated input, which we now suspect is a list of symbols, or '(+ 2 2) in the first case.
(defmacro time-pure
"Evaluates expr and returns the time it took.
Modified the native time macro to return the time taken."
[expr]
(prn :time-pure-sees expr)
`(let [start# (now)
ret# ~expr]
(prn :ret-is ret# (type (first ret#)))
(/ (double (- (now) start#)) 1000000.0)))
I print the type of the first of ret to drive home that + is a symbol, not a function. Evaluating:
(time-each (+ 2 2 )(+ 3 3))
yields:
:time-pure-sees p1__24013__24014__auto__
:ret-is (+ 2 2) clojure.lang.Symbol
:ret-is (+ 3 3) clojure.lang.Symbol
Seeing (+ 2 2) might make you think all is well, but the key is that the rightside is '(+ 2 2) so ret# gets bound to that symbolic expression instead of the intended computation.
Again, printing the type of the first hopefully makes clear that time-pure was working on a list of symbols at runtime.
Sorry if all that was confusing, but the moral is simple: use macroexpand-1 and embedded print statements when macros mess with your head. In time you will internalize the two different times, macroexpansion and run.
I find I very rarely use let in Clojure. For some reason I took a dislike to it when I started learning and have avoided using it ever since. It feels like the flow has stopped when let comes along. I was wondering, do you think we could do without it altogether ?
You can replace any occurrence of (let [a1 b1 a2 b2...] ...) by ((fn [a1 a2 ...] ...) b1 b2 ...) so yes, we could. I am using let a lot though, and I'd rather not do without it.
Let offers a few benefits. First, it allows value binding in a functional context. Second, it confers readability benefits. So while technically, one could do away with it (in the sense that you could still program without it), the language would be impoverished without a valuable tool.
One of the nice things about let is that it helps formalize a common (mathematical) way of specifying a computation, in which you introduce convenient bindings and then a simplified formula as a result. It's clear the bindings only apply to that "scope" and it's tie in with a more mathematical formulation is useful, especially for more functional programmers.
It's not a coincidence that let blocks occur in other languages like Haskell.
Let is indispensable to me in preventing multiple execution in macros:
(defmacro print-and-run [s-exp]
`(do (println "running " (quote ~s-exp) "produced " ~s-exp)
s-exp))
would run s-exp twice, which is not what we want:
(defmacro print-and-run [s-exp]
`(let [result# s-exp]
(do (println "running " (quote ~s-exp) "produced " result#)
result#))
fixes this by binding the result of the expression to a name and referring to that result twice.
because the macro is returning an expression that will become part of another expression (macros are function that produce s-expressions) they need to produce local bindings to prevent multiple execution and avoid symbol capture.
I think I understand your question. Correct me if it's wrong. Some times "let" is used for imperative programming style. For example,
... (let [x (...)
y (...x...)
z (...x...y...)
....x...y...z...] ...
This pattern comes from imperative languages:
... { x = ...;
y = ...x...;
...x...y...;} ...
You avoid this style and that's why you also avoid "let", don't you?
In some problems imperative style reduces amount of code. Furthermore, some times It's more efficient to write in java or c.
Also in some cases "let" just holds values of subexpressions regardless of evaluation order. For example,
(... (let [a (...)
b (...)...]
(...a...b...a...b...) ;; still fp style
There are at least two important use cases for let-bindings:
First, using let properly can make your code clearer and shorter. If you have an expression that you use more than once, binding it in a let is very nice. Here's a portion of the standard function map that uses let:
...
(let [s1 (seq c1) s2 (seq c2)]
(when (and s1 s2)
(cons (f (first s1) (first s2))
(map f (rest s1) (rest s2)))))))
...
Even if you use an expression only once, it can still be helpful (to future readers of the code) to give it a semantically meaningful name.
Second, as Arthur mentioned, if you want to use the value of an expression more than once, but only want it evaluated once, you can't simply type out the entire expression twice: you need some kind of binding. This would be merely wasteful if you have a pure expression:
user=> (* (+ 3 2) (+ 3 2))
25
but actually changes the meaning of the program if the expression has side-effects:
user=> (* (+ 3 (do (println "hi") 2))
(+ 3 (do (println "hi") 2)))
hi
hi
25
user=> (let [x (+ 3 (do (println "hi") 2))]
(* x x))
hi
25
Stumbled upon this recently so ran some timings:
(testing "Repeat vs Let vs Fn"
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
(* (+ 3 2) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)
n (+ 3 2)]
(dotimes [x 1000000]
(* n n))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
((fn [x] (* x x)) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))))
Output
Testing Repeat vs Let vs Fn
116
18
60
'let' wins over 'pure' functional.
I have been rewriting Land Of Lisp's orc-battle game in Clojure. During the process I am using a more functional style. I have come up with two methods for writing part of the higher level game loop. One involving loop/recur and the other using doseq and atoms. Here are the two functions:
(defn monster-round [player monsters]
(loop [n 0 p player]
(if (>= n (count monsters))
p
(recur (inc n)
(if (monster-dead? (nth monsters n))
p
(let [r (monster-attack (nth monsters n) p)]
(print (:attack r))
(:player r)))))))
(defn monster-round-2 [player monsters]
(let [p (atom player)]
(doseq [m monsters]
(if (not (monster-dead? m))
(let [r (monster-attack m #p)]
(print (:attack r))
(reset! p (:player r)))))
#p))
I like the second method better because the code is more concise and is easier to follow. Is there any reason why the first approach is better? Or am I missing a different way to do this?
is this equivalent? if so, i prefer it - it's compact, clearer than your solutions (imho!), and functional
(defn monster-round [monsters player]
(if-let [[monster & monsters] monsters]
(recur monsters
(if (monster-dead? monster)
player
(let [r (monster-attack monster player)]
(print (:attack r))
(:player r))))
player))
(note: i changed the argument order to monster-round so that the recur looked nicer)
more generally, you should not have introduced n in your "functional" version (it's not really really functional if you've got an index...). indexing into a sequence is very, very rarely needed. if you had fought the temptation to do that a little harder, i think you would have written the routine above...
but, after writing that, i thought: "hmmm. that's just iterating over monsters. why can't we use a standard form? it's not a for loop because player changes. so it must be a fold (ie a reduce), which carries the player forwards". and then it was easy to write:
(defn- fight [player monster]
(if (monster-dead? monster)
player
(let [r (monster-attack monster player)]
(print (:attack r))
(:player r))))
(defn monster-round [player monsters]
(reduce fight player monsters))
which (if it does what you want) is the Correct Answer(tm).
(maybe i am not answering the question? i think you missed the better way, as above. in general, you should be able to thread the computation around the data structure, which does normally not require mutation; often you can - and should - use the standard forms like map and reduce because they help document the process for others).