Should you close *out* (standard output)? For example, via the with-open macro:
(require '[clojure.java.io :as io])
(with-open [w (io/writer *out*)]
(.write w "foo"))
If I run this via Leiningen's (v2.7.0) lein run, I get an exception:
Exception in thread "main" java.io.IOException: Stream closed
at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:45)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:140)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at clojure.core$flush.invokeStatic(core.clj:3691)
at clojure.main$main.invokeStatic(main.clj:423)
at clojure.main$main.doInvoke(main.clj:386)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.main.main(main.java:37)
Otherwise, the code works fine.
Related
My attempt:
(import 'java.lang.Runtime)
(. (Runtime/getRuntime) exec (into-array ["youtube-dl" "--no-playlist" "some youtube video link"]))
I also tried sh. But both approaches don't do what I want - running a program similarly like shell does (sh waits until program exits, exec launches it and doesn't wait for its exit; both don't output anything to standard output). I want live showing of process output, e.g. when I run youtube-dl I want to see progress of a video download.
How to do this simple simple task in Clojure?
You must start the process and listen to its output stream. One solution is :
(:require [clojure.java.shell :as sh]
[clojure.java.io :as io])
(let [cmd ["yes" "1"]
proc (.exec (Runtime/getRuntime) (into-array cmd))]
(with-open [rdr (io/reader (.getInputStream proc))]
(doseq [line (line-seq rdr)]
(println line))))
The problem: I have channel that consumer reads from and might stop reading when got enough data. When reader stops it closes channel with clojure.core.async/close!
The documentation says that at this moment all puts to channel after close is invoked should return false and do nothing. But the documentation also says that
Logically closing happens after all puts have been delivered. Therefore, any blocked or parked puts will remain blocked/parked until a taker releases them.
Does it mean that to release producers that were already blocked in parked puts at the moment of closing channel I should always also drain channel (read all remaining items) at consumer side? Following code shows that go block never finishes:
(require '[clojure.core.async :as a])
(let [c (a/chan)]
(a/go
(prn "Go")
(prn "Put" (a/>! c 333)))
(Thread/sleep 300) ;; Let go block to be scheduled
(a/close! c))
If this is true, and I do not want to read all events then I should implement e.g. timeouts at producer side to detect that no more data is necessary?
Is there simpler way for consumer to tell "enough" to push back so producer stops also gracefully?
I found out that clojure.core.async/put! does not block and allows to avoid unnecessary blocking. Are there disadvantages of using it instead of clojure.core.aasync/>!?
closing chans frees all who are reading from them them, and leaves writers blocked
here is the reading case (where it works nicely):
user> (def a-chan (async/chan))
#'user/a-chan
user> (future (async/<!! a-chan)
(println "continuting after take"))
#future[{:status :pending, :val nil} 0x5fb5a025]
user> (async/close! a-chan)
nil
user> continuting after take
And here is a test of the writing case where, as you say, draining it may be a good idea:
user> (def b-chan (async/chan))
#'user/b-chan
user> (future (try (async/>!! b-chan 4)
(println "continuting after put")
(catch Exception e
(println "got exception" e))
(finally
(println "finished in finally"))))
#future[{:status :pending, :val nil} 0x17be0f7b]
user> (async/close! b-chan)
nil
I don't find any evidence of the stuck writer unblocking here when the chan is closed
This behavior is intended, since they explicitly state it in the docs!
In your case, do (while (async/poll! c)) after closing channel c to release all blocked/parked (message sending) threads/go-blocks.
If you want to do anything with the content you can do:
(->> (repeatedly #(async/poll! c))
(take-while identity))
How do you detect non-empty standard input (*in*) without reading from it in a non-blocking way in Clojure?
At first, I thought calling using the java.io.Reader#ready() method would do, but (.ready *in*) returns false even when standard input is provided.
Is this what you are looking for? InputStream .available
(defn -main [& args]
(if (> (.available System/in) 0)
(println "STDIN: " (slurp *in*))
(println "No Input")))
$ echo "hello" | lein run
STDIN: hello
$ lein run
No Input
Update: It does seem that .available is a race condition checking STDIN. n alternative is to have a fixed timeout for STDIN to become available otherwise assume no data is coming from STDIN
Here is an example of using core.async to attempt to read the first byte from STDIN and append it to the rest of the STDIN or timeout.
(ns stdin.core
(:require
[clojure.core.async :as async :refer [go >! timeout chan alt!!]])
(:gen-class))
(defn -main [& args]
(let [c (chan)]
(go (>! c (.read *in*)))
(if-let [ch (alt!! (timeout 500) nil
c ([ch] (if-not (< ch 0) ch)))]
(do
(.unread *in* ch)
(println (slurp *in*)))
(println "No STDIN"))))
Have you looked at PushbackReader? You can use it like:
Read a byte (blocking). Returns char read or -1 if stream is closed.
When returns, you know a byte is ready.
If the byte is something you're not ready for, put it back
If stream is closed (-1 return val), exit.
Repeat.
https://docs.oracle.com/javase/8/docs/api/index.html?java/io/PushbackReader.html
If you need it to be non-blocking stick it into a future, a core.async channel, or similar.
What is the most idiomatic way to write a data structure to disk in Clojure, so I can read it back with edn/read? I tried the following, as recommended in the Clojure cookbook:
(with-open [w (clojure.java.io/writer "data.clj")]
(binding [*out* w]
(pr large-data-structure)))
However, this will only write the first 100 items, followed by "...". I tried (prn (doall large-data-structure)) as well, yielding the same result.
I've managed to do it by writing line by line with (doseq [i large-data-structure] (pr i)), but then I have to manually add the parens at the beginning and end of the sequence to get the desired result.
You can control the number of items in a collection that are printed via *print-length*
Consider using spit instead of manually opening the writer and pr-str instead of manually binding to *out*.
(binding [*print-length* false]
(spit "data.clj" (pr-str large-data-structure))
Edit from comment:
(with-open [w (clojure.java.io/writer "data.clj")]
(binding [*print-length* false
*out* w]
(pr large-data-structure)))
Note: *print-length* has a root binding of nil so you should not need to bind it in the example above. I would check the current binding at the time of your original pr call.
I must be missing something very obvious here but I'm trying to setup a very basic program to put an item onto a channel then block until I can take it off again. The entire program is below:
(ns shopping-2.core
(:require [cljs.core.async :as async :refer [>!! <!! put! chan <! close! <!!]]))
(let [c (chan)]
(>!! c "hello")
(.write js/document (<!! c))
(close! c))
The JavaScript error I'm getting is:
Uncaught TypeError: Cannot call method 'call' of undefined
I had that error before when I forgot to :refer chan in (if I just open the channel, then close it again the program runs fine)
However this code seems to choke when I want to use the <!! or >!! macros.
There are some differences on what's available in clojurescript from the clojure version of core.async.
Because clojure on the JVM has real threads, it makes available both concurrency patterns with real threads and with go blocks:
Real threads use the macro thread to enclose the core.async magic
and its concurrency macros and functions end with two bangs, like
<!!, >!!, alt!! and alts!!.
Inversion of control threads
(fake threads) use the go macro to enclose the core.async magic
and uses the functions with one bang at the end, like <!, >!,
alt! and alts!.
In clojurescript (which runs in js) there are no real threads, so only the IoC (inversion of control) threads are available, which means that you have to use the second variant of the concurrency constructs.
Your example would be like:
(ns shopping-2.core
(:require-macros [cljs.core.async.macros :refer [go]])
(:require [cljs.core.async :as async :refer [put! chan <! >! close!]]))
(go
(let [c (chan)]
(>! c "hello")
(.write js/document (<! c))
(close! c)))
Anyway, that example has a concurrency problem, since the <! >! functions are blocking and you are putting into a chan in the same routine, the routine will block on the (>! c "hello") instruction and it will never read, definitely starving your program.
You could fix this using the put! fn which puts without blocking, or efectively running those instructions in different routines which I think demonstrates better what you intended to do.
(ns shopping-2.core
(:require-macros [cljs.core.async.macros :refer [go]])
(:require [cljs.core.async :as async :refer [put! chan <! >! close!]]))
;; put! version
(go
(let [c (chan)]
(put! c "hello")
(.write js/document (<! c))
(close! c)))
;; Concurrent version
;; 2 *threads* concurrently running. One is the putter and the other is the
;; reader
(let [c (chan)]
(go
(.write js/document (<! c))
(close! c))
(go
(>! c "hello")))
In the concurrent threads version, you will see that even the code that runs first is the read, it is effectively another routine so the code that runs later (the >!) effectively runs unblocking the first routine.
You can think of the go macro as spawning a new thread that will eventually start executing concurrently and that returns control to the next instructions of code after it immediately.
I suggest reading the code walk-through ignoring the clojure specific parts (>!! <!! etc) and some of the swannodette's tutorials which are great (like Clojurescript 101 and Communicating Sequential Processes)
The ClojureScript version of core.async doesn't include <!! or >!!.
I couldn't find a source for this besides the actual source: https://github.com/clojure/core.async/blob/56ded53243e1ef32aec71715b1bfb2b85fdbdb6e/src/main/clojure/cljs/core/async.cljs