I want to start a server with clojure boot that must keep on running. So I made a -main function so that I can run ./build.boot and that will execute the main function. In the main function I start a non-daemon thread with the idea that the JVM keeps on running as long as there is a non-daemon thread. However the boot script stops and my server stops as well. What am I doing wrong?
#!/usr/bin/env boot
(defn -main [& args]
(let [t (Thread. #(loop []
(println (java.util.Date.))
(Thread/sleep 500)
(recur)))]
(.setDaemon t false)
(.start t)
(Thread/sleep 3000)
(println "I have started a non-daemon thread, but still I exit???")))
Output
#inst "2016-06-23T08:39:23.453-00:00"
#inst "2016-06-23T08:39:24.005-00:00"
#inst "2016-06-23T08:39:24.507-00:00"
#inst "2016-06-23T08:39:25.009-00:00"
#inst "2016-06-23T08:39:25.511-00:00"
#inst "2016-06-23T08:39:26.013-00:00"
I have started a non-daemon thread, but still I exit???
After which the script ends.
I guess boot calls System/exit once the tasks are done (their function call ends).
You need to modify your task to join to your spawned thread so the -main function doesn't complete until your spawned thread hasn't finished:
(.join t)
Related
My attempt:
(import 'java.lang.Runtime)
(. (Runtime/getRuntime) exec (into-array ["youtube-dl" "--no-playlist" "some youtube video link"]))
I also tried sh. But both approaches don't do what I want - running a program similarly like shell does (sh waits until program exits, exec launches it and doesn't wait for its exit; both don't output anything to standard output). I want live showing of process output, e.g. when I run youtube-dl I want to see progress of a video download.
How to do this simple simple task in Clojure?
You must start the process and listen to its output stream. One solution is :
(:require [clojure.java.shell :as sh]
[clojure.java.io :as io])
(let [cmd ["yes" "1"]
proc (.exec (Runtime/getRuntime) (into-array cmd))]
(with-open [rdr (io/reader (.getInputStream proc))]
(doseq [line (line-seq rdr)]
(println line))))
The problem: I have channel that consumer reads from and might stop reading when got enough data. When reader stops it closes channel with clojure.core.async/close!
The documentation says that at this moment all puts to channel after close is invoked should return false and do nothing. But the documentation also says that
Logically closing happens after all puts have been delivered. Therefore, any blocked or parked puts will remain blocked/parked until a taker releases them.
Does it mean that to release producers that were already blocked in parked puts at the moment of closing channel I should always also drain channel (read all remaining items) at consumer side? Following code shows that go block never finishes:
(require '[clojure.core.async :as a])
(let [c (a/chan)]
(a/go
(prn "Go")
(prn "Put" (a/>! c 333)))
(Thread/sleep 300) ;; Let go block to be scheduled
(a/close! c))
If this is true, and I do not want to read all events then I should implement e.g. timeouts at producer side to detect that no more data is necessary?
Is there simpler way for consumer to tell "enough" to push back so producer stops also gracefully?
I found out that clojure.core.async/put! does not block and allows to avoid unnecessary blocking. Are there disadvantages of using it instead of clojure.core.aasync/>!?
closing chans frees all who are reading from them them, and leaves writers blocked
here is the reading case (where it works nicely):
user> (def a-chan (async/chan))
#'user/a-chan
user> (future (async/<!! a-chan)
(println "continuting after take"))
#future[{:status :pending, :val nil} 0x5fb5a025]
user> (async/close! a-chan)
nil
user> continuting after take
And here is a test of the writing case where, as you say, draining it may be a good idea:
user> (def b-chan (async/chan))
#'user/b-chan
user> (future (try (async/>!! b-chan 4)
(println "continuting after put")
(catch Exception e
(println "got exception" e))
(finally
(println "finished in finally"))))
#future[{:status :pending, :val nil} 0x17be0f7b]
user> (async/close! b-chan)
nil
I don't find any evidence of the stuck writer unblocking here when the chan is closed
This behavior is intended, since they explicitly state it in the docs!
In your case, do (while (async/poll! c)) after closing channel c to release all blocked/parked (message sending) threads/go-blocks.
If you want to do anything with the content you can do:
(->> (repeatedly #(async/poll! c))
(take-while identity))
I want to use Clojure core.async to write an application that sits in a loop polling a service, but my attempts so far terminate unexpectedly.
When I run this program it prints the message then exits:
(defn -main
[]
(println "Running forever...?")
(async/go-loop [n 0]
(prn n)
(async/<!! (async/timeout 1000))
(recur (inc n))))
I would like the program to run forever (until the JVM process is killed).
What is the accepted way to achieve this?
The main thread is what will keep the JVM process running (it won't care about the threads in the go pool).
Keep it running by blocking on the main thread. e.g.
(defn -main
[]
(println "Running forever...?")
(async/<!! (async/go-loop [n 0]
(prn n)
(async/<! (async/timeout 1000))
(recur (inc n)))))
Note: you shouldn't use <!! inside of a go block. You should instead use <!.
How many ways are there to block the main thread......?
(defn -main []
(println "Running forever...?")
; go-loop runs forever in another (daemon) thread
(async/go-loop [n 0]
(prn n)
(async/<!! (async/timeout 1000))
(recur (inc n))
; hang the main thread
(Thread/sleep 111222333444)) ; long enough....
(or you could use Long/MAX_VALUE).
If I wrap a function in a future and invoke this from leiningen on the commandline, it adds 1 full minute to the runtime. Can any one tell me why this is? Can you also tell me how to stop this behavior? I'd like to avoid this extra minute.
Example code:
(ns futest.core
(:gen-class))
(defn testme []
(Thread/sleep 2000)
42)
(defn -main
[& args]
(if (= (nth args 0) "a")
;; either run a simple function that takes 2 seconds
(do
(println "no future sleep")
(let [t (testme)]
(println t))
(println "done."))
;; or run a simple function that takes 2 seconds as a future
(do
(println "future sleep")
(let [t (future (testme))]
(println #t))
(println "done.")
;; soo, why does it wait for 1 minute here?
)))
this is because agents uses two threadpools, the first is a fixed threadpool and the second a cached threadpool. cached threadpool terminates running threads that were inactive for a certain duration, the default being 60 seconds. This is the reason you see the 60 seconds delay. Of course, if you manually call shutdown-agents both these threadpools terminate leaving no non-daemon threads that blocks your exit.
As noted in the answer to this question you need to call shutdown-agents at the end of your -main method.
I'm posting this as self-answered Q&A since that question doesn't mention future, so it didn't turn up on my google searches. Sure enough, if I add:
;; soo, why does it wait for 1 minute here?
(shutdown-agents)
)))
the problem goes away.
As I tested, a separate thread is used for each new agent, when I create them.
Could several agents be run in one thread?
My idea is to create 10K+ light-weight agents (like actors in erlang), so is it a challenge for Clojure?
Thanks
This is incorrect. Agents use a thread pool which is the number of core + 2 in size. So on a quad core machine even 10k+ agents will only use 6 worker threads.
With send, that is. With send-off new threads will be started.
Consider using a j.u.c.DelayQueue
Here's a sketch of how it would work,
the (delayed-function is a bit cumbersome here, but it basically constructs an instance of j.u.c.Delayed for submission to the queue.)
(import [java.util.concurrent Delayed DelayQueue TimeUnit])
(defn delayed-function [f]
(let [execute-time (+ 5000 (System/currentTimeMillis))
remaining-delay (fn [t] (.convert t
(- execute-time
(System/currentTimeMillis))
TimeUnit/MILLISECONDS))]
(reify
Delayed (getDelay [_ t] (remaining-delay t))
Runnable (run [_] (f))
Comparable (compareTo [_ _] 0))))
;;; Use java's DelayQueue from clojure.
;;; See http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/DelayQueue.html
(def q (DelayQueue.))
(defn delayed
"put the function f on the queue, it will only execute after the delay
expires"
[f]
(.offer q (delayed-function f)))
(defn start-processing
"starts a thread that endlessly reads from the delay queue and
executes the function found in the queue"
[]
(.start
(Thread.
#(while true
(.run (.take q))))))
user> (start-processing)
user> (delayed #(println "Hello"))
; 5 seconds passes
Hello
the at function of the at-at library that was developed to support the (in my opinion fantastic) Overtone music synthesizer provides a nice clean interfase for running functions at a specific point in time.
(use 'overtone.at-at)
(def my-pool (mk-pool))
(after 1000 #(println "hello from the past!") my-pool)