(def queue-agent (agent (clojure.lang.PersistentQueue/EMPTY)))
(send queue-agent conj "some data for the queue")
(println "test output")
If I run this code, after a couple (!) of seconds the console will output test output and then nothing happens (program is not terminating). I've just checked against a couple of sources that all said the send function is asynchronous and should return immediately to the calling thread. So what's wrong with this? Why is it not returning? Is there something wrong with me? Or with my environment?
So you have two issues: long startup time, and the program does not exit.
Startup: Clojure does not do any tree shaking. When you run a Clojure program, you load and bootstrap the compiler, and initialize namespaces, on every run. A couple of seconds sounds about right for a bare bones Clojure program.
Hanging: If you use the agent thread pool, you must run shutdown-agents if you want the program to exit. The vm simply doesn't know it is safe to shut them down.
Related
I'm trying to use lwjgl with clojure for game developement.
My first step is trying to display something on an OpenGL screenfrom the REPL. After launching the repl with lein repl this is what I have done so far:
(import org.lwjgl.opengl GL11 Display DisplayMode
(Display/setDisplayMode (DisplayMode. 800 600))
(Display/create) ; This shows a black 800x600 window as expected
(GL11/glClearColor 1.0 0.0 0.0 1.0)
(GL11/glClear (bit-or GL11/GL_COLOR_BUFFER_BIT GL11/GL_DEPTH_BUFFER_BIT))
(Display/update)
Note that this works, if done quick enough. But after a while (even if I just wait) I start getting errors about the current OpenGL context not being bound to the current thread.
(Display/update)
IllegalStateException No context is current org.lwjgl.opengl.LinuxContextImplementation.swapBuffers (LinuxContextImplementation.java:72)
(GL11/glClear ...)
RuntimeException No OpenGL context found in the current thread. org.lwjgl.opengl.GLContext.getCapabilities (GLContext.java:124)
But maybe the most intriguing of the errors happens when I try to call Display/destroy
(Display/destroy)
IllegalStateException From thread Thread[nREPL-worker-4,5,main]: Thread[nREPL-worker-0,5,] already has the context current org.lwjgl.opengl.ContextGL.checkAccess (ContextGL.java:184)
It all looks as if the repl randomly spawned another thread after some time of inactivity. As I've been able to read, LWJGL only lets you make OpenGL calls from the thread from where it originally was created, so I bet this is causing those errors.
But how could the REPL be randomly switching threads? And especially if I'm not doing anything, just waiting.
It's a known issue already reported against nREPL project (and discussed on Clojure Google Group). It seems that nREPL uses thread pool which terminates idle threads (probably according to keepalive setting).
Until it's fixed you can use a workaround for this (a bit awkward, I admit):
(import '(java.util.concurrent Executors))
(def opengl-executor (Executors/newSingleThreadExecutor))
(defmacro with-executor [executor & body]
`(.submit ~executor (fn [] ~#body)))
(on-executor opengl-executor
(println (.getId (Thread/currentThread))))
By using your own executor, all code wrapped in on-executor will be executed in its thread. newSingleThreadExecutor creates one single thread which according to doc will replace it only when the current one fails due to exception. When you try to execute the last expression with long delays, the printed thread ID should remain the same.
Remember that you should shutdown the executor when stopping your application.
I'm trying to retrieve data from my Kafka 0.8.1 cluster. I have brought into existence an instance of ZookeeperConsumerConnector and then attempt to call createMessageStreams on it. However, no matter what I do, it seems createMessageStreams just hangs and never returns, even if it is the only thing I have done with Kafka.
Reading mailing lists it seems this can sometimes happen for a few reasons, but as far as I can tell I haven't done any of those things.
Further, I'll point out that I'm actually doing this in Clojure using clj-kafka, but I suspect clj-kafka is not the issue because I have the problem even if I run this code:
(.createMessageStreams
(clj-kafka.consumer.zk/consumer {"zookeeper.connect" "127.0.0.1:2181"
"group.id" "my.consumer"
"auto.offset.reset" "smallest"
"auto.commit.enable" "false"})
{"mytopic" (int 1)})
And clj-kafka.consumer.zk/consumer just uses Consumer.createJavaConsumerConnector to create a ZookeeperConsumerConnector without doing anything too fancy.
Also, there are definitely messages in "mytopic" because from the command line I can run the following and get back everything I've already sent to the topic:
% kafka-console-consumer.sh --zookeeper 127.0.0.1:2181 --topic mytopic --from-beginning
So it's also not that the topic is empty.
Feeling stumped at this point. Ideas?
ETA: By "hang" I guess what I really mean is that it seems to spin up a thread and then stay stuck in it never doing anything. If I run this code from the REPL I can get out of it by hitting control-c and then I get this error:
IllegalMonitorStateException java.util.concurrent.locks.ReentrantLock$Sync.tryRelease (ReentrantLock.java:155)
I was experiencing the same issue with the same exception when interrupting the REPL. The reason it hangs is due to the lazy-iterate function in the consumer.zk namespace. The queue from which messages are read is a LinkedBlockingQueue and the call to .hasNext in the lazy-iterate function calls .take on this queue. This creates a read lock on the queue and will block and wait until something is available to take off the queue. This means that the lazy-iterate function will never actually return. lazy-iterate is called by the 'messages' function and if you don't do something like
(take 2 (messages "mytopic" some-consumer))
then the messages function will never return and hang indefinitely. It's my opinion that this is a bug (or design flaw) in clj-kafka. To illustrate that this is indeed what's happening, try setting "consumer.timeout.ms" "0" in your consumer config. It will throw a TimeoutExpection and return control to the REPL.
This further creates a problem with the 'with-resource' macro. The macro takes a binding to a consumer, a shutdown function, and a body; it calls the body and then the shutdown fn. If inside the body, you make a call to 'messages', the body will never return and thus the shutdown function will never be called. The messages function WOULD return if shutdown was called, because shutdown puts a message on the queue that signals the consumer to clean up its resources and threads in preparation for GC. This macro puts the application into a state where the only way to get out of the main loop is to kill the application (or a thread calling it) itself. The library certainly has a ways to go before it's ready for a production environment.
The scenario I try to resolve is a s follows, I have a testing program that makes a web to a web endpoint on a system.
This test program has a jetty web server running on which it expects a callback from the external system that completes a successful test cycle. In case that the callback is not received during an specific time range (timeout), the test fails.
To achieve this, I want the test runner to wait on an "event" that the jetty handler will set upon callback.
I thought about using java's CyclicBarrier but I wonder if there is an idiomatic way in clojure to solve this.
Thanks
You can use promise you asked about recently :) Something like this:
(def completion (promise))
; In test runner.
; Wait 5 seconds then fail.
(let [result (deref completion 5000 :fail)]
(if (= result :success)
(println "Great!")
(println "Failed :(")))
; In jetty on callback
(deliver completion :success)
In straight Clojure, using an agent that tracks outstanding callbacks would make sense, though in practice I would recommend using Aleph, which is a library for asynchronous web programming that makes even driven handlers rather easy. It produces ring handlers, which sounds like it would fit nicely with your existing code.
I have a simple single threaded Clojure program that creates a temp file for swapping data. When the program exits normally this file is deleted, however when the program is exited via Ctrl+C, Ctrl+D, or Ctrl+Z that bit of code never executes. I need it to execute regardles sof how the program exits. I know that I need to catch that signal (I've done this before in other languages), but I can't seem to figure out how to do it in Clojure.
I don't know if Clojure has wrapped method for that purpose. In java, you can use Runtime.addShutdownHook()
Registers a new virtual-machine shutdown hook.
The Java virtual machine shuts down in response to two kinds of events:
The program exits normally, when the last non-daemon thread exits or when the exit (equivalently, System.exit) method is invoked, or
The virtual machine is terminated in response to a user interrupt, such as typing ^C, or a system-wide event, such as user logoff or system shutdown.
Update
Here is a simple demo
(.addShutdownHook (Runtime/getRuntime) (Thread. (fn [] (println "Shutting down..."))))
user=> ;; Ctrl-C
Shutting down...
Have a look at the deleteOnExit method in the java.io.File:
(import '(java.io File))
(doto (File/createTempFile "foo" nil nil) (.deleteOnExit))
I have a multithreaded application written in Clojure. There is a problem of making a text in the console display correctly when multiple threads write to STDOUT. How can I do this correctly in Clojure, so the lines won't look interlaced? I think this would involve some kind of separate IO agent, but I'm not really sure how to do that.
I think this would involve some kind of separate IO agent
Yes, that should work. Create an agent (def printer (agent nil)) and call it with the appropriate print statement, e.g, (send printer #(println msg)). The messages are put in a queue and are executed (asynchronously) one at a time.
For logging purposes you could also look at tools.logging which uses agents under the hood.