How to set a timeout on a Hystrix command in Clojure? - clojure

I'm learning about Hystrix and Clojure and don't understand how to (properly) set a timeout on a Hystrix command in Clojure.
I searched StackOverflow and the web more generally. I looked at Hystrix's Clojure wrapper source code (https://github.com/Netflix/Hystrix/blob/master/hystrix-contrib/hystrix-clj/src/main/clojure/com/netflix/hystrix/core.clj). There is a init-fn function parameter that looked promising, but the comments seem to suggest that this won't be a sustainable solution. But would this be a simple start?
I have a ridiculously simple Hystrix command running in Clojure and would appreciate help in extending this to set, say, a 200ms timeout:
(ns hystrix-timeout.core
(:require [clojure.string :as str])
(:require [com.netflix.hystrix.core :as hystrix])
(:gen-class))
(defn my-primary [a]
(if (= a true) (throw (Exception. "Primary failed")) (str "primary: " a)))
(defn my-fallback [a]
(str "fallback: " a))
(hystrix/defcommand my-command
{:hystrix/fallback-fn my-fallback}
[a]
(my-primary a))
(defn -main
"Executes a simple Hystrix command. Will use a timeout when I know how to do this in Clojure."
[& args]
(println (my-command false))
(println (my-command true))
(System/exit 0) ; Exit explicitly as Hystrix threads are still running.
)
I've put my lein project up at https://github.com/oliverbaier/learning/tree/master/hystrix-timeout in case this makes answering easier.
Thanks a lot,
Oliver

The simplest route is to just use System properties. You can set either a global default:
(System/setProperty "hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds" "200")
or a command specific value:
(System/setProperty "hystrix.command.my-command.execution.isolation.thread.timeoutInMilliseconds" "200")
You can perform this in your -main method. If you are running a real web-app sans -main, you can add a ring init in project.clj :ring {:handler your-handler :init your-config-method} your-config-method will be invoked on startup.

Related

Clojure load-file on repl and call -main with arguments

I've done some coding in clojure inside core.clj which have a -main method that can take 0 or more arguments:
(defn -main
"dequeue function"
[& args]
I'm loading this clj file with:
(load-file "src/exercise/core.clj")
And then i'm trying to learn how to call this within repl in order to develop core_test.clj (if there is any tips about how to develop this auto tests, please give me some hints as well). What I'm trying to do now is:
(-main "resources\\sample-input.json" "a")
But this is printing "Arguments 0" which is a message that I told the code to print just to see how many arguments are being passed with
(println "Arguments" (count *command-line-args*))
How am I suposed to do this?
Thanks!
i'm trying to learn how to call this within repl in order to develop core_test.clj
Usually you'd write other functions that are called from -main and test those rather than the app's entry point -main.
But you should be able to call -main like any other function. I have a src/sandbox/main.clj file:
(ns sandbox.main)
(defn -main [& args]
(prn args))
Starting a REPL in the project folder, I can call -main like this:
(use 'sandbox.main) ;; (load-file "src/sandbox/main.clj") also works
=> nil
(in-ns 'sandbox.main)
=> #object[clojure.lang.Namespace 0x67ccce04 "sandbox.main"]
(-main "some" "args")
;; ("some" "args")
=> nil
There's a key difference in my example though: it's printing -main's args binding; *command-line-args* is nil because you're not running the code from the command line with args, you're running it from a REPL.
Regardless, it's probably a better idea to use an existing library to work with CLI args rather than *command-line-args* directly.

Why does lein run hang?

I have created a Leiningen project for Exercise 2 from here. My code looks like this:
(ns random-quotes.core
(:require [clojure.string :as str])
(:gen-class))
(defn word-count [s]
(frequencies (str/split (first (str/split s #"\n")) #"\s")))
(def quote-url "http://www.braveclojure.com/random-quote")
(def total-word-count (atom {}))
(defn update-word-count []
(future
(swap! total-word-count
(partial merge-with +)
(word-count (slurp quote-url)))))
(defn quote-word-count [n]
(doseq [quote-future (doall (repeatedly n update-word-count))]
#quote-future)
#total-word-count)
(defn -main [n]
(doseq [entry (sort-by val (quote-word-count (bigdec n)))]
(println entry)))
All pretty straightforward. When I run, e.g., (-main 5) in lein repl, it runs, prints, and returns as expected. However, when I try lein run 5 instead, it runs and prints but never exits, so I am forced to use Ctrl+C to get my terminal back.
Any idea why this happens?
Clojure has a thread pool that it keeps running for use by the agents. Because those threads are still alive, the JVM can't tell that you're program is done. It's just sitting there waiting for the agents to exit. You can make them finish by calling (shutdown-agents) at the end of your program as described here. Futures use agents.
clojure.core/future-call calls an agent like this:
(let [f (binding-conveyor-fn f)
fut (.submit clojure.lang.Agent/soloExecutor ^Callable f)]
which actually starts your code running. You would not be the only one to voice some criticism of this, and we all hope a more elegant solution is found.

Simple(st) Datomic test program hangs at exit

The following program, when run from an überjar, exits at the end only when using the in-memory Datomic database; when connecting to the Datomic server, it hangs indefinitely rather than exiting the JVM:
(ns myns.example
(:use [datomic.api :only [db q] :as d])
(:gen-class))
;; WORKS: (def uri "datomic:mem://testdb")
(def uri "datomic:free://localhost:4334/testdb2")
(defn -main []
(println 1)
(when (d/create-database uri)
(d/connect uri))
(shutdown-agents)
(println 2))
Run as:
lein uberjar && java -cp target/myns-0.1.0-SNAPSHOT-standalone.jar myns.example
Outputs:
1
2
and hangs. It only hangs if the DB doesn't exist when the program starts.
Anyone know why, or how to fix? This is with both datomic-free-0.8.4020.26 and datomic-free-0.8.3941.
UPDATE -- the above program does actually terminate, but it takes a very long time (> 1 minute). I'd like to know why.
shutdown-agents takes up to one minute to complete (assuming no agents are running an action).
This is due to the way java.util.concurrent cached thread pools work.
Use datomic.api/shutdown
shutdown
function
Usage: (shutdown shutdown-clojure)
Shut down all peer
resources. This method should be called as part of clean shutdown of
a JVM process. Will release all Connections, and, if shutdown-clojure
is true, will release Clojure resources. Programs written in Clojure
can set shutdown-clojure to false if they manage Clojure resources
(e.g. agents) outside of Datomic; programs written in other JVM
languages should typically set shutdown-clojure to true.
Added in Datomic Clojure version 0.8.3861
(ns myns.example
(:require [datomic.api :as d])
(:gen-class))
(def uri "datomic:free://localhost:4334/testdb2")
(defn -main []
(d/create-database uri)
(let [conn (d/connect uri)]
(try
;; do something
(finally (d/shutdown true)))

clojure.tools.logging EvalReader use?

I think this is a bug in clojure/tools.logging. I have the following db.clj file. What it does is unimportant. The important part is that for safety I have disabled the *read-eval*. I invoke db/start with no problems. However, if I uncomment the #_(log/info "Failed to bootstrap") form, it throws an EvalReader not allowed error. I've tried all sorts of combinations for the log/info call. If it's outside of the try block, it's fine. Inside the try block anywhere, whether it's in the body, catch, or finally, it raises this exception. However, when I wrap a try around log/info elsewhere, it's fine.
What gives?
(ns extenium.db
(:require [clojure.tools.logging :as log]
[clojure.java.io :as io])
(:import com.thinkaurelius.titan.core.TitanGraph
com.thinkaurelius.titan.core.TitanFactory))
(def ^:private
sentinel- (Object.))
(def ^:private
db- (atom nil))
...
(defn start [^String path]
(locking sentinel-
(log/info "Starting database at path" path)
(let [exists (.exists (io/file path))
^TitanGraph db_ (TitanFactory/open path)]
(if exists
(log/info "Path" path "exists")
(log/info "Path" path "does not exist"))
(log/info "Starting database engine")
(swap! db- (constantly db_))
(log/info "Started database engine")
(if (not exists)
(try
(bootstrap-)
(catch Throwable t
#_(log/info "Failed to bootstrap")
(stop)
(.delete (io/file path))
(throw t)))))
(log/info "Started database")
true))
EDIT: Trimmed down code per #alex-taggart. bootstrap- implementation not shown. I had originally included everything because this seemed like a context-specific bug and I felt it was safer to provide as much context as possible.
EDIT: Per #chouser, added how I'm disabling *read-eval*. This is the template that is generated by lein new app.
(defn -main
"The main entry point into Extenium."
[& args]
;; Prevent arbitrary eval injection
(alter-var-root #'*read-eval* (constantly false))
;; Initialize system settings from the command line and configuration file
(init!- args)
;; Start the service
(start!-))
It's not a bug, really. The clojure.tools.logging library is just an abstraction upon other Java logging facilities. To discover which one is available, it uses eval expression. You're welcome to check for your own: here is a quick search result and a certain file where it gets used.
In your case, I believe it's not necessary to care about the global read-eval. This is an internal feature and who knows what other libraries depend on it. If you validate the user input and prevent it from being evaluated, it's fine to leave the flag as is. I would say, SQL injections and XSS are the things you should be worried about first.

Why does read-line not return after hitting ENTER (seems like a hang) using lein run, but works with lein repl?

The problem at hand is that when I run my program with lein run it gets to the (read-line) part and I can't get out of it, meaning: read-line never returns.
Here is the relevant code:
(def command (atom ""))
(defn print-prompt []
(print "prompt> ")
(flush)
)
(defn ask-for-input []
(print-prompt)
(let [x (str (read-line))]
(println (str "User input: " x))
(reset! command x)
)
)
I never see the "User input: " string on screen.
The strange part is, if I run lein repl and call (ask-for-input) then it works correctly :S
Try lein trampoline run, it works.
The following is from leiningen FAQ:
Q: I don't have access to stdin inside my project.
A: This is a limitation of the JVM's process-handling methods; none of them expose stdin correctly. This means that functions like read-line will not work as expected in most contexts, though the repl task necessarily includes a workaround. You can also use the trampoline task to launch your project's JVM after Leiningen's has exited rather than launching it as a subprocess.
I tried your source code, but omitted the flush. It worked without a problem. What version of Clojure are you using? I tried the following code with Clojure 1.3.
(def command (atom 0))
(defn print-prompt []
(print "prompt> ")
)
(defn ask-for-input
[]
(print-prompt)
(let [x (str (read-line))]
(println (str "User input: " x))
(reset! command x)
))
Edit:
I altered one of your functions that I copied and tested with, and it works now with standalone and lein run. You had (flush) in your original example.
(defn print-prompt []
(print "prompt> ")
(flush)
)
From what I can garner, println causes a flush, print doesn't, and you need a flush after print.
I am adding this information in case it might be of help. I have a Clojure project called repl-test. Here is my repl-test project's core.clj file header. Your source, already posted, is in this file with some other functions, not related to your post.
(ns repl-test.core
(:gen-class)
(:use clojure.contrib.command-line)
(:require [clojure.contrib.string :as cstr])
(:require [clojure.contrib.trace :as ctr])
(:require [clojure.string :as sstr])
(:use clojure-csv.core))
And here is the project.clj file:
(defproject repl-test "0.0.1-SNAPSHOT"
:description "TODO: add summary of your project"
:dependencies [[org.clojure/clojure "1.3.0"]
[org.clojure/clojure-contrib "1.2.0"]
[clojure-csv/clojure-csv "1.2.4"]
[org.clojure/tools.cli "0.1.0"]
[clj-http "0.1.3"]]
:aot [repl-test.core]
:main repl-test.core)