I'm playing with someone else's code by examining it in the repl.
It keeps calling System/exit, which brings down my repl. This is infuriating.
In all the code I have access to, I've mocked the calls out.
But it's also calling some library code I don't have the source to, both java and clojure, and this occasionally causes exits too.
Is there any way to catch these calls globally, so that an attempt to call them doesn't kill the repl thread? Ideally it would just throw an exception instead.
I think in java I could install a new SecurityManager to get this effect, but I've never done it
there seems to be something in that line here:
http://jroller.com/ethdsy/entry/disabling_system_exit
So I'm thinking something like:
(System/setSecurityManager (SecurityManager.))
only I somehow need to attach
public void checkPermission( Permission permission ) {
if( "exitVM".equals( permission.getName() ) ) {
throw new ExitTrappedException() ;
}
}
My best shot so far is:
(System/setSecurityManager
(proxy [SecurityManager] []
(checkPermission [p]
(when (= "exitVM" (.getName p))
(throw (Exception. "exit"))))))
or maybe
(System/setSecurityManager
(proxy [SecurityManager] []
(checkExit [n] false)))
But they both just destroy the repl
Or is there a better way of doing this?
Use AspectJ and intercept all Calls to System.exit() with a no op.
But you are right, just configuring the security manager would be saner.
You can also use clj-sandbox to restrict code you don't trust.
This one works for me in both, Clojures simple REPL, and in the Lein REPL, too
(def SM (proxy [SecurityManager] []
(checkPermission
[^java.security.Permission p]
(when (.startsWith (.getName p) "exitVM")
(throw (SecurityException. "exit"))))))
(System/setSecurityManager SM)
Ah. Even in the Cider REPL in Emacs.
The name is actually "exitVM.n", where n is the numeric exit code passed to System/exit
I still have an issue to extend this security manager. Surprisingly many Clojure functions call the security manager, and thus, when used inside of it, give an infinite loop, aka StackOverflowException.
(For the latter I opened anothe question: Security Manager in Clojure )
You can try this: http://download.oracle.com/javase/6/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)
Related
I was exploring Clojure java.jmx API Reference and trying the examples mentioned there e.g.
;; Can I serve my own beans? Sure, just drop a Clojure ref
;; into an instance of clojure.java.jmx.Bean, and the bean
;; will expose read-only attributes for every key/value pair
;; in the ref:
(jmx/register-mbean
(create-bean
(ref {:string-attribute "a-string"}))
"my.namespace:name=Value")
It works great, the bean's attribute value is visible in the console but it is read-only.
Is there a way to create a writeable bean(so that it is listed in "Operations" folder and manageable from console)?
It looks like the clojure.java.jmx code supports setAttribute.
(see (deftype Bean ...) in https://github.com/clojure/java.jmx/blob/master/src/main/clojure/clojure/java/jmx.clj
The easiest way seems to be just using an atom instead of a ref.
You could then have atom watchers to run some code if JMX changes it.
Maybe give that a try. I've forgotten most of JMX ;)
EDIT: Quickly tried it. It looks like the attribute is still read-only :(
I gotta look deeper. Nevertheless, the source code is pretty nice and hopefully easy to understand.
EDIT2:
The issue is with build-attribute-info which passes false to the writeable? flag in the `MBeanAttributeInfo.
You can monkey patch that:
(import 'java.jmx.MBeanAttributeInfo)
(require '[clojure.java.jmx :as jmx])
(defn build-my-attribute-info
"Construct an MBeanAttributeInfo. Normally called with a key/value pair from a Clojure map."
([attr-name attr-value]
(build-my-attribute-info
(name attr-name)
(.getName (class attr-value)) ;; you might have to map primitive types here
(name attr-name) true true false)) ;; false -> true compared to orig code
([name type desc readable? writable? is?] (println "Build info:" name type readable? writable?) (MBeanAttributeInfo. name type desc readable? writable? is? )))
;; the evil trick one should only use in tests, maybe
;; overwrite the original definition of build-attribute-info with yours
(with-redefs [jmx/build-attribute-info build-my-attribute-info]
(jmx/register-mbean (jmx/create-bean (atom {:foo "bar"})) "my.bean:name=Foo"))
;; write to it
(jmx/write! "my.bean:name=Foo" "foo" "hello world")
;; read it
(jmx/read "my.bean:name=Foo" "foo")
=> "hello world"
Now, unfortunately, Java Mission Control still can't change the value, but I'm not sure why. The MBean info is correct.
Must be a permission thing.
I've got a situation where I watch a specific directory for filesystem changes. If a certain file in that directory is changed, I re-read it, attach some existing cached information, and store it in an atom.
The relevant code looks like
(def posts (atom []))
(defn load-posts! []
(swap!
posts
(fn [old]
(vec
(map #(let [raw (json/parse-string % (fn [k] (keyword (.toLowerCase k))))]
(<snip some processing of raw, including getting some pieces from old>))
(line-seq (io/reader "watched.json")))))))
;; elsewhere, inside of -main
(watch/start-watch
[{:path "resources/"
:event-types [:modify]
:callback (fn [event filename]
(when (and (= :modify event) (= "watched.json" filename))
(println "Reloading posts.json ...")
(posts/load-posts!)))}
...])
This ends up working fine locally, but when I deploy it to my server, the swap! call hangs about half-way through.
I've tried debugging it via println, which told me
The filesystem trigger is being fired.
swap! is not running the function more than once
The watched file is being opened and parsed
Some entries from the file are being processed, but that processing stops at entry 111 (which doesn't seem to be significantly different from any preceding entries).
The update does not complete, and the old value of that atom is therefore preserved
No filesystem events are fired after this one hangs.
I suspect that this is either a memory issue somewhere, or possibly a bug in Clojure-Watch (or the underlying FS-watching library).
Any ideas how I might go about fixing it or diagnosing it further?
The hang is caused by an error being thrown inside of the function passed as a :callback to watch/start.
The root cause in this case is that the modified file is being copied to the server by scp (which is not atomic, and the first event therefore triggers before the copy is complete, which is what causes the JSON parse error to be thrown).
This is exacerbated by the fact that watch/start fails silently if its :callback throws any kind of error.
The solutions here are
Use rsync to copy files. It does copy atomically but it will not generate any :modify events on the target file, only related temp-files. Because of the way its atomic copy works, it will only signal :create events.
Wrap the :callback in a try/catch, and have the catch clause return the old value of the atom. This will cause load-posts! to run multiple times, but the last time will be on file copy completion, which should finally do the right thing.
(I've done both, but either would have realistically solved the problem).
A third option would be using an FS-watching library that reports errors, such as Hawk or dirwatch (or possibly hara.io.watch? I haven't used any of these, so I can't comment).
Diagnosing this involved wrapping the :callback body with
(try
<body>
(catch Exception e
(println "ERROR IN SWAP!" e)
old))
to see what was actually being thrown. Once that printed a JSON parsing error, it was pretty easy to gain a theory of what was going wrong.
I'm creating a library for an API server, here's a simpliefied version of I have:
(defonce ^:dynamic *my-token* nil)
(defmacro def-my-token
[token1 & body]
`(binding [*my-token* ~token1] ~#body))
And the main "post" method:
(defn my-post-request [url1]
(try
(let [res (client/post (str "api/url/base" url1)
{:body (json/write-str (:secret my-token)) ; my-token should come from the macro
;......
And here's how I want to use it:
(defn -main [& args]
(def-my-token "fdsfdsfdsfds"
; now "my-token" should be created and visible in "my-post-request", shouldn't it?
(print
(my-post-request "/some_end_point"))))
But it says "Unable to resolve symbol: my-token in this context"
I wonder why? doens't def-my-token, being a macros, define it? why not? And how to fix that?
UPDATE:
Also without (defonce ^:dynamic *token* nil) it doesn't work. Why not?
Why isn't defining the macro enough?
Answer to your UPDATE:
According to the documentation for binding, you can only override already existing vars. That's why your solution doesn't work without establishing a root binding to your dynamic var.
Sidenote:
I would recommend doing what jmargolisvt said and use a plain def instead of defonce, as I've never seen any dynamic var definition in the wild using defonce.
EDIT:
doens't def-my-token, being a macros, define it? why not? And how to fix that?
Macros by themselves don't define things, they are small programs transforming your source code in the macro-expansion step of most Lisp REPL's. It could define anything you want it to, but then you should've wrote the def special form. What you used instead was binding which deals with already existing vars.
You might get more insight by toying with it in the REPL and/or reading the answer of this stackoverflow answer.
If you need some further explanation why overriding is needed:
It's practical to conceptualize vars as stacks. The root binding that you establish with using def is the first layer. Everything in your program will see this value unless you put "something" over it. As you can imagine in your example having *my-token* seen as nil from your functions would cause issues.
binding to the resuce!
It allows you put anything "on top" of the root binding (in your case nil) thread-locally inside of the body it, like so:
you bound *my-token*, not my-token. Try:
{:body (json/write-str (:secret *my-token*))
The asterisks are just a naming convention for dynamic vars, they are still part of the actual var name.
I have a rabbitMQ connection that seems to be started at compile time (when I type lein compile) and then blocks the building of my project. Here are more details on the problem. Let us say this is the clojure file bla_test.clj
(import (com.rabbitmq.client ConnectionFactory Connection Channel QueueingConsumer))
;; And then we have to translate the equivalent java hello world program using
;; Clojure's excellent interop.
;; It feels very strange writing this sort of ceremony-oriented imperative code
;; in Clojure:
;; Make a connection factory on the local host
(def connection-factory
(doto (ConnectionFactory.)
(.setHost "localhost")))
;; and get it to make you a connection
(def connection (.newConnection connection-factory))
;; get that to make you a channel
(def channel (. connection createChannel))
;;HERE I WOULD LIKE TO USE THE SAME CONNECTION AND THE SAME CHANNEL INSTANCE AS OFTEN AS
;; I LIKE
(dotimes [ i 10 ]
(. channel basicPublish "" "hello" nil (. (format "Hello World! (%d)" i) getBytes)))
The clojure file above is part of a bigger clojure program that I build using lein. My problem is that when I compile with "lein compile", a connection is done because of the line (def connection (.newConnection connection-factory)) and then the compilation is stopped! How can I avoid this? Is there a way to compile without building connection? How can I manage to use the same instance of channel over several calls coming from external components?
Any help would be appreciated.
Regards,
Horace
The Clojure compiler must evaluate all top-level forms, because it can be required to run arbitrary code when expanding calls to macros.
The usual solution to issues like the one you describe is to define a top-level Var holding an object of a dereferenceable type, for example an atom or a promise, and have an initialization function provide the value at runtime. (You could also use a delay and specify the value inline; this is less flexible, since it makes it more difficult to use a different value for testing etc.)
I'm confused by how calls with carmine should be done. I found the wcar macro described in carmine's docs:
(defmacro wcar [& body] `(car/with-conn pool spec-server1 ~#body))
Do I really have to call wcar every time I want to talk to redis in addition to the redis command? Or can I just call it once at the beginning? If so how?
This is what some code with tavisrudd's redis library looked like (from my toy url shortener project's testsuite):
(deftest test_shorten_doesnt_exist_create_new_next
(redis/with-server test-server
(redis/set "url_counter" 51)
(shorten test-url)
(is (= "1g" (redis/get (str "urls|" test-url))))
(is (= test-url (redis/get "shorts|1g")))))
And now I can only get it working with carmine by writing it like this:
(deftest test_shorten_doesnt_exist_create_new_next
(wcar (car/set "url_counter" 51))
(shorten test-url)
(is (= "1g" (wcar (car/get (str "urls|" test-url)))))
(is (= test-url (wcar (car/get "shorts|1g")))))
So what's the right way of using it and what underlying concept am I not getting?
Dan's explanation is correct.
Carmine uses response pipelining by default, whereas redis-clojure requires you to ask for pipelining when you want it (using the pipeline macro).
The main reason you'd want pipelining is for performance. Redis is so fast that the bottleneck in using it is often the time it takes for the request+response to travel over the network.
Clojure destructuring provides a convenient way of dealing with the pipelined response, but it does require writing your code differently to redis-clojure. The way I'd write your example is something like this (I'm assuming your shorten fn has side effects and needs to be called before the GETs):
(deftest test_shorten_doesnt_exist_create_new_next
(wcar (car/set "url_counter" 51))
(shorten test-url)
(let [[response1 response2] (wcar (car/get (str "urls|" test-url))
(car/get "shorts|1g"))]
(is (= "1g" response1))
(is (= test-url response2))))
So we're sending the first (SET) request to Redis and waiting for the reply (I'm not certain if that's actually necessary here). We then send the next two (GET) requests at once, allow Redis to queue the responses, then receive them all back at once as a vector that we'll destructure.
At first this may seem like unnecessary extra effort because it requires you to be explicit about when to receive queued responses, but it brings a lot of benefits including performance, clarity, and composable commands.
I'd check out Touchstone on GitHub if you're looking for an example of what I'd consider idiomatic Carmine use (just search for the wcar calls). (Sorry, SO is preventing me from including another link).
Otherwise just pop me an email (or file a GitHub issue) if you have any other questions.
Don't worry, you're using it the correct way already.
The Redis request functions (such as the get and set that you're using above) are all routed through another function send-request! that relies on a dynamically bound *context* to provide the connection. Attempting to call any of these Redis commands without that context will fail with a "no context" error. The with-conn macro (used in wcar) sets that context and provides the connection.
The wcar macro is then just a thin wrapper around with-conn making the assumption that you will be using the same connection details for all Redis requests.
So far this is all very similar to how Tavis Rudd's redis-clojure works.
So, the question now is why does Carmine need multiple wcar's when Redis-Clojure only required a single with-server?
And the answer is, it doesn't. Apart from sometimes, when it does. Carmine's with-conn uses Redis's "Pipelining" to send multiple requests with the same connection and then package the responses together in a vector. The example from the README shows this in action.
(wcar (car/ping)
(car/set "foo" "bar")
(car/get "foo"))
=> ["PONG" "OK" "bar"]
Here you will see that ping, set and get are only concerned with sending the request, leaving the receiving of response up to wcar. This precludes asserts (or any result access) from inside of wcar and leads to the separation of requests and multiple wcar calls that you have.