I've installed samestep/boot-refresh 0.1.0. In the boot REPL, when I change a source file and type:
boot.user=> (boot (refresh))
I get:
java.lang.IllegalStateException: Can't set!: *e from non-binding thread
What am I doing wrong?
Here's the full stack trace:
boot.user=> *e
#error {
:cause "Can't set!: *e from non-binding thread"
:via
[{:type java.lang.IllegalStateException
:message "Can't set!: *e from non-binding thread"
:at [clojure.lang.Var set "Var.java" 218]}]
:trace
[[clojure.lang.Var set "Var.java" 218]
[clojure.tools.namespace.repl$print_and_return invokeStatic "repl.clj" 22]
[clojure.tools.namespace.repl$print_and_return invoke "repl.clj" 20]
[clojure.tools.namespace.repl$do_refresh invokeStatic "repl.clj" 96]
[clojure.tools.namespace.repl$do_refresh invoke "repl.clj" 82]
[clojure.tools.namespace.repl$refresh invokeStatic "repl.clj" 145]
[clojure.tools.namespace.repl$refresh doInvoke "repl.clj" 128]
[clojure.lang.RestFn invoke "RestFn.java" 397]
[samestep.boot_refresh$eval541$fn__542$fn__547$fn__548$fn__549 invoke "boot_refresh.clj" 14]
[clojure.lang.AFn applyToHelper "AFn.java" 152]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 646]
[clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1881]
[clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1881]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[samestep.boot_refresh$eval541$fn__542$fn__547$fn__548 invoke "boot_refresh.clj" 13]
[boot.core$run_tasks invoke "core.clj" 1019]
[boot.core$boot$fn__918 invoke "core.clj" 1029]
[clojure.core$binding_conveyor_fn$fn__4676 invoke "core.clj" 1938]
[clojure.lang.AFn call "AFn.java" 18]
[java.util.concurrent.FutureTask run "FutureTask.java" 266]
[java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1142]
[java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 617]
[java.lang.Thread run "Thread.java" 745]]}
My own code isn't mentioned in the stack trace. I have seen (boot (refresh)) work from the REPL before, but I haven't been able to find out what I'm doing differently to cause this error.
Update 1
After a long binary search with many print statements—since, as explained in #amalloy's answer, the stack trace for the real exception is inaccessible—I discovered this:
In a namespace called move-test, this statement:
(def subset-sum-spec fargish.workspace-test/subset-sum-spec)
was causing (boot (refresh)) to fail. When I replaced it with:
(:require … [fargish.workspace-test :refer [subset-sum-spec]] …)
in the ns statement, (boot (refresh)) worked again. That fixes the current problem for now, but I'd still like to know what's going on.
Update 2
Continuing to try to get (boot (refresh)) to work, it's become clear that the problem is different every time. #amalloy's answer suggests that the real answer to this question would be some way to find the exception and stack trace that c.t.n.repl/print-and-return is failing to store in *e. I've tried a few ideas, but the relevant variables seem to be private and hard to dig out.
How can you find out the error that causes (boot (refresh)) to fail?
I don't know about boot or c.t.namespace, but that error trace looks like boot is running refresh from a new thread, and refresh is running into some error. It tries to propagate the error for you by setting *e, but it can't set *e because it's not on the repl thread. So instead of seeing the real error, you're getting this useless "error caused by failing to report an error".
The error seems to be detected here, and c.t.namespace tries to avoid setting *e when there's no repl running (by checking whether *e is bound), but it incorrectly assumes that if a repl is running then the repl thread must be the one it's being called from, whereas apparently boot calls it from a different thread. Have you tried just calling (refresh)? I don't know what the (boot ...) wrapper is supposed to do, but you might not need it, and it seems to be causing trouble. This also explains why you've seen (boot (refresh)) work: the (boot ...) wrapper (probably) doesn't break things itself, but instead just causes the error reporting to get worse when something else is broken.
Of course, once you fix that issue, the result will not be that your refresh works: you'll just be able to see the actual error, instead of this meta-error! Hopefully that will be enough to help you make progress.
The repl usually has *e bound in the environment of where you execute your code. If anything that needs *e is ran in another thread or outside of the repl it will fail to have an *e to bind to.
I have by-passed this issue by using with-binding to ensure that the refresh fn will always have *eto bind the error to.
(with-bindings {#'*e nil} (refresh))
Once refresh has *e to bind the error to, it should prn the error for you.
Related
I'm trying to retrieve a URL. At the time of writing: https://upvote.pub/
Using the java.net.URL, I can retrieve this fine:
=> (count (slurp (.openStream (java.net.URL. "https://upvote.pub/"))))
44353
Ditto clj-http:
=> (count (:body (clj-http.client/get "https://upvote.pub/")))
44353
But when I use HTTPKit I run into problems:
=> #(http/get "https://upvote.pub/")
{:opts {:method :get, :url "https://upvote.pub/"}, :error #error {
:cause "No status"
:via
[{:type org.httpkit.ProtocolException
:message "No status"
:at [org.httpkit.client.RespListener onCompleted "RespListener.java" 126]}]
:trace
[[org.httpkit.client.RespListener onCompleted "RespListener.java" 126]
[org.httpkit.client.Request finish "Request.java" 51]
[org.httpkit.client.HttpClient doRead "HttpClient.java" 156]
[org.httpkit.client.HttpClient run "HttpClient.java" 426]
[java.lang.Thread run "Thread.java" 745]]}}
This site uses Let's Encrypt. However, other Let's Encrypt sites seem to work:
=> (-> "https://letsencrypt.org/" http/get deref :body count)
21490
The certificate has different properties, but it's from the same authority.
Furthermore, if I include :insecure? true, the error still happens.
I thought this might be related to this HTTPS + proxy bug, as the symptoms are similar and Google App Engine seems to be load-balancing the site. However, this fix was rolled into release 2.2.0.
Is this a bug with HTTP-Kit? Can I work round it?
openjdk version "1.8.0_92-internal"
Clojure version 1.8.0
http-kit version "2.2.0"
It looks like this was caused by SNI (Server Name Identification). I used HTTP Kit version “2.3.0-alpha5” and followed this blog post. It now works.
I use humane-test-output defined in my ~/.lein/profiles.clj like so:
{:user {:injections [(require 'pjstadig.humane-test-output)
(pjstadig.humane-test-output/activate!)]}}
However some old Clojure projects don't function well with humane-test-output active (and also sometimes I just want standard clojure.test output).
I thought I could occasionally disable humane-test-output by adding a new profile like:
{:user {:injections [(require 'pjstadig.humane-test-output)
(pjstadig.humane-test-output/activate!)]}
:nohumane {:injections ^:replace []}}
So I that I can easily invoke this whenever I need to like:
lein with-profile nohumane test
However when I do this I see:
Caused by: java.lang.ClassNotFoundException: leiningen.core.injected
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at clojure.lang.DynamicClassLoader.findClass(DynamicClassLoader.java:69)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at clojure.lang.DynamicClassLoader.loadClass(DynamicClassLoader.java:77)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
Which I think is caused the by fact that I have removed this essential injection that is part of lein's core and needs to be active in lein's test profile.
So my question is, how can I remove the humane-test-output injection in a profile but leave this core injection intact? Note, leiningen.core.project/hooke-injection is also private, and I have so far been unable to refer to it, even using the #' trick.
I've got a situation where I watch a specific directory for filesystem changes. If a certain file in that directory is changed, I re-read it, attach some existing cached information, and store it in an atom.
The relevant code looks like
(def posts (atom []))
(defn load-posts! []
(swap!
posts
(fn [old]
(vec
(map #(let [raw (json/parse-string % (fn [k] (keyword (.toLowerCase k))))]
(<snip some processing of raw, including getting some pieces from old>))
(line-seq (io/reader "watched.json")))))))
;; elsewhere, inside of -main
(watch/start-watch
[{:path "resources/"
:event-types [:modify]
:callback (fn [event filename]
(when (and (= :modify event) (= "watched.json" filename))
(println "Reloading posts.json ...")
(posts/load-posts!)))}
...])
This ends up working fine locally, but when I deploy it to my server, the swap! call hangs about half-way through.
I've tried debugging it via println, which told me
The filesystem trigger is being fired.
swap! is not running the function more than once
The watched file is being opened and parsed
Some entries from the file are being processed, but that processing stops at entry 111 (which doesn't seem to be significantly different from any preceding entries).
The update does not complete, and the old value of that atom is therefore preserved
No filesystem events are fired after this one hangs.
I suspect that this is either a memory issue somewhere, or possibly a bug in Clojure-Watch (or the underlying FS-watching library).
Any ideas how I might go about fixing it or diagnosing it further?
The hang is caused by an error being thrown inside of the function passed as a :callback to watch/start.
The root cause in this case is that the modified file is being copied to the server by scp (which is not atomic, and the first event therefore triggers before the copy is complete, which is what causes the JSON parse error to be thrown).
This is exacerbated by the fact that watch/start fails silently if its :callback throws any kind of error.
The solutions here are
Use rsync to copy files. It does copy atomically but it will not generate any :modify events on the target file, only related temp-files. Because of the way its atomic copy works, it will only signal :create events.
Wrap the :callback in a try/catch, and have the catch clause return the old value of the atom. This will cause load-posts! to run multiple times, but the last time will be on file copy completion, which should finally do the right thing.
(I've done both, but either would have realistically solved the problem).
A third option would be using an FS-watching library that reports errors, such as Hawk or dirwatch (or possibly hara.io.watch? I haven't used any of these, so I can't comment).
Diagnosing this involved wrapping the :callback body with
(try
<body>
(catch Exception e
(println "ERROR IN SWAP!" e)
old))
to see what was actually being thrown. Once that printed a JSON parsing error, it was pretty easy to gain a theory of what was going wrong.
I am having trouble to use test.check together with normal tests. I tried the whole day to figure out what is going on, but I am still not sure.
This is what I have now:
(deftest user-can-only-be-inserted-once
(;normal test))
(defspec insert-one-user-should-let-me-retrieve-that-one-user
gen-quantity
(prop/for-all ; test.check test))
Now, as long as I execute "lein test" from the console after each change in the test file it works as expected. However, running tests from the cursive repl or with the quickie plugin via "lein quickie" the repl will reload the namespaces not correct.
The effect is that, whenever a test fails and I fix that test it will still fail in the repl, so it looks like something is not working correctly there.
If anyone has an idea how to fix this, I'd be happy to hear it. I would also like to hear if someone has a different setup which combines normal tests and test.check or if this is not intended to work at all.
Thanks,
Sven
Update Actually I found out that the problem is not the combination of normal and test.check but the fact that I am trying to check an assertion is throw on a specific case. In normal test you would do this with (is (thrown? Assertion (function... But when I try this in test.check it just fails.
If I understand your update correctly, your test expects that the exception is thrown. If the exception does not occur, then the test should fail.
If so, then you will need to write the property in defspec to catch the exception, return it as a value, and verify that the exception occurs.
You could do something like this:
(defn catcher [f]
(try (f)
(catch Throwable t t)))
(defspec some-specification-name
(prop/for-all [...generator-bindings...]
(not-nil? (catcher (my-function generated-values)))))
I'm playing with someone else's code by examining it in the repl.
It keeps calling System/exit, which brings down my repl. This is infuriating.
In all the code I have access to, I've mocked the calls out.
But it's also calling some library code I don't have the source to, both java and clojure, and this occasionally causes exits too.
Is there any way to catch these calls globally, so that an attempt to call them doesn't kill the repl thread? Ideally it would just throw an exception instead.
I think in java I could install a new SecurityManager to get this effect, but I've never done it
there seems to be something in that line here:
http://jroller.com/ethdsy/entry/disabling_system_exit
So I'm thinking something like:
(System/setSecurityManager (SecurityManager.))
only I somehow need to attach
public void checkPermission( Permission permission ) {
if( "exitVM".equals( permission.getName() ) ) {
throw new ExitTrappedException() ;
}
}
My best shot so far is:
(System/setSecurityManager
(proxy [SecurityManager] []
(checkPermission [p]
(when (= "exitVM" (.getName p))
(throw (Exception. "exit"))))))
or maybe
(System/setSecurityManager
(proxy [SecurityManager] []
(checkExit [n] false)))
But they both just destroy the repl
Or is there a better way of doing this?
Use AspectJ and intercept all Calls to System.exit() with a no op.
But you are right, just configuring the security manager would be saner.
You can also use clj-sandbox to restrict code you don't trust.
This one works for me in both, Clojures simple REPL, and in the Lein REPL, too
(def SM (proxy [SecurityManager] []
(checkPermission
[^java.security.Permission p]
(when (.startsWith (.getName p) "exitVM")
(throw (SecurityException. "exit"))))))
(System/setSecurityManager SM)
Ah. Even in the Cider REPL in Emacs.
The name is actually "exitVM.n", where n is the numeric exit code passed to System/exit
I still have an issue to extend this security manager. Surprisingly many Clojure functions call the security manager, and thus, when used inside of it, give an infinite loop, aka StackOverflowException.
(For the latter I opened anothe question: Security Manager in Clojure )
You can try this: http://download.oracle.com/javase/6/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)