I use a dynamic var to make an easy way to use ssh. However, it suddenly stops working in a multi-parameter for!
So, here is my core.clj (which is kind of sketchy now):
(use 'clj-ssh.ssh)
(def the-agent (ssh-agent {}))
(def ^:dynamic *session* nil)
(defmacro on-host [host & body]
`(binding [*session* (clj-ssh.ssh/session the-agent ~host {})]
~#body))
(defn cmd [& args]
(split (:out (ssh *session* {:cmd (join " " args)})) #"\n"))
(defn attempt-1 []
(cmd "ls -a"))
(defn attempt-2 []
(for [f (cmd "ls -a")]
f))
(defn attempt-3 []
(for [r (range 3)
f (cmd "ls -a")]
[r f]))
For some reason, first two trial functions work, and the third doesn't (the hosts and files are censured):
user=> (on-host "(some host)" (attempt-1))
["." ".." ".ackrc" ...]
user=> (on-host "(some host)" (attempt-2))
("." ".." ".ackrc" ...)
user=> (on-host "(some host)" (attempt-3))
IllegalArgumentException No implementation of method: :connected? of protocol: #'clj-ssh.ssh.protocols/Session found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:544)
Just in case you need a stacktrace:
user=> (use 'clojure.stacktrace)
nil
user=> (print-stack-trace *e 7)
java.lang.IllegalArgumentException: No implementation of method: :connected? of protocol: #'clj-ssh.ssh.protocols/Session found for class: nil
at clojure.core$_cache_protocol_fn.invoke (core_deftype.clj:544)
clj_ssh.ssh.protocols$eval1554$fn__1566$G__1543__1571.invoke (protocols.clj:4)
clj_ssh.ssh$connected_QMARK_.invoke (ssh.clj:411)
clj_ssh.ssh$ssh.invoke (ssh.clj:712)
census.core$cmd.doInvoke (core.clj:15)
clojure.lang.RestFn.invoke (RestFn.java:408)
census.core$attempt_3$iter__1949__1955$fn__1956.invoke (core.clj:29)
nil
I'm really not sure what it is all about. Can you help me? Thank you!
Use doseq, not for
What you have there is a lazy sequence that will be evaluated after the binding form has returned. doseq forces evaluation.
For further reading:
Difference between doseq and for in Clojure
http://cemerick.com/2009/11/03/be-mindful-of-clojures-binding/
Related
In my app I'm providing some interface to users that they can provide code and app evaluates that code within sandbox(so eval fn not allowed).The thing is I need to catch if user overrides some built-in function such as =
Any ideas how to catch and prevent that thing?(The idea is they should not be able to do that)
Code:
(defn =
[]
//some code)
WARNING: = already refers to: #'clojure.core/= in namespace: user, being replaced by: #'user/=
One solution might be:
I was trying to get the warning message as String but with-out-str function did not work.
(with-out-str
(defn = []))
;=> ""
Also wrote that with-err-str(changed with-out-str little bit) did not work as well.
(defmacro with-err-str
[& body]
`(let [s# (new java.io.StringWriter)]
(binding [*err* s#]
~#body
(str s#))))
(with-err-str
(defn = []))
;=> ""
Need: "WARNING: = already refers to: #'clojure.core/= in namespace: user, being replaced by: #'user/="
It does work when you use eval:
user=> (with-err-str (eval '(defn - [] 11)))
"WARNING: - already refers to: #'clojure.core/- in namespace: user, being replaced by: #'user/-\n"
user=> (re-seq #"WARNING" (with-err-str (eval '(defn / [] 11))))
("WARNING")
Or you could redefine the defn macro in user's code, but nothing prevents them to use other clojure tools to redefine a var:
user=> (defmacro defn-safe
#_=> [nam & decls]
#_=> (if (resolve (symbol "clojure.core" (name nam)))
#_=> (print "Whoops")
#_=> (list* `defn (with-meta nam (assoc (meta nam) :private true)) decls)))
#'user/defn-safe
user=> (defn-safe foo [x] (+ x 2))
#'user/foo
user=> (foo 22)
24
user=> (defn-safe = [a b] (- a b))
Whoopsnil
user=>
Another option, and probably your best bet is using
https://github.com/clojure/tools.analyzer
clojail handles this (and many other things as well). If you're looking to sandbox Clojure, I'd recommend taking a look.
One solution might be like this:
(def before (set (vals (ns-map *ns*))))
(defn = [])
(def after (set (vals (ns-map *ns*))))
(clojure.set/difference before after)
;=> #{#'clojure.core/=}
Hi I have this school project that I am almost done with so I don't need help with the code, problem is I've never coded in clojure but for this assignment had to do a try and catch macro in clojure with bound forms, there are some REPL commands that are expected to give different responses for the assignment to pass,,
Anyways Im getting an error that I've been googling but nothing is specific to this problem and most explanations basically need there own explanation nothing seems to be beginner adapted so it doesnt do much for me.
(defmacro safe [bindings & code]
(if (list? bindings)
`(try
~bindings
(catch Throwable except# except#))
(if (= (count bindings) 0)
`(try ~code
(catch Throwable except# except#))
`(let ~(subvec bindings 0 2)
(try
(safe ~(subvec bindings 2) ~#code)
(catch Throwable except# except#)
(finally
(. ~(bindings 0) close))))))) ;;safe
(def divider(safe (/ 1 0)))
(def reader (safe [s (FileReader. (java.io.File. "C:/text.txt"))] (. s read)))
So the error Im getting is
=> (def v (safe [s (FileReader. (java.io.File. "C:/text.txt"))] (. s read)))
#'myProject.core/v
=> v
#<ClassCastException java.lang.ClassCastException: java.lang.Integer cannot be cast to clojure.lang.IFn>
So kindly anyone that knows clojure please give me some hints on what is wrong, all the hints Im getting is that there should be a paranthesis missplaced, I've checked the code over and over but cant find any misstypes etc. Thank you !
Use macroexpand and friends to help debug
(def form (quote (safe [s (FileReader. (java.io.File. "test.txt"))] (. s read))))
(pprint (macroexpand form)) ;=>
(let*
[s (FileReader. (java.io.File. "test.txt"))]
(try
(user/safe [] (. s read)) ; <--- Macro is recursive
(catch java.lang.Throwable except__1104__auto__ except__1104__auto__)
(finally (. s user/close))))
Macro is recursive, so this is not the full expansion
(pprint (clojure.walk/macroexpand-all form)) ;=>
(let*
[s (new FileReader (new java.io.File "test.txt"))]
(try
(try
((. s read)) ; <-- Too many parenthesis!
(catch ...
The read call returns an integer, which is in turn being called as a function in the recursive macro expansion.
(FileReader. (java.io.File. "/tmp/text.txt"))
is giving you the value of the first character on the file
cat /tmp/text.txt => abcd
(let [s (FileReader. (java.io.File. "/tmp/text.txt"))] (. s read)) => 97
same as
(int \a)
and your macro expect a function. try:
(safe [s (FileReader. (java.io.File. "/tmp/text.txt"))] #(+ 6)) or (safe [s (FileReader. (java.io.File. "/tmp/text.txt"))] (fn [] (. s read)
Solved!
Missing unquote slicing in the third try block, causing exception
java.lang.ClassCastException: java.lang.Integer cannot be cast to clojure.lang.IFn
this was caused due to extra paranthesis when reading s recursivly,
macroexpand ex.
(pprint (macroexpand-all form))
(let*[s
(new FileReader
(new File "test.txt"))]
(try
(try
((.s read)) ;<= here
Had to change the third try/catch from
(try
(safe ~(subvec bindings 2) ~#code)
to:
(try
(safe ~#(subvec bindings 2) ~#code)
To my understanding the problem was that I was returned a list with a paranthesis like (1(2 3)) causing exception when read, unquoting makes it look (1 2 3) and can be read or (in this case) casted from java to clojure.
reservations for misspellz or calling shit the wrong name! :)
Many thanks to A.Webb and patz.
Here is the sample code I want to get to work:
(letfn [(CONC [f] f)
(CONT [f] (str "\newline" f))]
((voodoo "CONC") "hamster"))
Is there some voodo that will make it call the CONC function with hamster as the parameter? That is, is there some way to convert the string "CONC" into a function that is not bound to a namespace but rather to a local binding?
EDIT:
To be clearer, the way this will be called is:
(map #((voodoo (:tag %)) (:value %))
[
{:tag "CONC" :value "hamster"}
{:tag "CONT" :value "gerbil"}
]
)
I'd probably solve this by creating a map of functions indexed by strings:
(def voodoo
{"CONC" (fn [f] f)
"CONT" (fn [f] (str "\newline" f))})
Then your desired code should work directly (exploiting the fact that a map is a function that looks up it's argument)
(map #((voodoo (:tag %)) (:value %))
[
{:tag "CONC" :value "hamster"}
{:tag "CONT" :value "gerbil"}
]
)
Note that the functions here are fully anonymous - you don't need them to be referenced anywhere in the namespace for this to work. In my view this is a good thing, because unless you also need the functions somewhere else then it's best to avoid polluting your top-level namespace too much.
No. Eval does not have access to the local/lexical environment, ever.
Edit: This is not a very good answer, and not really accurate either. You could write voodoo as a macro, and then it doesn't need runtime access to the lexical environment, just compile-time. However, this means it would only work if you know at compile time that the function you want to call is x, and so it wouldn't be very useful - why not just type x instead of (voodoo "x")?
(defmacro voodoo [fname]
(symbol fname))
(letfn [(x [y] (inc y))]
((voodoo "x") 2))
;; 3
(letfn [(x [y] (inc y))]
(let [f "x"]
((voodoo f) 2)))
;; error
Well, it's sort of possible:
(defmacro voodoo [s]
(let [env (zipmap (map (partial list 'quote) (keys &env))
(keys &env))]
`(if-let [v# (~env (symbol ~s))]
v#
(throw (RuntimeException. "no such local")))))
...and now we can do weird stuff like this:
user> (defn example [s]
(letfn [(foo [x] {:foo x})
(bar [x] {:bar x})]
((voodoo s) :quux)))
#'user/example
user> (example "foo")
{:foo :quux}
user> (example "bar")
{:bar :quux}
user> (example "quux")
; Evaluation aborted.
user> *e
#<RuntimeException java.lang.RuntimeException: no such local>
That "Evaluation aborted" means an exception was thrown.
You could also replace the throw branch of the if in voodoo with (resolve (symbol ~s)) to defer to the globals if no local is found:
(defmacro voodoo [s]
(let [env (zipmap (map (partial list 'quote) (keys &env))
(keys &env))]
`(if-let [v# (~env (symbol ~s))]
v#
(resolve (symbol ~s)))))
...and now this works with definition of example as above (though note that if you are experimenting at the REPL, you will need to recompile example after redefining voodoo):
user> (defn quux [x] {:quux x})
#'user/quux
user> (example "quux")
{:quux :quux}
Now, this is an abuse of Clojure's facilities which one would do well to try to do without. If one cannot, one should probably turn to evalive by Michael Fogus; it's a library which provides an "eval-with-locals" facility in the form of an evil function and a couple of utilities. The functionality seems to be well factored too, e.g. something like the ~(zipmap ...) thing above is encapsulated as a macro and evil there appears to be almost a drop-in replacement for eval (add the env parameter and you're good to go). I haven't read the source properly, but I probably will now, looks like fun. :-)
Im not really clear what you are asking for so i'll try a couple answers:
if you have a string that is the name of the function you wish to call:
(def name "+")
((find-var (symbol (str *ns* "/" name))) 1 2 3)
this would give voodoo a deffinition like this:
(defn voodoo [name args] (apply (find-var (symbol (str *ns* "/" name))) args))
#'clojure.core/voodoo
clojure.core=> (voodoo "+" [1 2 3])
6
clojure.core=>
this assumes your function is in the current namepace ns.
if you want to turn a string into a function you could use this pattern
(let [f (eval (read-string "(fn [] 4)"))] (f))
I want to send var-args of a function to a macro, still as var-args.
Here is my code:
(defmacro test-macro
[& args]
`(println (str "count=" ~(count args) "; args=" ~#args)))
(defn test-fn-calling-macro
[& args]
(test-macro args))
The output of (test-macro "a" "b" "c") is what I want: count=3; args=abc
The output of (test-fn-calling-macro "a" "b" "c") is : count=1; args=("a" "b" "c") because args is sent as a single argument to the macro. How can I expand this args in my function in order to call the macro with the 3 arguments?
I guess I'm just missing a simple core function but I'm not able to find it. Thanks
EDIT 2 - My "real" code, shown in EDIT section below is not a valid situation to use this technique.
As pointed out by #Brian, the macro xml-to-cass can be replaced with a function like this:
(defn xml-to-cass
[zipper table key attr & path]
(doseq [v (apply zf/xml-> zipper path)] (cass/set-attr! table key attr v)))
EDIT - the following section goes beyond my original question but any insight is welcome
The code above is just the most simple I could come with to pinpoint my problem. My real code deals with clj-cassandra and zip-filter. It may also look over-engineering but it's just a toy project and I'm trying to learn the language at the same time.
I want to parse some XML found on mlb.com and insert values found into a cassandra database. Here is my code and the thinking behind it.
Step 1 - Function which works fine but contains code duplication
(ns stats.importer
(:require
[clojure.xml :as xml]
[clojure.zip :as zip]
[clojure.contrib.zip-filter.xml :as zf]
[cassandra.client :as cass]))
(def root-url "http://gd2.mlb.com/components/game/mlb/year_2010/month_05/day_01/")
(def games-table (cass/mk-cf-spec "localhost" 9160 "mlb-stats" "games"))
(defn import-game-xml-1
"Import the content of xml into cassandra"
[game-dir]
(let [url (str root-url game-dir "game.xml")
zipper (zip/xml-zip (xml/parse url))
game-id (.substring game-dir 4 (- (.length game-dir) 1))]
(doseq [v (zf/xml-> zipper (zf/attr :type))] (cass/set-attr! games-table game-id :type v))
(doseq [v (zf/xml-> zipper (zf/attr :local_game_time))] (cass/set-attr! games-table game-id :local_game_time v))
(doseq [v (zf/xml-> zipper :team [(zf/attr= :type "home")] (zf/attr :name_full))] (cass/set-attr! games-table game-id :home_team v))))
The parameter to import-game-xml-1 can be for example "gid_2010_05_01_colmlb_sfnmlb_1/". I remove the "gid_" and the trailing slash to make it the key of the ColumnFamily games in my database.
I found that the 3 doseq were a lot of duplication (and there should be more than 3 in the final version). So code templating using a macro seemed appropriate here (correct me if I'm wrong).
Step 2 - Introducing a macro for code templating (still works)
(defmacro xml-to-cass
[zipper table key attr & path]
`(doseq [v# (zf/xml-> ~zipper ~#path)] (cass/set-attr! ~table ~key ~attr v#)))
(defn import-game-xml-2
"Import the content of xml into cassandra"
[game-dir]
(let [url (str root-url game-dir "game.xml")
zipper (zip/xml-zip (xml/parse url))
game-id (.substring game-dir 4 (- (.length game-dir) 1))]
(xml-to-cass zipper games-table game-id :type (zf/attr :type))
(xml-to-cass zipper games-table game-id :local_game_time (zf/attr :local_game_time))
(xml-to-cass zipper games-table game-id :home_team :team [(zf/attr= :type "home")] (zf/attr :name_full))))
I believe that's an improvement but I still see some duplication in always reusing the same 3 parameters in my calls to xml-to-cass. That's were I introduced an intermediate function to take care of those.
Step 3 - Adding a function to call the macro (the problem is here)
(defn import-game-xml-3
"Import the content of xml into cassandra"
[game-dir]
(let [url (str root-url game-dir "game.xml")
zipper (zip/xml-zip (xml/parse url))
game-id (.substring game-dir 4 (- (.length game-dir) 1))
save-game-attr (fn[key path] (xml-to-cass zipper games-table game-id key path))]
(save-game-attr :type (zf/attr :type)) ; works well because path has only one element
(save-game-attr :local_game_time (zf/attr :local_game_time))
(save-game-attr :home :team [(zf/attr= :type "home"] (zf/attr :name_full))))) ; FIXME this final line doesn't work
Here's a some simple code which may be illuminating.
Macros are about code generation. If you want that to happen at runtime, for some reason, then you have to build and evaluate the code at runtime. This can be a powerful technique.
(defmacro test-macro
[& args]
`(println (str "count=" ~(count args) "; args=" ~#args)))
(defn test-fn-calling-macro
[& args]
(test-macro args))
(defn test-fn-expanding-macro-at-runtime
[& args]
(eval (cons `test-macro args)))
(defmacro test-macro-expanding-macro-at-compile-time
[& args]
(cons `test-macro args))
;; using the splicing notation
(defmacro test-macro-expanding-macro-at-compile-time-2
[& args]
`(test-macro ~#args))
(defn test-fn-expanding-macro-at-runtime-2
[& args]
(eval `(test-macro ~#args)))
(test-macro "a" "b" "c") ;; count=3; args=abc nil
(test-fn-calling-macro "a" "b" "c") ;; count=1; args=("a" "b" "c") nil
(test-fn-expanding-macro-at-runtime "a" "b" "c") ; count=3; args=abc nil
(test-macro-expanding-macro-at-compile-time "a" "b" "c") ; count=3; args=abc nil
(test-macro-expanding-macro-at-compile-time-2 "a" "b" "c") ; count=3; args=abc nil
(test-fn-expanding-macro-at-runtime "a" "b" "c") ; count=3; args=abc nil
If contemplation of the above doesn't prove enlightening, might I suggest a couple of my own blog articles?
In this one I go through macros from scratch, and how clojure's work in particular:
http://www.learningclojure.com/2010/09/clojure-macro-tutorial-part-i-getting.html
And in this one I show why run-time code generation might be useful:
http://www.learningclojure.com/2010/09/clojure-faster-than-machine-code.html
The typical way to use a collection as individual arguments to a function is to use (apply function my-list-o-args)
(defn test-not-a-macro [& args]
(print args))
(defn calls-the-not-a-macro [& args]
(apply test-not-a-macro args))
though you wont be able to use apply because test-macro is a macro. to solve this problem you will need to wrap test macro in a function call so you can apply on it.
(defmacro test-macro [& args]
`(println ~#args))
(defn calls-test-macro [& args]
(eval (concat '(test-macro) (args)))) ;you almost never need eval.
(defn calls-calls-test-macro [& args]
(calls-test-macro args))
This is actually a really good example of one of the ways macros are hard to compose. (some would say they cant be composed cleanly, though i think thats an exageration)
Macros are not magic. They are a mechanism to convert code at compile-time to equivalent code; they are not used at run-time. The pain you are feeling is because you are trying to do something you should not be trying to do.
I don't know the library in question, but if cass/set-attr! is a function, I see no reason why the macro you defined has to be a macro; it could be a function instead. You can do what you want to do if you can rewrite your macro as a function instead.
Your requirements aren't clear. I don't see why a macro is necessary here for test-macro, unless you're trying to print the unevaluated forms supplied to your macro.
These functions provide your expected results, but that's because your sample data was self-evaluating.
(defn test-args
[& args]
(println (format "count=%d; args=%s"
(count args)
(apply str args))))
or
(defn test-args
[& args]
(print (format "count=%d; args=" (count args)))
(doseq [a args]
(pr a))
(newline))
You can imagine other variations to get to the same result.
Try calling that function with something that doesn't evaluate to itself, and note the result:
(test-args (+ 1 2) (+ 3 4))
Were you looking to see the arguments printed as "37" or "(+ 1 2)(+ 3 4)"?
If you were instead trying to learn about macros and their expansion in general, as opposed to solving this particular problem, please tune your question to probe further.
I have a URL checker that I use in Perl. I was wondering how something like this would be done in Clojure. I have a file with thousands of URLs and I'd like the output file to contain the URL (minus http://, https://) and a simple :1 for valid and :0 for false. Ideally, I could check each site concurrently, considering that this is one of Clojure's strengths.
Input
http://www.google.com
http://www.cnn.com
http://www.msnbc.com
http://www.abadurlisnotgood.com
Output
www.google.com:1
www.cnn.com:1
www.msnbc.com:1
www.abadurlisnotgood.com:0
I assume by "valid URL" you mean HTTP response 200. This might work. It requires clojure-contrib. Change map to pmap to attempt to make it parallel, like Arthur Ulfeldt mentioned.
(use '(clojure.contrib duck-streams
java-utils
str-utils))
(import '(java.net URL
URLConnection
HttpURLConnection
UnknownHostException))
(defn check-url [url]
(str (re-sub #"^(?i)http:/+" "" url)
":"
(try
(let [c (cast HttpURLConnection
(.openConnection (URL. url)))]
(if (= 200 (.getResponseCode c))
1
0))
(catch UnknownHostException _
0))))
(defn check-urls-from-file [filename]
(doseq [line (map check-url
(read-lines (as-file filename)))]
(println line)))
Given your example as input:
user> (check-urls-from-file "urls.txt")
www.google.com:1
www.cnn.com:1
www.msnbc.com:1
www.abadurlisnotgood.com:0
Write a small function that appends a ":1" or ":0" to a url and then use pmap to apply it in parallel to all the urls.
(defn check-a-url [url] .... )
(pmap #(if (check-a-url %) (str url ":1") (str url ":0")))
Clojure now has a as-url function in clojure.java.io:
(as-url "http://google.com") ;;=> #object[java.net.URL 0x5dedf9bd "http://google.com"]
(str (as-url "http://google.com")) ;;=> "http://google.com"
(as-url "notanurl") ;; java.net.MalformedURLException
Based on that we could write a function like so:
(defn check-url
"checks if the url is well formed"
[url]
(str (clojure.string/replace-first url #"(http://|https://)" "")
":"
(try (as-url url) ;; built-in, does not perform an actual request, and does very little validation
1
(catch Exception e 0))))
(defn check-urls-from-file
"from Brian Carper answer"
[filename]
(doseq [line (map check-url (read-lines (as-file filename)))]
(println line)))
Instead of pmap, I used agents with send-off in conjunction with the above solution. I think this is better when there is blocking I/O. I believe pmap has limited concurrency too. Here's what I have so far. I wonder how this will scale with thousands of URLs.
(use '(clojure.contrib duck-streams
java-utils
str-utils))
(import '(java.net URL
URLConnection
HttpURLConnection
UnknownHostException))
(defn check-url [url]
(str (re-sub #"^(?i)http:/+" "" url)
":"
(try
(let [c (cast HttpURLConnection
(.openConnection (URL. url)))]
(if (= 200 (.getResponseCode c))
1
0))
(catch UnknownHostException _
0))))
(def urls (read-lines "urls.txt"))
(def agents (for [url urls] (agent url)))
(doseq [agent agents]
(send-off agent check-url))
(apply await agents)
(def x '())
(doseq [url (filter deref agents)]
(def x (cons #url x)))
(prn x)
(shutdown-agents)