I am working through the example of making parallel http requests in Clojure,
http://lethain.com/a-couple-of-clojure-agent-examples/
In particular
(ns parallel-fetch
(:import [java.io InputStream InputStreamReader BufferedReader]
[java.net URL HttpURLConnection]))
(defn get-url [url]
(let [conn (.openConnection (URL. url))]
(.setRequestMethod conn "GET")
(.connect conn)
(with-open [stream (BufferedReader.
(InputStreamReader. (.getInputStream conn)))]
(.toString (reduce #(.append %1 %2)
(StringBuffer.) (line-seq stream))))))
(defn get-urls [urls]
(let [agents (doall (map #(agent %) urls))]
(doseq [agent agents] (send-off agent get-url))
(apply await-for 5000 agents)
(doall (map #(deref %) agents))))
(prn (get-urls '("http://lethain.com" "http://willarson.com")))
When I run this in the
IllegalStateException await-for in transaction
What does this mean and how do I fix it?
Taking the comment on the question into account:
A transaction is being set up in the process of loading your namespace, and since it has a call to get-urls at the top-level, the await-for happens in that transaction and throws the exception.
The best way to fix that is to put the prn / get-urls form inside a function and only call it once the namespace is loaded. (If you wanted to run this code as a standalone app, with lein run or java -jar on an überjar, you'd put a call to that function inside -main.)
Incidentally, the transaction is set up when you use :reload-all, but not without it. (See the private functions load-lib, which checks for the presence of :reload-all and decides to use the private function load-all if it's there, and load-all itself, which is where the transaction is set up. Here's a link to the 1.5.1 source.)
Related
I love re-frame, but I realize that I'm having a bit of trouble finding a nice pattern for handling AJAX responses.
My situation is the following:
I have a "global" event handler that triggers some AJAX call and dispatches to some other global event handler, depending on whether that call was successful, e.g.:
(reg-event-db :store-value
(fn [db [_ value]]
(do-ajax-store value
:on-success #(dispatch [:store-value-success %])
:on-error #(dispatch [:store-value-error %])
db))
(reg-event-db :store-value-success
(fn [db [_ result]]
(assoc db :foobar result)))
(reg-event-db :store-value-error
(fn [db [_ error]]
(assoc db :foobar nil
:last-error error)))
(I am aware of reg-event-fx and stuff, I'm just avoiding it here for the sake of brevity and because I think it does not make any difference for my problem).
I also have (multiple, distinct) UI components that might trigger the :store-value event, like so:
(defn button []
(let [processing? (reagent/atom false)]
(fn button-render []
[:button {:class (when #processing? "processing")
:on-click (fn []
(reset! processing? true)
(dispatch [:store-value 123]))}])))
So in this case the component has local state (processing?) that is supposed to depend on whether the AJAX call is still in progress or not.
Now, what is the proper pattern here to have the button component react to the events :store-value-success and :store-value-error in order to reset the processing? flag back to false after the AJAX call has finished?
Currently, I'm working around that problem by passing down callbacks but that seems really ugly because it clutters the event handlers' code with stuff that does not really belong there.
The best solution that I've thought of would be to have the button component being able to hook into the :store-value-success and :store-value-error events and install its own handler for those events, like this:
(defn button []
(let [processing? (reagent/atom false)]
(reg-event-db :store-value-success
(fn [db _]
(reset! processing? false)))
(reg-event-db :store-value-error
(fn [db _]
(reset! processing? false)))
(fn button-render []
[:button {:class (when #processing? "processing")
:on-click (fn []
(reset! processing? true)
(dispatch [:store-value 123]))}])))
However, that does not work. As it seems, re-frame does not allow multiple event handlers per event. Instead, a subsequent invocation of reg-event-db on one single event id will replace the previous event handler.
So how do you guys handle situations like this?
I think reg-event-fx (src) might indeed help solve your problem.
You could add a subscription that watches app-state e.g.
(rf/reg-sub
:app-state
(fn [db]
(get db :app-state)))
and add this to your button, perhaps with a state function e.g.
(defn btn-state [state]
(if (= state :processing)
"processing"))
And then in the AJAX handler, you could add an fx to update the state-
(reg-event-fx ;; -fx registration, not -db registration
:ajax-success
(fn [{:keys [db]} [_ result]]
{:db (assoc db :app-state :default)
:dispatch [:store-value-success result]}))
(reg-event-fx ;; -fx registration, not -db registration
:ajax-error
(fn [{:keys [db]} [_ result]]
{:db (assoc db :app-state :default)
:dispatch [:store-value-error result]}))
and update the AJAX handler
(reg-event-db :store-value
(fn [db [_ value]]
(do-ajax-store value
:on-success #(dispatch [:ajax-success %])
:on-error #(dispatch [:ajax-error %])
db))
This would be one way to handle it via -fx. I think you have already started to see the need for tracking app state, and I think bumping it up into the subscriptions would help with complexity, at which point your button render is greatly simplified.
(defn button []
[:button {:class (btn-state #app-state)
:on-click (dispatch [:store-value 123]))}])))
As others have mentioned, I would recommend to use http-fx and make processing? part of your global state. The code would look like this:
Events:
(reg-event-fx
:request
(fn [{:keys [db]} [_ method url data]]
{:http-xhrio {:method method
:uri url
:params data
:format (ajax/json-request-format)
:response-format (ajax/json-response-format {:keywords? true})
:on-success [:success-response method url]
:on-failure [:error-response method url]}
:db (assoc db :processing? true)}))
(reg-event-db
:success-response
(fn [db [_ method url result]]
(assoc db :foobar response
:processing? false)}))
(reg-event-db
:error
(fn [db [_ method url result]]
(assoc db :foobar nil
:last-error result
:processing? false)}))
Subscriptions:
(reg-sub
:processing
(fn [db _]
(:processing? db)))
View:
(defn button []
(let [processing? #(rf/subscribe [:processing])]
[:button {:class (when processing? "processing")
:on-click #(dispatch [:store-value 123]))}])))
Hint: You could reuse this code with all your requests.
I'm having some trouble to get started with Light Table.
Here's my code (Clojure)
(ns prova1-ed.core
(:gen-class))
(use 'clojure.java.io)
(defn -main [& args]
(println "Type the name of the file to read: ")
(let [fileName (read-line)]
(let [rdr (reader fileName)]
(doseq [line (line-seq rdr)]
(println line)
)))
)
I'm sure it works. I've tested with lein run. As you can see, the program should read a file which the name is given by the user.
I've tried CTRL+SPACE in Light Table, but this is what I receive:
ERROR: Unhandled REPL handler exception processing message {:data {:auto? false, :pos {:line 14, :ch 1}, :mime "text/x-clojure", :tags [:editor.clj :editor.clojure], :type-name "Clojure", :line-ending "\r\n", :ns user, :path "C:\\Users\\Tiago\\Documents\\Clojure\\prova1_ed\\src\\prova1_ed\\core.clj", :print-length nil, :name "core.clj", :local true, :code "(ns prova1-ed.core\n (:gen-class))\n\n(use 'clojure.java.io)\n\n(defn -main [& args]\n\n (println \"Type the name of the file to read: \")\n\n (let [fileName (read-line)]\n (let [rdr (reader fileName)]\n (doseq [line (line-seq rdr)]\n (println line)\n )))\n)\n"}, :id 90, :op editor.eval.clj.sonar, :session 65d1da68-a730-4ffe-9365-9527726384e3}
How can i run it in the Light Tables' enviroment, so that I can input the file name?
TLDR
I don't think you can run (read-line) in Light Table as it'd have to add explicit support for allowing input. There's no standard input basically.
An Alternative
I'd suggest you modify your -main function to accept an explicit file-name argument instead of trying to read it from a standard input that isn't available.
I've got a Clojure webapp that I work on in Light Table.
I've got a -main function in a namespace named my-app.web. It looks something like this:
(defn -main [& [port]]
(let [port (Integer. (or port (env :port) 5000))
store (cookie/cookie-store {:key (env :session-secret)})]
(jetty/run-jetty (-> #'secured-app
wrap-with-logging
wrap-current-user
wrap-current-auth
wrap-error-page
(site {:session {:store store}}))
{:port port :join? false})))
In a separate file I've named light-table-start.clj, I've got the following code to run my app inside Light Table:
(require '[my-app.web :as web])
(require '[ring.adapter.jetty :as jetty])
(defonce server (web/-main "5000"))
;; (.start server)
;; (.stop server)
I run the Eval: Eval editor contents command (Ctrl+Shift+Enter on Windows and Linux or ⌘+Shift+Enter on Mac OS) the first time I want to run my app (or later, if the connection is closed for some reason). When I want to start or stop the server I can just highlight the code on the respective commented lines and run the Eval: Eval a form in editor command (Ctrl+Enter on Windows and Linux or ⌘+Enter on Mac OS).
I'm writing a small tool in clojure and want to know when there's been a change on the clipboard. Here's a simplified version of what's going on.
(:import java.awt.Toolkit)
(:import (java.awt.datatransfer Clipboard
ClipboardOwner
Transferable
StringSelection
DataFlavor
FlavorListener))
(defn get-clipboard [] (. (Toolkit/getDefaultToolkit)
(getSystemClipboard)))
(defn get-content []
(.getContents (get-clipboard) nil))
(def content (agent (get-content)))
(defn watch [key f]
(add-watch content key f))
(defn -main []
(while (not= content "banana-man")
(watch :watcher
(fn [key agent old-state new-state]
(prn "-- agent Changed --")
(prn "key" key)
(prn "atom" agent)
(prn "old-state" old-state)
(prn "new-state" new-state)))))
I've added in a while loop just to keep the main function from shutting down immediately.
This runs without throwing any errors, but does not report when changes have been made on the clipboard or stop the while loop when I copy bannan-man to the clipboard. I've been struggling with this for a few weeks now and I'm sure I'm missing something simple. If anyone has some advice I would really appreciate it!
For starters, content is an agent, so it will never be equal to a string. You should deref the agent using # in order to make that comparison.
The while loop is not needed to prevent exit. If you use the agent thread pool, Clojure will not shut down until you explicitly run shutdown-agents. But we will need it to manage your agent updates.
content is not going to change after your initial assignment unless you explicitly send it an updating function with send or send-off. Don't let the name mislead you, agents are not autonomous, and are not scheduled or repeated tasks. Try something like this:
(defn -main []
(watch :watcher
(fn [key agent old-state new-state]
(prn "-- agent Changed --")
(prn "key" key)
(prn "atom" agent)
(prn "old-state" old-state)
(prn "new-state" new-state)))
(while (not= #content "banana-man")
(send-off content (fn [& _] (get-content)))
(Thread/sleep 250))
(shutdown-agents))
I'm still newbie in clojure and I'm trying to build application which read two files and write the diffrence on JSON file
(defn read-csv
"reads data."
[]
(with-open [rdr (
io/reader "resources/staples_data.csv")]
(doseq [line (rest(line-seq rdr))]
(println(vec(re-seq #"[^,]+" line))))))
(defn read-psv
"reads data."
[]
(with-open [rdr (
io/reader "resources/external_data.psv")]
(doseq [line (rest(line-seq rdr))]
; (print(vec(re-seq #"[^|]+" line))))))
(doall(vec(re-seq #"[^|]+" line))))))
(defn process-content []
(let [csv-records (agent read-csv)
psv-records (agent read-psv)]
(json/write-str {"my-data" #csv-records "other-data" #psv-records}))
)
Im getting an exception: Exception Don't know how to write JSON of class $read_csv clojure.data.json/write-generic (json.clj:385)
Please some help with some explanation, thanks in advance!
You are giving the agent a function as its initial value. Perhaps you meant to do an asynchronous call to that function instead? In that case, a future is a better match for your scenario as shown. agent is synchronous, it's send and send-off that are async, and they assume you are propagating some state across calls which doesn't match your usage here.
(defn process-content []
(let [csv-records (future-call read-csv)
psv-records (future-call read-psv)]
(json/write-str {"my-data" #csv-records "other-data" #psv-records})))
The problem after that is that doseq is only for side effects, and always returns nil. If you want the results read from the csv files (evaluating eagerly so you are still in the scope of the with-open call), use (doall (for ...)) as a replacement for (doseq ...). Also, the println in read-csv will need to be removed, or replaced with (doto (vec (re-seq #"[^,]+" line)) println) because println always returns nil, and I assume you want the actual data from the file, not a list of nils.
I am working in clojure with a java class which provides a retrieval API for a domain specific binary file holding a series of records.
The java class is initialized with a file and then provides a .query method which returns an instance of an inner class which has only one method .next, thus not playing nicely with the usual java collections API. Neither the outer nor inner class implements any interface.
The .query method may return null instead of the inner class. The .next method returns a record string or null if no further records are found, it may return null immediately upon the first call.
How do I make this java API work well from within clojure without writing further java classes?
The best I could come up with is:
(defn get-records
[file query-params]
(let [tr (JavaCustomFileReader. file)]
(if-let [inner-iter (.query tr query-params)] ; .query may return null
(loop [it inner-iter
results []]
(if-let [record (.next it)]
(recur it (conj results record))
results))
[])))
This gives me a vector of results to work with the clojure seq abstractions. Are there other ways to expose a seq from the java API, either with lazy-seq or using protocols?
Without dropping to lazy-seq:
(defn record-seq
[q]
(take-while (complement nil?) (repeatedly #(.next q))))
Instead of (complement nil?) you could also just use identity if .next does not return boolean false.
(defn record-seq
[q]
(take-while identity (repeatedly #(.next q))))
I would also restructure a little bit the entry points.
(defn query
[rdr params]
(when-let [q (.query rdr params)]
(record-seq q)))
(defn query-file
[file params]
(with-open [rdr (JavaCustomFileReader. file)]
(doall (query rdr params))))
Seems like a good fit for lazy-seq:
(defn query [file query]
(.query (JavaCustomFileReader. file) query))
(defn record-seq [query]
(when query
(when-let [v (.next query)]
(cons v (lazy-seq (record-seq query))))))
;; usage:
(record-seq (query "filename" "query params"))
Your code is not lazy as it would be if you were using Iterable but you can fill the gap with lazy-seq as follows.
(defn query-seq [q]
(lazy-seq
(when-let [val (.next q)]
(cons val (query-seq q)))))
Maybe you shoul wrap the query method to protect yourself from the first null value as well.