I am trying to make a Clojurescript app that shows recipes.
Relevant code follows (also available as a gist):
(defn load-recipes [data]
(go (if (not (:loaded? #data))
(let [recipes-data (<! (fetch-recipes data))]
(om/update! data :recipes recipes-data)
(om/update! data :loaded? true))
(println "Data already loaded"))))
(defn define-routes [data]
(defroute home-path "/" []
(om/update! data :view :home))
(defroute "/random" []
(go (loop [loaded? (:loaded? (om/value data))]
(if-not loaded? (do (println "Waiting for data...")
(recur (:loaded? (om/value data))))
(do (om/update! data :tag
(rand-nth
(vec (apply set/union (map :tags (:recipes #data))))))
(om/update! data :view :random)))))))
(defn app-view [data owner]
(reify
om/IWillMount
(will-mount [_]
(do
(load-recipes data)
(define-routes data)))
om/IDidMount
(did-mount [_]
#_(fetch-recipes data))
om/IRender
(render [_]
(html data))))
What I want to accomplish:
First fetch the recipes using an async http call. I am using cljs-http.client that returns a channel
Define routes using the secretary library. In the /random route I want to select a random recipe. This can only happen when the data has been fetched and has been updated in the application atom.
What I get now is an infinite loop in the browser. What's happening?
Another option is wrapping all my routes who need data fetching first in a go block and put (<! (load-recipes)) on the first line.
PS: I ended up with
(defn ensure-recipes-loaded [data]
(go (if (not (:loaded? (om/value data)))
(do (om/update! data :view :loading)
(let [recipes-data (<! (fetch-recipes data))]
(om/update! data :recipes recipes-data)
(om/update! data :loaded? true)))
(println "Data already loaded"))))
(defn define-routes [data]
(defroute home-path "/" []
(om/update! data :view :home))
(defroute "/random" []
(go
(<! (ensure-recipes-loaded data))
(do (om/update! data :tag
(rand-nth
(vec (apply set/union (map :tags (:recipes #data))))))
(om/update! data :view :random))))
(defroute "/random/:tagname" [tagname]
(go (<! (ensure-recipes-loaded data))
(om/update! data :tag tagname)
(om/update! data :view :random)))
(defroute "/recipe/:link" [link]
(go (<! (ensure-recipes-loaded data))
(om/update! data :view :recipe)
(om/update! data :permalink link)))
(defroute "*" [*]
(go (<! (ensure-recipes-loaded data))
(om/update! data :view :default))))
Using a loop construct without any (parking) channel operations inside it to wait is pretty much entirely contrary to the spirit of core.async. Keep in mind that JavaScript is single-threaded, so if you don't park a thread of execution, there's no opportunity for anything else to run.
Have a channel that you close when the loading operation is complete. Try to read from that channel when you need to block; if it returns nil, it's done, and if it blocks, then you have the desired operation of waiting. If you want to periodically do something while waiting, then read from both that channel and a timeout.
Related
I am new to clojure. I am trying to write a program which reads data from a file (comma seperated file) after reading the data I am trying to split each line while delimiter "," but I am facing the below error:
CompilerException java.lang.NullPointerException,
compiling:(com\clojure\apps\StudentRanks.clj:26:5)
Here is my code:
(ns com.clojure.apps.StudentRanks)
(require '[clojure.string :as str])
(defn student []
(def dataset (atom []))
(def myList (atom ()))
(def studObj (atom ()))
(with-open [rdr (clojure.java.io/reader "e:\\example.txt")]
(swap! dataset into (reduce conj [] (line-seq rdr)))
)
(println #dataset)
(def studentCount (count #dataset))
(def ind (atom 0))
(loop [n studentCount]
(when (>= n 0)
(swap! myList conj (get #dataset n))
(println (get #dataset n))
(recur (dec n))))
(println myList)
(def scount (count #dataset))
(loop [m scount]
(when (>= m 0)
(def data(get #dataset m))
(println (str/split data #","))
(recur (dec m))))
)
(student)
Thanks in advance.
As pointed out in the comments, the first problem is that you are not writing correct Clojure.
To start, def should never be nested -- it's not going to behave like you hope. Use let to introduce local variables (usually just called locals because it's weird to call variables things that don't vary).
Second, block-like constructs (such as do, let or with-open evaluates to the value of their last expression.
So this snippet
(def dataset (atom []))
(with-open [rdr (clojure.java.io/reader "e:\\example.txt")]
(swap! dataset into (reduce conj [] (line-seq rdr))))
should be written
(let [dataset
(with-open [rdr (clojure.java.io/reader "e:\\example.txt")]
(into [] (line-seq rdr)))]
; code using dataset goes here
)
Then you try to convert dataset (a vector) to a list (myList) by traversing it backwards and consing on the list under construction. It's not needed. You can get a sequence (list-like) out of a vector by just calling seq on it. (Or rseq if you want the list to be reversed.)
Last, you iterate once again to split and print each item held in dataset. Explicit iteration with indices is pretty unusual in Clojure, prefer reduce, doseq, into etc.
Here are two ways to write student:
(defn student [] ; just for print
(with-open [rdr (clojure.java.io/reader "e:\\example.txt")]
(doseq [data (line-seq rdr)]
(println (str/split data #",")))))
(defn student [] ; to return a value
(with-open [rdr (clojure.java.io/reader "e:\\example.txt")]
(into []
(for [data (line-seq rdr)]
(str/split data #",")))))
I hope this will help you to better get Clojure.
I suggest you use a csv library:
(require '[clojure.data.csv :as csv])
(csv/read-csv (slurp "example.txt"))
Unless this is some file io exercise.
This is a snippet from the Reagent project. Looking at complete-all and clear-done, I understand the point is to swap out the modified map. I don't understand how it's being done. The definition of mmap calls for 3 parameters — and complete-all seems to be calling it with two, namely map and #(assoc-in % [1 :done] v). clear-done calls with remove and #(get-in % [1 :done]). I tried using the repl to experiment but couldn't get the requires to work out.
(ns todomvc.core
(:require [reagent.core :as r]))
(defonce todos (r/atom (sorted-map)))
(defonce counter (r/atom 0))
(defn add-todo [text]
(let [id (swap! counter inc)]
(swap! todos assoc id {:id id :title text :done false})))
(defn toggle [id] (swap! todos update-in [id :done] not))
(defn save [id title] (swap! todos assoc-in [id :title] title))
(defn delete [id] (swap! todos dissoc id))
(defn mmap [m f a] (->> m (f a) (into (empty m))))
(defn complete-all [v] (swap! todos mmap map #(assoc-in % [1 :done] v)))
(defn clear-done [] (swap! todos mmap remove #(get-in % [1 :done])))
The existing map is passed as the first argument to the function. When all else fails...
Problem: I get a null pointer exception when I call the caller function in the below code if I pass it data via json from a javascript socket client. I don't get the error when I run this same code from the repl and I pass it the same data i.e.
In repl:
(def data (json/write-str {:controller "hql" :function "something"}))
(caller data)
This works fine
From javascript: this gives me s null pointer exception when calling the function caller from the handler.
<script>
var msg = {'controller': 'hql','function':'something'};
socket = new WebSocket('ws://192.168.0.7:9090');
socket.onopen = function() {
socket.send(JSON.stringify(msg));
}
socket.onmessage = function(s) {
console.log(s);
socket.close();
}
socket.onclose = function() {
console.log('Connection Closed');
}
</script>
(ns hqlserver.core
(:use [org.httpkit.dbcp :only [use-database! close-database! insert-record update-values query delete-rows]]
[org.httpkit.server])
(:require [clojure.data.json :as json]))
(defn hql [data]
(use-database! "jdbc:mysql://localhost/xxxx" "user" "xxxxx")
(let [rows (query ("select username,firstname,lastname from users order by lastname,firstname")]
(close-database!)
(json/write-str rows)))
(defn get-controller [data]
(if data
(let [data (json/read-str data :key-fn keyword)]
(data :controller))))
(defn caller [data]
((ns-resolve *ns* (symbol (get-controller data))) data))
(defn handler [request]
(with-channel request channel
(on-close channel (fn [status] (println "channel closed: " status)))
(on-receive channel (fn [data] (send! channel (caller data))))))
(defn -main [& args]
(run-server handler {:port 9090}))
Don't know if it's the best way but this is the solution:
Thanks to noisesmith for helping and tips....
(ns hqlserver.core
(:use [org.httpkit.dbcp :only [use-database! close-database! insert- record update-values query delete-rows]]
[org.httpkit.server]
[hqlserver.s300.login :only [login]])
(:require [clojure.data.json :as json]
[hqlserver.s300.site :refer [site-index site-update]]
[clojure.string :as str]))
(defn get-site [data]
(if data
(let [data (json/read-str data :key-fn keyword)]
(-> data :site))))
(defn get-controller [data]
(if data
(let [data (json/read-str data :key-fn keyword)]
(-> data :controller))))
(defn get-function [data]
(if data
(let [data (json/read-str data :key-fn keyword)]
(-> data :function))))
(defn caller [data]
(let [site (get-site data)]
(let [controller (get-controller data)]
(let [function (get-function data)]
(cond
(and (= site "s300")(= controller "login")(= function "login")) (login data)
(and (= site "s300")(= controller "site")(= function "site-index")) (site-index data)
(and (= site "s300")(= controller "site")(= function "site-update")) (site-update data)
:else (json/write-str {:error "No records found!"}))))))
(defn async-handler [ring-request]
(with-channel ring-request channel
(if (websocket? channel)
(on-receive channel (fn [data]
(send! channel (caller data))))
(send! channel {:status 200
:headers {"Content-Type" "text/plain"}
:body "Long polling?"}))))
(defn -main [& args]
(run-server async-handler {:port 9090}));
What's best way in clojure to implement something like an actor or agent (asynchronously updated, uncoordinated reference) that does the following?
gets sent messages/data
executes some function on that data to obtain new state; something like (fn [state new-msgs] ...)
continues to receive messages/data during that update
once done with that update, runs the same update function against all messages that have been sent in the interim
An agent doesn't seem quite right here. One must simultaneously send function and data to agents, which doesn't leave room for a function which operates on all data that has come in during the last update. The goal implicitly requires a decoupling of function and data.
The actor model seems generally better suited in that there is a decoupling of function and data. However, all actor frameworks I'm aware of seem to assume each message sent will be processed separately. It's not clear how one would turn this on it's head without adding extra machinery. I know Pulsar's actors accept a :lifecycle-handle function which can be used to make actors do "special tricks" but there isn't a lot of documentation around this so it's unclear whether the functionality would be helpful.
I do have a solution to this problem using agents, core.async channels, and watch functions, but it's a bit messy, and I'm hoping there is a better solution. I'll post it as a solution in case others find it helpful, but I'd like to see what other's come up with.
Here's the solution I came up with using agents, core.async channels, and watch functions. Again, it's a bit messy, but it does what I need it to for now. Here it is, in broad strokes:
(require '[clojure.core.async :as async :refer [>!! <!! >! <! chan go]])
; We'll call this thing a queued-agent
(defprotocol IQueuedAgent
(enqueue [this message])
(ping [this]))
(defrecord QueuedAgent [agent queue]
IQueuedAgent
(enqueue [_ message]
(go (>! queue message)))
(ping [_]
(send agent identity)))
; Need a function for draining a core async channel of all messages
(defn drain! [c]
(let [cc (chan 1)]
(go (>! cc ::queue-empty))
(letfn
; This fn does all the hard work, but closes over cc to avoid reconstruction
[(drainer! [c]
(let [[v _] (<!! (go (async/alts! [c cc] :priority true)))]
(if (= v ::queue-empty)
(lazy-seq [])
(lazy-seq (cons v (drainer! c))))))]
(drainer! c))))
; Constructor function
(defn queued-agent [& {:keys [buffer update-fn init-fn error-handler-builder] :or {:buffer 100}}]
(let [q (chan buffer)
a (agent (if init-fn (init-fn) {}))
error-handler-fn (error-handler-builder q a)]
; Set up the queue, and watcher which runs the update function when there is new data
(add-watch
a
:update-conv
(fn [k r o n]
(let [queued (drain! q)]
(when-not (empty? queued)
(send a update-fn queued error-handler-fn)))))
(QueuedAgent. a q)))
; Now we can use these like this
(def a (queued-agent
:init-fn (fn [] {:some "initial value"})
:update-fn (fn [a queued-data error-handler-fn]
(println "Receiving data" queued-data)
; Simulate some work/load on data
(Thread/sleep 2000)
(println "Done with work; ready to queue more up!"))
; This is a little warty at the moment, but closing over the queue and agent lets you requeue work on
; failure so you can try again.
:error-handler-builder
(fn [q a] (println "do something with errors"))))
(defn -main []
(doseq [i (range 10)]
(enqueue a (str "data" i))
(Thread/sleep 500) ; simulate things happening
; This part stinks... have to manually let the queued agent know that we've queued some things up for it
(ping a)))
As you'll notice, having to ping the queued-agent here every time new data is added is pretty warty. It definitely feels like things are being twisted out of typical usage.
Agents are the inverse of what you want here - they are a value that gets sent updating functions. This easiest with a queue and a Thread. For convenience I am using future to construct the thread.
user> (def q (java.util.concurrent.LinkedBlockingDeque.))
#'user/q
user> (defn accumulate
[summary input]
(let [{vowels true consonents false}
(group-by #(contains? (set "aeiouAEIOU") %) input)]
(-> summary
(update-in [:vowels] + (count vowels))
(update-in [:consonents] + (count consonents)))))
#'user/accumulate
user> (def worker
(future (loop [summary {:vowels 0 :consonents 0} in-string (.take q)]
(if (not in-string)
summary
(recur (accumulate summary in-string)
(.take q))))))
#'user/worker
user> (.add q "hello")
true
user> (.add q "goodbye")
true
user> (.add q false)
true
user> #worker
{:vowels 5, :consonents 7}
I came up with something closer to an actor, inspired by Tim Baldridge's cast on actors (Episode 16). I think this addresses the problem much more cleanly.
(defmacro take-all! [c]
`(loop [acc# []]
(let [[v# ~c] (alts! [~c] :default nil)]
(if (not= ~c :default)
(recur (conj acc# v#))
acc#))))
(defn eager-actor [f]
(let [msgbox (chan 1024)]
(go (loop [f f]
(let [first-msg (<! msgbox) ; do this so we park efficiently, and only
; run when there are actually messages
msgs (take-all! msgbox)
msgs (concat [first-msg] msgs)]
(recur (f msgs)))))
msgbox))
(let [a (eager-actor (fn f [ms]
(Thread/sleep 1000) ; simulate work
(println "doing something with" ms)
f))]
(doseq [i (range 20)]
(Thread/sleep 300)
(put! a i)))
;; =>
;; doing something with (0)
;; doing something with (1 2 3)
;; doing something with (4 5 6)
;; doing something with (7 8 9 10)
;; doing something with (11 12 13)
I'm have a following code:
(use 'clojure.java.io)
(defrecord Member [id name salary role])
(defrecord Role [id name])
(def member-records (ref ()))
(defn add-member [member]
(dosync (alter member-records conj member)))
;;Test-data -->
(def dev-r(->Role 1 "Developer"))
(def test-member1(->Member 1 "Kirill" 70000.00 dev-r))
;;Test-data <--
(defn save-data-2-file []
(with-open [wrtr (writer "C:/Platform/Work/test.cdf")]
(print-dup #member-records wrtr)))
(defn process-line [line]
(println line))
;;Test line content
;;#BTC.pcost.Member{:id 1, :name "Kirill", :salary 70000.0, :role #BTC.pcost.Role{:id 1, :name "Developer"}})
(defn load-data-from-file []
(with-open [rdr (reader "C:/Platform/Work/test.cdf")]
(doseq [line (line-seq rdr)]
(process-line line))))
I'm want to recreate records after reading file, but i can not understand how i can make it. Yes, i'm know that i can parse text and fill my structure by the elements of parsed line, but it's will be difficult, cause i'm have alot structs like "Member" and "Role". Can anyone to suggest me a way, that i can do?
You can use read-string, and slurp, to pull the records out of the file. read-string is limited to reading the first form of a string, but, from your sample, you are only storing a single form, as a list of records.
(defn load-data-from-file [file]
(read-string (slurp file)))
Lazy Reading
If you need more than the first form, or cannot read the entire stream into memory, you can use read directly, to make a lazy reader.
(defn lazy-read
([rdr] (let [eof (Object.)] (lazy-read rdr (read rdr false eof) eof)))
([rdr data eof]
(if (not= eof data)
(cons data (lazy-seq (lazy-read rdr (read rdr false eof) eof))))))
(defn load-all-data [file]
(with-open [rdr (java.io.PushbackReader. (reader file))]
(doall (lazy-read rdr))))
(load-all-data "C:/Platform/Work/test.cdf")
Security
Also, it is good to mention security when loading code with read-string or read. You should only use them with trusted sources, because, using #= or a Java constructor, the source can execute arbitrary code inside your application. For a longer explanation, take a look at the documentation for read.
Setting *read-eval* to false would prevent the issue, but it would also prevent the reconstruction of the records in your sample. To avoid the issue all together, you can use the clojure.edn/read and clojure.edn/read-string functions, with a whitelist of readers.
(defn edn-read [eof rdr]
(clojure.edn/read {:eof eof :readers {'BTC.pcost.Role map->Role
'BTC.pcost.Member map->Member}}
rdr))
(defn lazy-edn-read
([rdr] (let [eof (Object.)] (lazy-edn-read rdr (edn-read eof rdr) eof)))
([rdr data eof]
(if (not= eof data)
(cons data (lazy-seq (lazy-edn-read rdr (edn-read eof rdr) eof))))))
(defn load-all-data [file]
(with-open [rdr (java.io.PushbackReader. (reader file))]
(doall (take-while (complement nil?) (lazy-edn-read rdr)))))
(load-all-data "C:/Platform/Work/test.cdf")
You can use read.
This function will read one object from a file:
(defn load-data-from-file [filename]
(with-open [rdr (java.io.PushbackReader. (reader filename))]
(read rdr)))
Or this will read all objects from the file:
(defn load-all-data-from-file [filename]
(let [eof (Object.)]
(with-open [rdr (java.io.PushbackReader. (reader filename))]
(doall
(take-while #(not= % eof)
(repeatedly #(read rdr nil eof)))))))
Here's the API documentation for read.
This is a small variation that will read all objects from a string:
(defn load-all-data-from-string [string]
(let [eof (Object.)]
(with-open [rdr (-> string java.io.StringReader. java.io.PushbackReader.)]
(doall
(take-while #(not= % eof)
(repeatedly #(read rdr nil eof)))))))
This is, as far as I know, not possible to do using read-string. Instead we use read with a java.io.StringReader.