Why do I get a % printed at the end of the print in the main function?
If I use println or prn it no longer appears.
I've created a simple leiningen app structure to start the project (if that's related) so I'm using lein run to run the code.
(ns temp-test.core
(:gen-class))
;; consumes token from source
(defn scantoken [source]
(let [c (first source)]
(case c
\( [:LEFT_PAREN (rest source)]
\) [:RIGHT_PAREN (rest source)]
[nil (rest source) (str "Unexpected character " c)])))
(defn scanner [source]
;; scan source for tokens
(loop [src source
result []]
(if (empty? src)
(conj result :EOF)
(let [[token remsrc err] (scantoken src)]
(when err (println err))
(recur remsrc (conj result token))))))
(defn -main
"Scans for tokens"
[& args]
; (println ["Hello, World!"]))
(print (scanner (seq "("))))
Update
Something to do with the shell I'm using produces:
Related
I've been struggling to create a macro that'll allow me to dynamically bind whatever is in the &env into a binding form and then delegate to a pry like function to open a REPL that can see those bound &env symbols.
My simplistic pry func, which works as expected
(defn pry []
(print (str "pry(" *ns* ")> "))
(flush)
(let [expr (read)]
(when-not (= expr :exit)
(println (eval expr))
(recur))))
Using the pry func:
clojure-noob.core=> (def a 1)
#'clojure-noob.core/a
clojure-noob.core=> (pry)
pry(clojure-noob.core)> (+ a 1)
2
pry(clojure-noob.core)> :exit
nil
clojure-noob.core=>
My attempt at creating a dynamic invocation of binding:
(defmacro binding-pry []
(let [ks (keys &env)]
`(let [ks# '~ks
vs# [~#ks]
bs# (vec (interleave ks# vs#))]
(binding bs# (pry)))))
However, this fails because the inner symbol bs# is not expanded to an actual vector but instead is the generated symbol and binding tosses a clojure.core/binding requires a vector for its binding exception.
clojure-noob.core=> (let [a 1 b 2] (binding-pry))
Syntax error macroexpanding clojure.core/binding at (/tmp/form-init14332359378145135257.clj:1:16).
clojure.core/binding requires a vector for its binding in clojure-noob.core:
clojure-noob.core=>
The code quoted form with a debug print, the bs# symbol is resolved when printing but I don't know how to make it resolve to a vector when constructing the binding form.
(defmacro binding-pry []
(let [ks (keys &env)]
`(let [ks# '~ks
vs# [~#ks]
bs# (vec (interleave ks# vs#))]
(println bs#)
`(binding bs# (pry)))))
clojure-noob.core=> (let [a 1 b 2] (binding-pry))
[a 1 b 2]
(clojure.core/binding clojure-noob.core/bs__2464__auto__ (clojure-noob.core/pry))
clojure-noob.core=>
I'm very confident I'm tackling this incorrectly but I don't see another approach.
The Joy of Clojure demonstrates a break macro that does this already. I can't reproduce its source here, because it's EPL and not CC. But you can see its source at https://github.com/joyofclojure/book-source/blob/b76ef15/first-edition/src/joy/breakpoint.clj. It refers to a contextual-eval function as well: https://github.com/joyofclojure/book-source/blob/b76ef15/first-edition/src/joy/macros.clj#L4-L7.
A first step towards improving your attempt could be writing:
(defmacro binding-pry []
(let [ks (keys &env)]
`(binding [~#(interleave ks ks)] (pry))))
This still doesn't work because binding expects that the symbols can be resolved to existing dynamic vars. To tackle this problem you could make binding-pry introduce such vars as shown below:
(defmacro binding-pry []
(let [ks (keys &env)]
`(do
~#(map (fn [k] `(def ~(with-meta k {:dynamic true}))) ks)
(binding [~#(interleave ks ks)] (pry)))))
But this can have undesirable side-effects, like polluting the namespace with new var-names or making existing vars dynamic. So I would prefer an approach like the one mentioned in amalloy's answer but with a better implementation of eval-in-context (see my comment there).
To write a self-contained answer based on your pry function, let's first define eval-in which evaluates a form in an environment:
(defn eval-in [env form]
(apply
(eval `(fn* [~#(keys env)] ~form))
(vals env)))
Then let's modify pry to take an environment as an argument and use eval-in instead of eval:
(defn pry [env]
(let [prompt (str "pry(" *ns* ")> ")]
(loop []
(print prompt)
(flush)
(let [expr (read)]
(when-not (= expr :exit)
(println (eval-in env expr))
(recur))))))
An equivalent, less primitive version could be:
(defn pry [env]
(->> (repeatedly (let [prompt (str "pry(" *ns* ")> ")]
#(do (print prompt) (flush) (read))))
(take-while (partial not= :exit))
(run! (comp println (partial eval-in env)))))
Now we can define binding-pry as follows:
(defmacro binding-pry []
`(pry ~(into {}
(map (juxt (partial list 'quote) identity))
(keys &env))))
Finally, here is a direct/"spaghetti" implementation of binding-pry:
(defmacro binding-pry []
(let [ks (keys &env)]
`(->> (repeatedly (let* [prompt# (str "pry(" *ns* ")> ")]
#(do (print prompt#) (flush) (read))))
(take-while (partial not= :exit))
(run! (comp println
#((eval `(fn* [~~#(map (partial list 'quote) ks)] ~%))
~#ks))))))
This piece of code runs on the server and it detects the changes to a file and sends it to the client. This is working for the first time and after that the file length is not getting updated even the I changed the file and saved it. I guess the clojure immutability is the reason here. How can I make this work?
(def clients (atom {}))
(def rfiles (atom {}))
(def file-pointers (atom {}))
(defn get-rfile [filename]
(let [rdr ((keyword filename) #rfiles)]
(if rdr
rdr
(let [rfile (RandomAccessFile. filename "rw")]
(swap! rfiles assoc (keyword filename) rfile)
rfile))))
(defn send-changes [changes]
(go (while true
(let [[op filename] (<! changes)
rfile (get-rfile filename)
ignore (println (.. rfile getChannel size))
prev ((keyword filename) #file-pointers)
start (if prev prev 0)
end (.length rfile) // file length is not getting updated even if I changed the file externally
array (byte-array (- end start))]
(do
(println (str "str" start " end" end))
(.seek rfile start)
(.readFully rfile array)
(swap! file-pointers assoc (keyword filename) end)
(doseq [client #clients]
(send! (key client) (json/write-str
{:changes (apply str (map char array))
:fileName filename}))
false))))))
There is no immutability here. In the rfiles atom, you store standard Java objects that are mutable.
This code works well only if data are appended to the end of the file, and the size is always increasing.
If there is an update/addition (of length +N) in the file other than at the end, the pointers start and end won't point to the modified data, but just to the last N characters and you will send dummy stuff to the clients.
If there is a delete or any change that decrease the length,
array (byte-array (- end start))
will throw a NegativeArraySizeException you don't see (eaten by the go bloc?). You can add some (try (...) catch (...)) or test that (- end start) is alway positive or null, to manage it and do the appropriate behaviour: resetting the pointers?,...
Are you sure the files you scan for changes are only changed by appending data? If not, you need to handle this case by resetting or updating the pointers accordingly.
I hope it will help.
EDIT test environment.
I defined the following. There is no change to the code you provided.
;; define the changes channel
(def notif-chan (chan))
;; define some clients
(def clients (atom {:foo "foo" :bar "bar"}))
;; helper function to post a notif of change in the channel
(defn notify-a-change [op filename]
(go (>! notif-chan [op filename])))
;; mock of the send! function used in send-changes
(defn send! [client message]
(println client message))
;; main loop
(defn -main [& args]
(send-changes notif-chan))
in a repl, I ran:
repl> (-main)
in a shell (I tested with an editor too):
sh> echo 'hello there' >> ./foo.txt
in the repl:
repl> (notify-a-change "x" "./foo.txt")
str0 end12
:bar {"changes":"hello there\n","fileName":".\/foo.txt"}
:foo {"changes":"hello there\n","fileName":".\/foo.txt"}
repl> (notify-a-change "x" "./foo.txt")
str12 end12
:bar {"changes":"","fileName":".\/foo.txt"}
:foo {"changes":"","fileName":".\/foo.txt"}
in a shell:
sh> echo 'bye bye' >> ./foo.txt
in a repl:
repl> (notify-a-change "x" "./foo.txt")
str12 end20
:bar {"changes":"bye bye\n","fileName":".\/foo.txt"}
:foo {"changes":"bye bye\n","fileName":".\/foo.txt"}
I need to send a string that is a path of a directory to a function.how do I do that in Clojure?
I tried doing the following but it didn't work
(defn make-asm-file [d]
(doseq [f (.listFiles d)]
(if
( and (=(str(last (split (.getName f) #"\."))) "vm") (not (.isDirectory f)))
(translate f d))))
(make-asm-file "~\SimpleAdd")
How about the following,
(defn find-files
[regexp directory]
(filter #(and (.isFile %)
(re-find regexp (str %)))
(.listFiles (clojure.java.io/file directory))))
(doseq [f (find-files #".vm$" "~/SimpleAdd")]
(translate f))
In this case (java.io.File. d) would work instead of (clojure.java.io/file d) as well. You could also use file-seq instead of listfiles, but that would include all *.vm files in subdirs.
I'm trying to compute folder size in parallel.
Maybe it's naive approach.
What I do, is that I give computation of every branch node (directory) to an agent.
All leaf nodes have their file sizes added to my-size.
Well it doesn't work. :)
'scan' works ok, serially.
'pscan' prints only files from first level.
(def agents (atom []))
(def my-size (atom 0))
(def root-dir (clojure.java.io/file "/"))
(defn scan [listing]
(doseq [f listing]
(if (.isDirectory f)
(scan (.listFiles f))
(swap! my-size #(+ % (.length f))))))
(defn pscan [listing]
(doseq [f listing]
(if (.isDirectory f)
(let [a (agent (.listFiles f))]
(do (swap! agents #(conj % a))
(send-off a pscan)
(println (.getName f))))
(swap! my-size #(+ % (.length f))))))
Do you have any idea, what have i done wrong?
Thanks.
No need to keep state using atoms. Pure functional:
(defn psize [f]
(if (.isDirectory f)
(apply + (pmap psize (.listFiles f)))
(.length f)))
So counting filesizes in parallel should be so easy?
It's not :)
I tried to solve this issue better. I realized that i'm doing blocking I/O operations so pmap doesn't do the job.
I was thinking maybe giving chunks of directories (branches) to agents to process it independently would make sense. Looks it does :)
Well I haven't benchmarked it yet.
It works, but, there might be some problems with symbolic links on UNIX-like systems.
(def user-dir (clojure.java.io/file "/home/janko/projects/"))
(def root-dir (clojure.java.io/file "/"))
(def run? (atom true))
(def *max-queue-length* 1024)
(def *max-wait-time* 1000) ;; wait max 1 second then process anything left
(def *chunk-size* 64)
(def queue (java.util.concurrent.LinkedBlockingQueue. *max-queue-length* ))
(def agents (atom []))
(def size-total (atom 0))
(def a (agent []))
(defn branch-producer [node]
(if #run?
(doseq [f node]
(when (.isDirectory f)
(do (.put queue f)
(branch-producer (.listFiles f)))))))
(defn producer [node]
(future
(branch-producer node)))
(defn node-consumer [node]
(if (.isFile node)
(.length node)
0))
(defn chunk-length []
(min (.size queue) *chunk-size*))
(defn compute-sizes [a]
(doseq [i (map (fn [f] (.listFiles f)) a)]
(swap! size-total #(+ % (apply + (map node-consumer i))))))
(defn consumer []
(future
(while #run?
(when-let [size (if (zero? (chunk-length))
false
(chunk-length))] ;appropriate size of work
(binding [a (agent [])]
(dotimes [_ size] ;give us all directories to process
(when-let [item (.poll queue)]
(set! a (agent (conj #a item)))))
(swap! agents #(conj % a))
(send-off a compute-sizes))
(Thread/sleep *max-wait-time*)))))
You can start it by typing
(producer (list user-dir))
(consumer)
For result type
#size-total
You can stop it by (there are running futures - correct me if I'm wrong)
(swap! run? not)
If you find any errors/mistakes, you're welcome to share your ideas!
I have just start reading Let over lambda and I thought I would try and write a clojure version of the block-scanner in the closures chapter.
I have the following so far:
(defn block-scanner [trigger-string]
(let [curr (ref trigger-string) trig trigger-string]
(fn [data]
(doseq [c data]
(if (not (empty? #curr))
(dosync(ref-set curr
(if (= (first #curr) c)
(rest #curr)
trig)))))
(empty? #curr))))
(def sc (block-scanner "jihad"))
This works I think, but I would like know what I did right and what I could do better.
I would not use ref-set but alter because you don't reset the state to a completely new value, but update it to a new value which is obtained from the old one.
(defn block-scanner
[trigger-string]
(let [curr (ref trigger-string)
trig trigger-string]
(fn [data]
(doseq [c data]
(when (seq #curr)
(dosync
(alter curr
#(if (-> % first (= c))
(rest %)
trig)))))
(empty? #curr))))
Then it is not necessary to use refs since you don't have to coordinate changes. Here an atom is a better fit, since it can be changed without all the STM ceremony.
(defn block-scanner
[trigger-string]
(let [curr (atom trigger-string)
trig trigger-string]
(fn [data]
(doseq [c data]
(when (seq #curr)
(swap! curr
#(if (-> % first (= c))
(rest %)
trig))))
(empty? #curr))))
Next I would get rid of the imperative style.
it does more than it should: it traverses all data - even if we found a match already. We should stop early.
it is not thread-safe, since we access the atom multiple times - it might change in between. So we must touch the atom only once. (Although this is probably not interesting in this case, but it's good to make it a habit.)
it's ugly. We can do all the work functionally and just save the state, when we come to a result.
(defn block-scanner
[trigger-string]
(let [state (atom trigger-string)
advance (fn [trigger d]
(when trigger
(condp = d
(first trigger) (next trigger)
; This is maybe a bug in the book. The book code
; matches "foojihad", but not "jijihad".
(first trigger-string) (next trigger-string)
trigger-string)))
update (fn [trigger data]
(if-let [data (seq data)]
(when-let [trigger (advance trigger (first data))]
(recur trigger (rest data)))
trigger))]
(fn [data]
(nil? (swap! state update data)))))