Take the following program as an example:
(defn echo-ints []
(doseq [i (->> (BufferedReader. *in*)
(line-seq)
(map read-string)
(take-while integer?))]
(println i)))
The idea is to prompt the user for input and then echo it back if it's an integer. However, in this particular program almost every second input won't be echoed immediately. Instead the program will wait for additional input before processing two inputs at once.
Presumably this a consequence of some performance tweaks happening behind the scenes. However in this instance I'd really like to have an immediate feedback loop. Is there an easy way to accomplish this, or does the logic of the program have to be significantly altered?
(The main motivation here is to pass the infinite sequence of user inputs to another function f that transforms lazy sequences to other lazy sequences. If I wrote some kind of while-loop, I wouldn't be able to use f.)
It is generally not good to mix lazyness with side-effect (printing in this case), since most sequence functions have built-in optimizations that cause unintended effects while still being functionally correct.
Here's a good write up: https://stuartsierra.com/2015/08/25/clojure-donts-lazy-effects
What you are trying to do seems like a good fit for core.async channels. I would think as the problem as 'a stream of user input' instead of 'infinite sequence of user inputs', and 'f transforms lazy sequences to lazy sequences' becomes 'f transform a stream into another stream'. This will allow you to write f as transducers which you can arbitrarily compose.
I would do it like the following. Note we use spyx and spyxx from the Tupelo library to display some results.
First, write a simple version with canned test data:
(ns tst.demo.core
(:use tupelo.test)
(:require
[tupelo.core :as t] )
(:import [java.io BufferedReader StringReader]))
(t/refer-tupelo)
(def user-input
"hello
there
and
a
1
and-a
2
and
a
3.14159
and-a
4
bye" )
(defn echo-ints
[str]
(let [lines (line-seq (BufferedReader. (StringReader. str)))
data (map read-string lines)
nums (filter integer? data) ]
(doseq [it data]
(spyxx it))
(spyx nums)))
(newline)
(echo-ints user-input)
This gives us the results:
it => <#clojure.lang.Symbol hello>
it => <#clojure.lang.Symbol there>
it => <#clojure.lang.Symbol and>
it => <#clojure.lang.Symbol a>
it => <#java.lang.Long 1>
it => <#clojure.lang.Symbol and-a>
it => <#java.lang.Long 2>
it => <#clojure.lang.Symbol and>
it => <#clojure.lang.Symbol a>
it => <#java.lang.Double 3.14159>
it => <#clojure.lang.Symbol and-a>
it => <#java.lang.Long 4>
it => <#clojure.lang.Symbol bye>
nums => (1 2 4)
So, we see that it works and gives us the numbers we want.
Next, write a looping version. We make it terminate gracefully when our test data runs out.
(defn echo-ints-loop
[str]
(loop [lines (line-seq (BufferedReader. (StringReader. str)))]
(let [line (first lines)
remaining (rest lines)
data (read-string line)]
(when (integer? data)
(println "found:" data))
(when (not-empty? remaining)
(recur remaining)))))
(newline)
(echo-ints-loop user-input)
found: 1
found: 2
found: 4
Next, we write an infinite loop to read the keyboard. You need to terminate this one with CRTL-C at the keyboard:
(ns demo.core
(:require [tupelo.core :as t])
(:import [java.io BufferedReader StringReader]))
(t/refer-tupelo)
(defn echo-ints-inf
[]
(loop [lines (line-seq (BufferedReader. *in*))]
(let [line (first lines)
remaining (rest lines)
data (read-string line)]
(when (integer? data)
(println "found:" data))
(when (not-empty? remaining)
(recur remaining)))))
(defn -main []
(println "main - enter")
(newline)
(echo-ints-inf))
And we run it manually:
~/clj > lein run
main - enter
hello
there
1
found: 1
and
a
2
found: 2
and-a
3
found: 3
further more
4
found: 4
^C
~/clj >
~/clj >
Related
I'm experimenting with core.async on Clojure and ClojureScript, to try and understand how merge works. In particular, whether merge makes any values put on input channels available to take immediately on the merged channel.
I have the following code:
(ns async-merge-example.core
(:require
#?(:clj [clojure.core.async :as async] :cljs [cljs.core.async :as async])
[async-merge-example.exec :as exec]))
(defn async-fn-timeout
[v]
(async/go
(async/<! (async/timeout (rand-int 5000)))
v))
(defn async-fn-exec
[v]
(exec/exec "sh" "-c" (str "sleep " (rand-int 5) "; echo " v ";")))
(defn merge-and-print-results
[seq async-fn]
(let [chans (async/merge (map async-fn seq))]
(async/go
(while (when-let [v (async/<! chans)]
(prn v)
v)))))
When I try async-fn-timeout with a large-ish seq:
(merge-and-print-results (range 20) async-fn-timeout)
For both Clojure and ClojureScript I get the result I expect, as in, results start getting printed pretty much immediately, with the expected delays.
However, when I try async-fn-exec with the same seq:
(merge-and-print-results (range 20) async-fn-exec)
For ClojureScript, I get the result I expect, as in results start getting printed pretty much immediately, with the expected delays. However for Clojure even though the sh processes are executed concurrently (subject to the size of the core.async thread pool), the results appear to be initially delayed, then mostly printed all at once! I can make this difference more obvious by increasing the size of the seq e.g. (range 40)
Since the results for async-fn-timeout are as expected on both Clojure and ClojureScript, the finger is pointed at the differences between the Clojure and ClojureScript implementation for exec..
But I don't know why this difference would cause this issue?
Notes:
These observations were made in WSL on Windows 10
The source code for async-merge-example.exec is below
In exec, the implementation differs for Clojure and ClojureScript due to differences between Clojure/Java and ClojureScript/NodeJS.
(ns async-merge-example.exec
(:require
#?(:clj [clojure.core.async :as async] :cljs [cljs.core.async :as async])))
; cljs implementation based on https://gist.github.com/frankhenderson/d60471e64faec9e2158c
; clj implementation based on https://stackoverflow.com/questions/45292625/how-to-perform-non-blocking-reading-stdout-from-a-subprocess-in-clojure
#?(:cljs (def spawn (.-spawn (js/require "child_process"))))
#?(:cljs
(defn exec-chan
"spawns a child process for cmd with args. routes stdout, stderr, and
the exit code to a channel. returns the channel immediately."
[cmd args]
(let [c (async/chan), p (spawn cmd (if args (clj->js args) (clj->js [])))]
(.on (.-stdout p) "data" #(async/put! c [:out (str %)]))
(.on (.-stderr p) "data" #(async/put! c [:err (str %)]))
(.on p "close" #(async/put! c [:exit (str %)]))
c)))
#?(:clj
(defn exec-chan
"spawns a child process for cmd with args. routes stdout, stderr, and
the exit code to a channel. returns the channel immediately."
[cmd args]
(let [c (async/chan)]
(async/go
(let [builder (ProcessBuilder. (into-array String (cons cmd (map str args))))
process (.start builder)]
(with-open [reader (clojure.java.io/reader (.getInputStream process))
err-reader (clojure.java.io/reader (.getErrorStream process))]
(loop []
(let [line (.readLine ^java.io.BufferedReader reader)
err (.readLine ^java.io.BufferedReader err-reader)]
(if (or line err)
(do (when line (async/>! c [:out line]))
(when err (async/>! c [:err err]))
(recur))
(do
(.waitFor process)
(async/>! c [:exit (.exitValue process)]))))))))
c)))
(defn exec
"executes cmd with args. returns a channel immediately which
will eventually receive a result map of
{:out [stdout-lines] :err [stderr-lines] :exit [exit-code]}"
[cmd & args]
(let [c (exec-chan cmd args)]
(async/go (loop [output (async/<! c) result {}]
(if (= :exit (first output))
(assoc result :exit (second output))
(recur (async/<! c) (update result (first output) #(conj (or % []) (second output)))))))))
Your Clojure implementation uses blocking IO in a single thread. You are first reading from stdout and then stderr in a loop. Both do a blocking readLine so they will only return once they actually finished reading a line. So unless your process creates the same amount of output to stdout and stderr one stream will end up blocking the other one.
Once the process is finished the readLine will no longer block and just return nil once the buffer is empty. So the loop just finishes reading the buffered output and then finally completes explaining the "all at once" messages.
You'll probably want to start a second thread that deals reading from stderr.
node does not do blocking IO so everything happens async by default and one stream doesn't block the other.
I'm learning core.async and have written a simple producer consumer code:
(ns webcrawler.parallel
(:require [clojure.core.async :as async
:refer [>! <! >!! <!! go chan buffer close! thread alts! alts!! timeout]]))
(defn consumer
[in out f]
(go (loop [request (<! in)]
(if (nil? request)
(close! out)
(do (print f)
(let [result (f request)]
(>! out result))
(recur (<! in)))))))
(defn make-consumer [in f]
(let [out (chan)]
(consumer in out f)
out))
(defn process
[f s no-of-consumers]
(let [in (chan (count s))
consumers (repeatedly no-of-consumers #(make-consumer in f))
out (async/merge consumers)]
(map #(>!! in %1) s)
(close! in)
(loop [result (<!! out)
results '()]
(if (nil? result)
results
(recur (<!! out)
(conj results result))))))
This code works fine when I step in through the process function in debugger supplied with Emacs' cider.
(process (partial + 1) '(1 2 3 4) 1)
(5 4 3 2)
However, if I run it by itself (or hit continue in the debugger) I get an empty result.
(process (partial + 1) '(1 2 3 4) 1)
()
My guess is that in the second case for some reason producer doesn't wait for consumers before exiting, but I'm not sure why. Thanks for help!
The problem is that your call to map is lazy, and will not run until something asks for the results. Nothing does this in your code.
There are 2 solutions:
(1) Use the eager function mapv:
(mapv #(>!! in %1) items)
(2) Use the doseq, which is intended for side-effecting operations (like putting values on a channel):
(doseq [item items]
(>!! in item))
Both will work and produce output:
(process (partial + 1) [1 2 3 4] 1) => (5 4 3 2)
P.S. You have a debug statement in (defn consumer ...)
(print f)
that produces a lot of noise in the output:
<#clojure.core$partial$fn__5561 #object[clojure.core$partial$fn__5561 0x31cced7
"clojure.core$partial$fn__5561#31cced7"]>
That is repeated 5 times back to back. You probably want to avoid that, as printing function "refs" is pretty useless to a human reader.
Also, debug printouts in general should normally use println so you can see where each one begins and ends.
I'm going to take a safe stab that this is being caused by the lazy behavior of map, and this line that's carrying out side effects:
(map #(>!! in %1) s)
Because you never explicitly use the results, it never runs. Change it to use mapv, which is strict, or more correctly, use doseq. Never use map to run side effects. It's meant to lazily transform a list, and abuse of it leads to behaviour like this.
So why is it working while debugging? I'm going to guess because the debugger forces evaluation as part of its operation, which is masking the problem.
As you can read from docstring map returns a lazy sequence. And I think the best way is to use dorun. Here is an example from clojuredocs:
;;map a function which makes database calls over a vector of values
user=> (map #(db/insert :person {:name %}) ["Fred" "Ethel" "Lucy" "Ricardo"])
JdbcSQLException The object is already closed [90007-170] org.h2.message.DbE
xception.getJdbcSQLException (DbException.java:329)
;;database connection was closed before we got a chance to do our transactions
;;lets wrap it in dorun
user=> (dorun (map #(db/insert :person {:name %}) ["Fred" "Ethel" "Lucy" "Ricardo"]))
DEBUG :db insert into person values name = 'Fred'
DEBUG :db insert into person values name = 'Ethel'
DEBUG :db insert into person values name = 'Lucy'
DEBUG :db insert into person values name = 'Ricardo'
nil
For the most part I understand what Clojure is telling me with it's error messages. But I am still clueless as to find out where the error happened.
Here is an example of what I mean
(defn extract [m]
(keys m))
(defn multiple [xs]
(map #(* 2 %) xs))
(defn process [xs]
(-> xs
(multiple) ; seq -> seq
(extract))) ; map -> seq ... fails
(process [1 2 3])
Statically typed languages would now tell me that I tried to pass a sequence to a function that expects a map on line X. And Clojure does this in a way:
ClassCastException java.lang.Long cannot be cast to java.util.Map$Entry
But I still have no idea where the error happened. Obviously for this instance it's easy because there are just 3 functions involved, you can easily just read through all of them but as programs grow bigger this gets old very quickly.
Is there a way find out where the errors happened other than just proof reading the code from top to bottom? (which is my current approach)
You can use clojure.spec. It is still in alpha, and there's still a bunch of tooling support coming (hopefully), but instrumenting functions works well.
(ns foo.core
(:require
;; For clojure 1.9.0-alpha16 and higher, it is called spec.alpha
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest]))
;; Extract takes a map and returns a seq
(s/fdef extract
:args (s/cat :m map?)
:ret seq?)
(defn extract [m]
(keys m))
;; multiple takes a coll of numbers and returns a coll of numbers
(s/fdef multiple
:args (s/cat :xs (s/coll-of number?))
:ret (s/coll-of number?))
(defn multiple [xs]
(map #(* 2 %) xs))
(defn process [xs]
(-> xs
(multiple) ; seq -> seq
(extract))) ; map -> seq ... fails
;; This needs to come after the definition of the specs,
;; but before the call to process.
;; This is something I imagine can be handled automatically
;; by tooling at some point.
(stest/instrument)
;; The println is to force evaluation.
;; If not it wouldn't run because it's lazy and
;; not used for anything.
(println (process [1 2 3]))
Running this file prints (among other info):
Call to #'foo.core/extract did not conform to spec: In: [0] val: (2
4 6) fails at: [:args :m] predicate: map? :clojure.spec.alpha/spec
#object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x2b935f0d
"clojure.spec.alpha$regex_spec_impl$reify__1200#2b935f0d"]
:clojure.spec.alpha/value ((2 4 6)) :clojure.spec.alpha/args ((2 4
6)) :clojure.spec.alpha/failure :instrument
:clojure.spec.test.alpha/caller {:file "core.clj", :line 29,
:var-scope foo.core/process}
Which can be read as: A call to exctract failed because the value passed in (2 4 6) failed the predicate map?. That call happened in the file "core.clj" at line 29.
A caveat that trips people up is that instrument only checks function arguments and not return values. This is a (strange if you ask me) design decision from Rich Hickey. There's a library for that, though.
If you have a REPL session you can print a stack trace:
(clojure.stacktrace/print-stack-trace *e 30)
See http://puredanger.github.io/tech.puredanger.com/2010/02/17/clojure-stack-trace-repl/ for various different ways of printing the stack trace. You will need to have a dependency such as this in your project.clj:
[org.clojure/tools.namespace "0.2.11"]
I didn't get a stack trace using the above method, however just typing *e at the REPL will give you all the available information about the error, which to be honest didn't seem very helpful.
For the rare cases where the stack trace is not helpful I usually debug using a call to a function that returns the single argument it is given, yet has the side effect of printing that argument. I happen to call this function probe. In your case it can be put at multiple places in the threading macro.
Re-typing your example I have:
(defn extract [m]
(keys m))
(defn multiply [xs]
(mapv #(* 2 %) xs))
(defn process [xs]
(-> xs
(multiply) ; seq -> seq
(extract))) ; map -> seq ... fails ***line 21***
(println (process [1 2 3]))
;=> java.lang.ClassCastException: java.lang.Long cannot be cast
to java.util.Map$Entry, compiling:(tst/clj/core.clj:21:21)
So we get a good clue in the exception where is says the file and line/col number tst.clj.core.clj:21:21 that the extract method is the problem.
Another indispensible tool I use is Plumatic Schema to inject "gradual" type checking into clojure. The code becomes:
(ns tst.clj.core
(:use clj.core tupelo.test)
(:require
[tupelo.core :as t]
[tupelo.schema :as tsk]
[schema.core :as s]))
(t/refer-tupelo)
(t/print-versions)
(s/defn extract :- [s/Any]
[m :- tsk/Map]
(keys m))
(s/defn multiply :- [s/Num]
[xs :- [s/Num]]
(mapv #(* 2 %) xs))
(s/defn process :- s/Any
[xs :- [s/Num]]
(-> xs
(multiply) ; seq -> seq
(extract))) ; map -> seq ... fails
(println (process [1 2 3]))
clojure.lang.ExceptionInfo: Input to extract does not match schema:
[(named (not (map? [2 4 6])) m)] {:type :schema.core/error, :schema [#schema.core.One{:schema {Any Any},
:optional? false, :name m}],
:value [[2 4 6]], :error [(named (not (map? [2 4 6])) m)]},
compiling:(tst/clj/core.clj:23:17)
So, while the format of the error message is a bit lengthy, it tells right away that we passed a parameter of the wrong type and/or shape into the method extract.
Note that you need a line like this:
(s/set-fn-validation! true) ; enforce fn schemas
I create a special file test/tst/clj/_bootstrap.clj so it is always in the same place.
For more information on Plumatic Schema please see:
https://github.com/plumatic/schema
https://youtu.be/o_jtwIs2Ot8
https://github.com/plumatic/schema/wiki/Basics-Examples
https://github.com/plumatic/schema/wiki/Defining-New-Schema-Types-1.0
I want to figure out how best to create an async component, or accommodate async code in a Component-friendly way. This is the best I can come up with, and... it just doesn't feel quite right.
The Gist: take words, uppercase them and reverse them, finally print them.
Problem 1: I can't get the system to stop at the end. I expect to see a println of the individual c-chans stopping, but don't.
Problem 2: How do I properly inject deps. into the producer/consumer fns? I mean, they're not components, and I think they should not be components since they have no sensible lifecycle.
Problem 3: How do I idiomatically handle the async/pipeline-creating side-effects named a>b, and b>c? Should a pipeline be a component?
(ns pipelines.core
(:require [clojure.core.async :as async
:refer [go >! <! chan pipeline-blocking close!]]
[com.stuartsierra.component :as component]))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; PIPELINES
(defn a>b [a> b>]
(pipeline-blocking 4
b>
(map clojure.string/upper-case)
a>))
(defn b>c [b> c>]
(pipeline-blocking 4
c>
(map (comp (partial apply str)
reverse))
b>))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; PRODUCER / CONSUMER
(defn producer [a>]
(doseq [word ["apple" "banana" "carrot"]]
(go (>! a> word))))
(defn consumer [c>]
(go (while true
(println "Your Word Is: " (<! c>)))))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; SYSTEM
(defn pipeline-system [config-options]
(let [c-chan (reify component/Lifecycle
(start [this]
(println "starting chan: " this)
(chan 1))
(stop [this]
(println "stopping chan: " this)
(close! this)))]
(-> (component/system-map
:a> c-chan
:b> c-chan
:c> c-chan)
(component/using {}))))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; RUN IT!
(def system (atom nil))
(let [_ (reset! system (component/start (pipeline-system {})))
_ (a>b (:a> #system) (:b> #system))
_ (b>c (:b> #system) (:c> #system))
_ (producer (:a> #system))
_ (consumer (:c> #system))
_ (component/stop #system)])
EDIT:
I started thinking about the following, but I'm not quite sure if it's closing properly...
(extend-protocol component/Lifecycle
clojure.core.async.impl.channels.ManyToManyChannel
(start [this]
this)
(stop [this]
(close! this)))
I rewrote your example a little to make it reloadable:
Reloadable Pipeline
(ns pipeline
(:require [clojure.core.async :as ca :refer [>! <!]]
[clojure.string :as s]))
(defn upverse [from to]
(ca/pipeline-blocking 4
to
(map (comp s/upper-case
s/reverse))
from))
(defn produce [ch xs]
(doseq [word xs]
(ca/go (>! ch word))))
(defn consume [ch]
(ca/go-loop []
(when-let [word (<! ch)]
(println "your word is:" word)
(recur))))
(defn start-engine []
(let [[from to] [(ca/chan) (ca/chan)]]
(upverse to from)
(consume from)
{:stop (fn []
(ca/close! to)
(ca/close! from)
(println "engine is stopped"))
:process (partial produce to)}))
this way you can just do (start-engine) and use it to process word sequences:
REPL time
boot.user=> (require '[pipeline])
boot.user=> (def engine (pipeline/start-engine))
#'boot.user/engine
running with it
boot.user=> ((engine :process) ["apple" "banana" "carrot"])
your word is: TORRAC
your word is: ANANAB
your word is: ELPPA
boot.user=> ((engine :process) ["do" "what" "makes" "sense"])
your word is: OD
your word is: SEKAM
your word is: ESNES
your word is: TAHW
stopping it
boot.user=> ((:stop engine))
engine is stopped
;; engine would not process anymore
boot.user=> ((engine :process) ["apple" "banana" "carrot"])
nil
State Management
Depending on how you intend to use this pipeline, a state management framework like Component might not be needed at all: no need to add anything "just in case", starting and stopping the pipeline in this case is a matter of calling two functions.
However in case this pipeline is used within a larger app with more states you could definitely benefit from a state management library.
I am not a fan of Component primarily because it requires a full app buyin (which makes it a framework), but I do respect other people using it.
mount
I would recommend to either not use anything specific in case the app is small: you, for example could compose this pipeline with other pipelines / logic and kick it off from -main, but if the app is any bigger and has more unrelated states, here is all you need to do to add mount to it:
(defstate engine :start (start-engine)
:stop ((:stop engine)))
starting pipeline
boot.user=> (mount/start)
{:started ["#'pipeline/engine"]}
running with it
boot.user=> ((engine :process) ["do" "what" "makes" "sense"])
your word is: OD
your word is: SEKAM
your word is: ESNES
your word is: TAHW
stopping it
boot.user=> (mount/stop)
engine is stopped
{:stopped ["#'pipeline/engine"]}
Here is a gist with a full example that includes build.boot.
You can just download and play with it via boot repl
[EDIT]: to answer the comments
In case you are already hooked on Component, this should get you started:
(defrecord WordEngine []
component/Lifecycle
(start [component]
(merge component (start-engine)))
(stop [component]
((:stop component))
(assoc component :process nil :stop nil)))
This, on start, would create a WordEngine object that would have a :process method.
You won't be able to call it as you would a normal Clojure function: i.e. from REPL or any namespace just by :requireing it, unless you pass a reference to the whole system around which is not recommended.
So in order to call it, this WordEngine would need to be plugged into a Component system, and injected into yet another Component which can then destructure the :process function and call it.
Suppose I have a very simple .clj file on disk with the following content:
(def a 2)
(def b 3)
(defn add-two [x y] (+ x y))
(println (add-two a b))
From the context of separate program, I would like to read the above program as a list of S-Expressions, '((def a 2) (def b 3) ... (add-two a b))).
I imagine that one way of doing this involves 1. Using slurp on (io/file file-name.clj) to produce a string containing the file's contents, 2. passing that string to a parser for Clojure code, and 3. injecting the sequence produced by the parser to a list (i.e., (into '() parsed-code)).
However, this approach seems sort of clumsy and error prone. Does anyone know of a more elegant and/or idiomatic way to read a Clojure file as a list of S-Expressions?
Update: Following up on feedback from the comments section, I've decided to try the approach I mentioned on an actual source file using aphyr's clj-antlr as follows:
=> (def file-as-string (slurp (clojure.java.io/file "src/tcl/core.clj")))
=> tcl.core=> (pprint (antlr/parser "src/grammars/Clojure.g4" file-as-string))
{:parser
{:local
#object[java.lang.ThreadLocal 0x5bfcab6 "java.lang.ThreadLocal#5bfcab6"],
:grammar
#object[org.antlr.v4.tool.Grammar 0x5b8cfcb9 "org.antlr.v4.tool.Grammar#5b8cfcb9"]},
:opts
"(ns tcl.core\n (:gen-class)\n (:require [clj-antlr.core :as antlr]))\n\n(def foo 42)\n\n(defn parse-program\n \"uses antlr grammar to \"\n [program]\n ((antlr/parser \"src/grammars/Clojure.g4\") program))\n\n\n(defn -main\n \"I don't do a whole lot ... yet.\"\n [& args]\n (println \"tlc is tcl\"))\n"}
nil
Does anyone know how to transform this output to a list of S-Expressions as originally intended? That is, how might one go about squeezing valid Clojure code/data from the result of parsing with clj-antlr?
(import '[java.io PushbackReader])
(require '[clojure.java.io :as io])
(require '[clojure.edn :as edn])
;; adapted from: http://stackoverflow.com/a/24922859/6264
(defn read-forms [file]
(let [rdr (-> file io/file io/reader PushbackReader.)
sentinel (Object.)]
(loop [forms []]
(let [form (edn/read {:eof sentinel} rdr)]
(if (= sentinel form)
forms
(recur (conj forms form)))))))
(comment
(spit "/tmp/example.clj"
"(def a 2)
(def b 3)
(defn add-two [x y] (+ x y))
(println (add-two a b))")
(read-forms "/tmp/example.clj")
;;=> [(def a 2) (def b 3) (defn add-two [x y] (+ x y)) (println (add-two a b))]
)
Do you need something like this?
(let [exprs (slurp "to_read.clj")]
;; adding braces to form a proper list
(-> (str "(" (str exprs")"))
;; read-string is potentially harmful, since it evals the string
;; there exist non-evaluating readers for clojure but I don't know
;; which one are good
(read-string)
(prn)))