I've got the following function in webnoir that serves up an image from disk.
(defpage [:get "/GetPhoto"] {:keys [photoName]}
(def file (io/input-stream (str photoName ".jpg")))
(resp/set-headers {"Content-Type" "image/jpeg"} file))
I assume I should close the file, but when I add (.close file) to the end of the function, I get an error java.io.IOException: Stream closed when accessing the URL. What's the right way to handle this?
The def on line two is very likely not what you want. I would recommend replacing it with a with-open statement. This is defining a new symbol for the whole namespace each time this function runs. As a general rule calls to def are used at the top level or inside a macro.
(defpage [:get "/GetPhoto"] {:keys [photoName]}
(with-open [file (io/input-stream (str photoName ".jpg"))]
(resp/set-headers {"Content-Type" "image/jpeg"} file))
Obviously you can't close the file at the end of the function, because all you do with noir/ring is to return a data structure describing the response. Ring then uses this data structure to actually respond to the client. In this case it sees a stream and tries to read from it. When you close it (explicit or with with-open) this will fail.
I would expect you don't have to do anything and ring will close the stream after exhausting it.
You don't want nested def. You don't want nested def. You don't want nested def. You don't want...
Related
I'd like to run an interactive CLI program from within Clojure (e.g., vim) and be able to interact with it.
In bash and other programming languages, I can do that with
vim > `tty`
I tried to do the same in Clojure:
(require '[clojure.java.shell :as shell])
(shell/sh "vim > `tty`")
but it just opens vim without giving me tty.
Background: I'm developing a Clojure CLI tool which parses emails and lets a user edit the parsed data before saving them on the disk. It works the following way:
Read a file with email content and parse it. Each email is stored as a separate file.
Show a user the parsed data and let the user edit the data in vim. Internally I create a temporary file with the parsed data, but I don't mind doing it another way if that would solve my issue.
After a user finished editing the parsed data (they might decide to keep it as it is) append the data to a file on a disk. So all parsed data are saved to the same file.
Go to 1st step if there are any files with emails left.
This code relies on Clojure Java interop to make use of Java's ProcessBuilder class.
(defn -main
[]
;use doseq instead of for because for is lazily evaluated
(doseq [i [1 2 3]]
;extract current directory from system variable
(let [file-name (str "test" i ".txt")
working-directory (trim-newline (:out (sh "printenv" "PWD")))]
(spit file-name "")
;this is where fun begins. We use ProcessBuilder to forward commands to terminal
;we pass a list of commands and their arguments to its constructor
(let [process-builder (java.lang.ProcessBuilder. (list "vim" (str working-directory "/" file-name)))
;inherit is a configuration constant
inherit (java.lang.ProcessBuilder$Redirect/INHERIT)]
;we configure input, output and error redirection
(.redirectOutput process-builder inherit)
(.redirectError process-builder inherit)
(.redirectInput process-builder inherit)
;waitFor used to block execution until vim is closed
(.waitFor (.start process-builder))
)
;additional processing here
)
)
;not necessary but script tends to hang for around 30 seconds at end of its execution
;so this command is used to terminate it instantly
(System/exit 0)
)
Is there a way to get a dump of all the source code I have entered into a repl session. I have created a bunch of functions using (defn ...) but did it 'on the fly' without entering them in a text file (IDE) first.
Is there a convenience way to get the source back out of the repl session?
I note that:
(dir user)
will give me a printed list of type:
user.proxy$java.lang.Object
so I can't appear to get that printed list into a Seq for mapping a function like 'source' over. And even if I could then:
(source my-defined-fn)
returns "source not found"...even though I personally entered it in to the repl session.
Any way of doing this? Thanks.
Sorry, but I suspect the answer is no :-/
The best you get is scrolling up in the repl buffer to where you defined it. The source function works by looking in the var's metadata for the file and line number where the functions code is (or was last time it was evaluated), opening the file, and printing the lines. It looks like this:
...
(when-let [filepath (:file (meta v))]
(when-let [strm (.getResourceAsStream (RT/baseLoader) filepath)]
(with-open [rdr (LineNumberReader. (InputStreamReader. strm))]
(dotimes [_ (dec (:line (meta v)))] (.readLine rdr))
...
Not including the full source in the metadata was done on purpose to save memory in the normal case, though it does make it less convenient here.
I have the following functions to process large files with constant memory usage.
(defn lazy-helper
"Processes a java.io.Reader lazily"
[reader]
(lazy-seq
(if-let [line (.readLine reader)]
(cons line (lazy-helper reader))
(do (.close reader) nil))))
(defn lazy-lines
"Return a lazy sequence with the lines of the file"
[^String file]
(lazy-helper (io/reader file)))
This works very well when the processing part is filtering or other mapping or reducing operation that works with lazy sequences quite well.
The problem starts when I have process the file and for example send every line over a channel to worker processes.
(thread
(doseq [line lines]
(blocking-producer work-chan line)))
The obvious downside of this is to process the file eagerly causing a heap overflow.
I was wondering what is the best way of iterating over each line in a file and do some IO with the lines.
It seems this might be unrelated how the file IO is handled, doseq should not hold onto the head of the reader.
As #joost-diepenmaat pointed out this might not be related to the file IO and he is right.
It seems the way I am working with JSON serialization and deserialization is the root cause here.
You can use (line-seq rdr) which "returns the lines of text from rdr as a lazy sequence of strings".
This turned out to be a problem with the JSON handling of the code and not the file IO. Explanation in the original post.
I'm having trouble downloading images using Clojure, there seems to be an issue with the way the following code works: -
(defn download-image [url filename]
(->> (slurp url) (spit filename)))
This will 'download' the file to the location I specify but the file is unreadable by any image application I try to open it with (for example, attempting to open it in a web browser just return a blank page, attempting to open it in Preview (osx) says it's a corrupted file)
I'm thinking this is might be because slurp should only really be used for text files rather than binary files
Could anyone point me in the right direction for my code to work properly? Any help would be greatly appreciated!
slurp uses java.io.Reader underneath, which will convert the representation to a string, and this is typically not compatible with binary data. Look for examples that use input-stream instead. In some ways, this can be better, because you can transfer the image from the input buffer to the output buffer without having to read the entire thing into memory.
edit
Since people seem to find this question once in awhile and I needed to rewrite this code again. I thought I'd add an example. Note, this does not stream the data, it collects it into memory and returns it an array of bytes.
(require '[clojure.java.io :as io])
(defn blurp [f]
(let [dest (java.io.ByteArrayOutputStream.)]
(with-open [src (io/input-stream f)]
(io/copy src dest))
(.toByteArray dest)))
Test...
(use 'clojure.test)
(deftest blurp-test
(testing "basic operation"
(let [src (java.io.ByteArrayInputStream. (.getBytes "foo" "utf-8"))]
(is (= "foo" (-> (blurp src) (String. "utf-8")))))))
Example...
user=> (blurp "http://www.lisperati.com/lisplogo_256.png")
#<byte[] [B#15671adf>
I have some printlns I need to capture from a Clojure program and I was wondering how I could capture the output?
I have tried:
(binding [a *out*]
(println "h")
a
)
: but this doesn't work
(with-out-str (println "this should return as a string"))
Just to expand a little on Michiel's answer, when you want to capture output to a file you can combine with-out-str with spit.
When you don't want to build up a huge string in memory before writing it out then you can use with-out-writer from the clojure.contrib.io library.
with-out-writer is a macro that nicely encapsulates the correct opening and closing of the file resource and the binding of a writer on that file to *out* while executing the code in its body.
Michiel's exactly right. Since I can't add code in a comment on his answer, here's what with-out-str does under the covers, so you can compare it with your attempt:
user=> (macroexpand-1 '(with-out-str (println "output")))
(clojure.core/let [s__4091__auto__ (new java.io.StringWriter)]
(clojure.core/binding [clojure.core/*out* s__4091__auto__]
(println "output")
(clojure.core/str s__4091__auto__)))
Your code was binding the existing standard output stream to a variable, printing to that stream, and then asking the stream for its value via the variable; however, the value of the stream was of course not the bytes that had been printed to it. So with-out-str binds a newly created StringWriter to *out* temporarily, and finally queries the string value of that temporary writer.