I'm trying to running some codes from the book "Web development with Clojure". There is a function which I can not understand:
(defn handle-upload [{:keys [filename] :as file}]
(upload-page
(if (empty? filename)
"please select a file to upload"
(try
(upload-file (gallery-path) file)
(save-thumbnail file)
(db/add-image (session/get :user) filename)
(image {:height "150px"}
(str "/img/"
(session/get :user)
"/"
thumb-prefix
(url-encode filename)))
(catch Exception ex
(str "error uploading file " (.getMessage ex)))))))
where
(defn upload-page [info]
(layout/common
[:h2 "Upload an image"]
[:p info]
(form-to {:enctype "multipart/form-data"}
[:post "/upload"]
(file-upload :file)
(submit-button "upload"))))
What is the meaning of the parameter of the function handle-upload?
And after the changing from
(defn handle-upload [{:keys [filename] :as file}]
...
to
(defn handle-upload [{:keys filename :as file}]
...
I got an error message:
java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(picture_gallery/routes/upload.clj:32:1)
Why?
{:keys [filename] :as file} means:
Take :filename key from passed argument and bind its value to filename
Leave the whole argument available as file
So if you pass:
{:filename "foo"
:somethingelse "bar"}
As an argument, then filename in the function scope will be equal to foo and the file will be equal to the whole hash map.
References:
Special Forms
Related
I'm trying to log to two different files from the same namespace with Timbre. Or if that's not possible, at least to different files from the two different namespaces.
Inspecting timbre/*config* I get the impression that I'd need two configuration maps to configure something like that. I can create another config map and use it with timbre/log* in place of the standard config map but I can't shake off the feeling that it's not how this is supposed to be used...?
(timbre/log* timbre/*config* :info "Test with standard config")
AFAIK, the easiest way is indeed to create two config maps:
(def config1
{:level :debug
:appenders {:spit1 (appenders/spit-appender {:fname "file1.log"})}})
(def config2
{:level :debug
:appenders {:spit2 (appenders/spit-appender {:fname "file2.log"})}})
(timbre/with-config config1
(info "This will print in file1") )
(timbre/with-config config2
(info "This will print in file2") )
A second way would be to write your own appender from the spit-appender:
https://github.com/ptaoussanis/timbre/blob/master/src/taoensso/timbre/appenders/core.cljx
(defn my-spit-appender
"Returns a simple `spit` file appender for Clojure."
[& [{:keys [fname] :or {fname "./timbre-spit.log"}}]]
{:enabled? true
:async? false
:min-level nil
:rate-limit nil
:output-fn :inherit
:fn
(fn self [data]
(let [{:keys [output_]} data]
(try
;; SOME LOGIC HERE TO CHOOSE THE FILE TO OUTPUT TO ...
(spit fname (str (force output_) "\n") :append true)
(catch java.io.IOException e
(if (:__spit-appender/retry? data)
(throw e) ; Unexpected error
(let [_ (have? enc/nblank-str? fname)
file (java.io.File. ^String fname)
dir (.getParentFile (.getCanonicalFile file))]
(when-not (.exists dir) (.mkdirs dir))
(self (assoc data :__spit-appender/retry? true))))))))})
I have a Ring handler that needs to:
Zip a few files
Stream the Zip to the client.
Now I have it sort of working, but only the first zipped entry gets streamed, and after that it stalls/stops. I feel it has something to do with flushing/streaming that is wrong.
Here is my (compojure) handler:
(GET "/zip" {:as request}
:query-params [order-id :- s/Any]
(stream-lessons-zip (read-string order-id) (:db request) (:auth-user request)))
Here is the stream-lessons-zip function:
(defn stream-lessons-zip
[]
(let [lessons ...];... not shown
{:status 200
:headers {"Content-Type" "application/zip, application/octet-stream"
"Content-Disposition" (str "attachment; filename=\"files.zip\"")
:body (futil/zip-lessons lessons)}))
And i use a piped-input-stream to do the streaming like so:
(defn zip-lessons
"Returns an inputstream (piped-input-stream) to be used directly in Ring HTTP responses"
[lessons]
(let [paths (map #(select-keys % [:file_path :file_name]) lessons)]
(ring-io/piped-input-stream
(fn [output-stream]
; build a zip-output-stream from a normal output-stream
(with-open [zip-output-stream (ZipOutputStream. output-stream)]
(doseq [{:keys [file_path file_name] :as p} paths]
(let [f (cio/file file_path)]
(.putNextEntry zip-output-stream (ZipEntry. file_name))
(cio/copy f zip-output-stream)
(.closeEntry zip-output-stream))))))))
So I have confirmed that the 'lessons' vector contains like 4 entries, but the zip file only contains 1 entry. Furthermore, Chrome doesn't seem to 'finalize' the download, ie. it thinks it is still downloading.
How can I fix this?
It sounds like producing a stateful stream using blocking IO is not supported by http-kit. Non-stateful streams can be done this way:
http://www.http-kit.org/server.html#async
A PR to introduce stateful streams using blocking IO was not accepted:
https://github.com/http-kit/http-kit/pull/181
It sounds like the option to explore is to use a ByteArrayOutputStream to fully render the zip file to memory, and then return the buffer that produces. If this endpoint isn't highly trafficked and the zip file it produces is not large (< 1 gb) then this might work.
So, it's been a few years, but that code still runs in production (ie. it works). So I made it work back then, but forgot to mention it here (and forgot WHY it works, to be honest,.. it was very much trial/error).
This is the code now:
(defn zip-lessons
"Returns an inputstream (piped-input-stream) to be used directly in Ring HTTP responses"
[lessons {:keys [firstname surname order_favorite_name company_name] :as annotation
:or {order_favorite_name ""
company_name ""
firstname ""
surname ""}}]
(debug "zipping lessons" (count lessons))
(let [paths (map #(select-keys % [:file_path :file_name :folder_number]) lessons)]
(ring-io/piped-input-stream
(fn [output-stream]
; build a zip-output-stream from a normal output-stream
(with-open [zip-output-stream (ZipOutputStream. output-stream)]
(doseq [{:keys [file_path file_name folder_number] :as p} paths]
(let [f (cio/as-file file_path)
baos (ByteArrayOutputStream.)]
(if (.exists f)
(do
(debug "Adding entry to zip:" file_name "at" file_path)
(let [zip-entry (ZipEntry. (str (if folder_number (str folder_number "/") "") file_name))]
(.putNextEntry zip-output-stream zip-entry)
(.close baos)
(.writeTo baos zip-output-stream)
(.closeEntry zip-output-stream)
(.flush zip-output-stream)
(debug "flushed")))
(warn "File '" file_name "' at '" file_path "' does not exist, not adding to zip file!"))))
(.flush zip-output-stream)
(.flush output-stream)
(.finish zip-output-stream)
(.close zip-output-stream))))))
I'm having some trouble to get started with Light Table.
Here's my code (Clojure)
(ns prova1-ed.core
(:gen-class))
(use 'clojure.java.io)
(defn -main [& args]
(println "Type the name of the file to read: ")
(let [fileName (read-line)]
(let [rdr (reader fileName)]
(doseq [line (line-seq rdr)]
(println line)
)))
)
I'm sure it works. I've tested with lein run. As you can see, the program should read a file which the name is given by the user.
I've tried CTRL+SPACE in Light Table, but this is what I receive:
ERROR: Unhandled REPL handler exception processing message {:data {:auto? false, :pos {:line 14, :ch 1}, :mime "text/x-clojure", :tags [:editor.clj :editor.clojure], :type-name "Clojure", :line-ending "\r\n", :ns user, :path "C:\\Users\\Tiago\\Documents\\Clojure\\prova1_ed\\src\\prova1_ed\\core.clj", :print-length nil, :name "core.clj", :local true, :code "(ns prova1-ed.core\n (:gen-class))\n\n(use 'clojure.java.io)\n\n(defn -main [& args]\n\n (println \"Type the name of the file to read: \")\n\n (let [fileName (read-line)]\n (let [rdr (reader fileName)]\n (doseq [line (line-seq rdr)]\n (println line)\n )))\n)\n"}, :id 90, :op editor.eval.clj.sonar, :session 65d1da68-a730-4ffe-9365-9527726384e3}
How can i run it in the Light Tables' enviroment, so that I can input the file name?
TLDR
I don't think you can run (read-line) in Light Table as it'd have to add explicit support for allowing input. There's no standard input basically.
An Alternative
I'd suggest you modify your -main function to accept an explicit file-name argument instead of trying to read it from a standard input that isn't available.
I've got a Clojure webapp that I work on in Light Table.
I've got a -main function in a namespace named my-app.web. It looks something like this:
(defn -main [& [port]]
(let [port (Integer. (or port (env :port) 5000))
store (cookie/cookie-store {:key (env :session-secret)})]
(jetty/run-jetty (-> #'secured-app
wrap-with-logging
wrap-current-user
wrap-current-auth
wrap-error-page
(site {:session {:store store}}))
{:port port :join? false})))
In a separate file I've named light-table-start.clj, I've got the following code to run my app inside Light Table:
(require '[my-app.web :as web])
(require '[ring.adapter.jetty :as jetty])
(defonce server (web/-main "5000"))
;; (.start server)
;; (.stop server)
I run the Eval: Eval editor contents command (Ctrl+Shift+Enter on Windows and Linux or ⌘+Shift+Enter on Mac OS) the first time I want to run my app (or later, if the connection is closed for some reason). When I want to start or stop the server I can just highlight the code on the respective commented lines and run the Eval: Eval a form in editor command (Ctrl+Enter on Windows and Linux or ⌘+Enter on Mac OS).
hopefully this is something simple for the more experienced out there. I am using clj-http and trying to pass the command line arg int it (to take a URL). I am an absolute Clojure beginer but I have managed to pass the args through to a ptintln which works.
(ns foo.core
(:require [clj-http.client :as client]))
(defn -main
[& args]
(def url (str args))
(println url)
(def resp (client/get url))
(def headers (:headers resp))
(def server (headers "server"))
(println server))
Error message
Ants-MacBook-Pro:target ant$ lein run "http://www.bbc.com"
("http://www.bbc.com")
Exception in thread "main" java.net.MalformedURLException: no protocol: ("http://www.bbc.com")
This works
(def resp (client/get "http://www.bbc.com"))
thanks in advance.
args is a list, which means that calling str on it returns the representation of the list, complete with parentheses and inner quotes, as you can see in your error trace:
(println (str '("http://www.bbc.com")))
;; prints ("http://www.bbc.com")
Of course, URLs don't start with parentheses and quotes, which is why the JVM tells you your URL is malformed.
What you really want to pass to get is not the string representation of your argument list, but your first argument:
(let [url (first args)]
(client/get url)) ;; Should work!
In addition, you should never use def calls within functions -- they create or rebind vars at the toplevel of your namespace, which don't want.
What you should be using instead is let forms, which create local variables (like url in my example). For more information on let, look at http://clojure.org/special_forms.
I'd probably structure your code like so:
(defn -main
[& args]
(let [url (first args)
resp (client/get url)
server (get-in resp [:headers "server"])]
(println url)
(println server)))
I know there are a lot of related questions, I have read them but still have not gained some fundamental understanding of how to read-process-write. Take the following function for example which uses clojure-csv library to parse a line
(defn take-csv
"Takes file name and reads data."
[fname]
(with-open [file (reader fname)]
(doseq [line (line-seq file)]
(let [record (parse-csv line)]))))
What I would like to obtain is data read into some collection as a result of (def data (take-csv "file.csv")) and later to process it. So basically my question is how do I return record or rather a list of records.
"doseq" is often used for operations with side effect. In your case to create collection of records you can use "map":
(defn take-csv
"Takes file name and reads data."
[fname]
(with-open [file (reader fname)]
(doall (map (comp first csv/parse-csv) (line-seq file)))))
Better parse the whole file at ones to reduce code:
(defn take-csv
"Takes file name and reads data."
[fname]
(with-open [file (reader fname)]
(csv/parse-csv (slurp file))))
You also can use clojure.data.csv instead of clojure-csv.core. Only should rename parse-csv to take-csv in previous function.
(defn put-csv [fname table]
(with-open [file (writer fname)]
(csv/write-csv file table)))
With all the things you can do with .csv files, I suggest using clojure-csv or clojure.data.csv. I mostly use clojure-csv to read in a .csv file.
Here are some code snippets from a utility library I use with most of my Clojure programs.
from util.core
(ns util.core
^{:author "Charles M. Norton",
:doc "util is a Clojure utilities directory"}
(:require [clojure.string :as cstr])
(:import java.util.Date)
(:import java.io.File)
(:use clojure-csv.core))
(defn open-file
"Attempts to open a file and complains if the file is not present."
[file-name]
(let [file-data (try
(slurp file-name)
(catch Exception e (println (.getMessage e))))]
file-data))
(defn ret-csv-data
"Returns a lazy sequence generated by parse-csv.
Uses open-file which will return a nil, if
there is an exception in opening fnam.
parse-csv called on non-nil file, and that
data is returned."
[fnam]
(let [csv-file (open-file fnam)
inter-csv-data (if-not (nil? csv-file)
(parse-csv csv-file)
nil)
csv-data
(vec (filter #(and pos? (count %)
(not (nil? (rest %)))) inter-csv-data))]
(if-not (empty? csv-data)
(pop csv-data)
nil)))
(defn fetch-csv-data
"This function accepts a csv file name, and returns parsed csv data,
or returns nil if file is not present."
[csv-file]
(let [csv-data (ret-csv-data csv-file)]
csv-data))
Once you've read in a .csv file, then what you do with its contents is another matter. Usually, I am taking .csv "reports" from one financial system, like property assessments, and formatting the data to be uploaded into a database of another financial system, like billing.
I will often either zipmap each .csv row so I can extract data by column name (having read in the column names), or even make a sequence of zipmap'ped .csv rows.
Just to add this good answers, here is a full example
First, add clojure-csv into your dependencies
(ns scripts.csvreader
(:require [clojure-csv.core :as csv]
[clojure.java.io :as io]))
(defn take-csv
"Takes file name and reads data."
[fname]
(with-open [file (io/reader fname)]
(-> file
(slurp)
(csv/parse-csv))))
usage
(take-csv "/path/youfile.csv")