I have the following collection of hash maps:
{a:"Completed" b:1 c:"Friday" d:4}
{a:"Started" b:1 c:"Monday" d:4}
{a:"In Progress" b:1 c:"Sunday" d:1}
{a:"Completed" b:3 c:"Tuesday" d:9}
How can I convert this to a CSV file in clojure?
i.e.
a,b,c,d
Completed,1,Friday,4
Started,1,Monday,4
In Progress,1,Sunday,1
Completed,3,Tuesday,9
Any help with this would be much appreciated.
You didn't explicitly say you were starting with JSON, but let's assume you are.
I would use cheshire
to parse your JSON string to get, in this case, a vector of maps
representing your data. We'll assume you've done that for sake
of simplicity and just use a var called data to avoid
cluttering the example.
Now we can create a vector of vectors, then use
clojure.data.csv to save the
results to a file.
You can try this out at the REPL using lein try.
Follow the setup instructions for lein try if you don't already have it, then run
lein try clojure.data.csv 0.1.2. Once you're in a REPL with this dependency:
(require '[clojure.data.csv :as csv] '[clojure.java.io :as io])
(def data
[{:a "Completed" :b 1 :c "Friday" :d 4}
{:a "Started" :b 1 :c "Monday" :d 4}
{:a "In Progress" :b 1 :c "Sunday" :d 1}
{:a "Completed" :b 3 :c "Tuesday" :d 9}])
(defn write-csv [path row-data]
(let [columns [:a :b :c :d]
headers (map name columns)
rows (mapv #(mapv % columns) row-data)]
(with-open [file (io/writer path)]
(csv/write-csv file (cons headers rows)))))
(write-csv "/tmp/results.csv" data)
Now you can see the results of your handiwork:
$ cat /tmp/results.csv
a,b,c,d
Completed,1,Friday,4
Started,1,Monday,4
In Progress,1,Sunday,1
Completed,3,Tuesday,9
I read all the answers but I wanted more generic and reusable solution.
Here is what I came up with. You can use write-csv and maps->csv-data independently or together via write-csv-from-maps. More importantly, you don't need to hard code headers. Just pass your vector of maps and you should be all set.
A. DIY Approach
(require '[clojure.data.csv :as csv]
'[clojure.java.io :as io])
(defn write-csv
"Takes a file (path, name and extension) and
csv-data (vector of vectors with all values) and
writes csv file."
[file csv-data]
(with-open [writer (io/writer file)]
(csv/write-csv writer csv-data)))
(defn maps->csv-data
"Takes a collection of maps and returns csv-data
(vector of vectors with all values)."
[maps]
(let [columns (-> maps first keys)
headers (mapv name columns)
rows (mapv #(mapv % columns) maps)]
(into [headers] rows)))
(defn write-csv-from-maps
"Takes a file (path, name and extension) and a collection of maps
transforms data (vector of vectors with all values)
writes csv file."
[file maps]
(->> maps maps->csv-data (write-csv file)))
B. Semantic CSV Approach
Alternatively, you can use semantic-csv library.
maps->csv-data would then become semantic-csv.core/vectorize
(require '[clojure.data.csv :as csv]
'[clojure.java.io :as io]
'[semantic-csv.core :as sc])
(defn write-csv
"Takes a file (path, name and extension) and
csv-data (vector of vectors with all values) and
writes csv file."
[file csv-data]
(with-open [writer (io/writer file)]
(csv/write-csv writer csv-data)))
(defn write-csv-from-maps
"Takes a file (path, name and extension) and a collection of maps
transforms data (vector of vectors with all values)
writes csv file."
[file maps]
(->> maps sc/vectorize (write-csv file)))
--
Finally:
(def data
[{:a "Completed" :b 1 :c "Friday" :d 4}
{:a "Started" :b 1 :c "Monday" :d 4}
{:a "In Progress" :b 1 :c "Sunday" :d 1}
{:a "Completed" :b 3 :c "Tuesday" :d 9}])
(write-csv "/tmp/results.csv" data)
Hope that could be useful to someone!
Use data.json to convert the json to a sequence of clojure maps.
Use map destructuring to convert to a sequence of strings: (map #(let [{a :a b :b c :c d :d} %] (str a "," b "," c "," d, "\n")) <your sequence of clojure maps>).
Dump the sequence of strings to a file
(def data [{:a "Completed" :b 1 :c "Friday" :d 4}
{:a "Started" :b 1 :c "Monday" :d 4}
{:a "In Progress" :b 1 :c "Sunday" :d 1}
{:a "Completed" :b 3 :c "Tuesday" :d 9}])
(for [i (range (count data))]
(if (= i 0)
(do
(spit "data.csv" (str (clojure.string/join "," (map name (keys (data i)))) "\n") :append true)
(spit "data.csv" (str (clojure.string/join "," (vals (data i))) "\n") :append true))
(spit "data.csv" (str (clojure.string/join "," (vals (data i))) "\n") :append true)))
To see the results:
$ cat data.csv
c,b,d,a
Friday,1,4,Completed
Monday,1,4,Started
Sunday,1,1,In Progress
Tuesday,3,9,Completed
This answer is better because this method takes everything from the data and you don't have to hardcode anything. And also this method does not make use of any extra libraries except clojure.string/join.
I know this code is not csv,
But If you want to save it as xlsx, this worked for me.
(use 'dk.ative.docjure.spreadsheet)
(defn save-to-xls [f ms]
(let [vs (let [-keys (sort (keys (first ms)))
ordered-vals (apply juxt -keys)]
(concat [(map name -keys)]
(map ordered-vals ms)))
wb (create-workbook "sheet1" vs)]
(save-workbook! f wb))))
Use the clojure-csv library, to easily read and write well-formed csv.
Related
I was wondering if there was a way to access the arguments value of a thread-first macro in Clojure while it is being executed on.
for example:
(def x {:a 1 :b 2})
(-> x
(assoc :a 20) ;; I want the value of x after this step
(assoc :b (:a x))) ;; {:a 20, :b 1}
It has come to my attention that this works:
(-> x
(assoc :a 20)
((fn [x] (assoc x :b (:a x))))) ;; {:a 20, :b 20}
But are there any other ways to do that?
You can use as->:
(let [x {:a 1 :b 2}]
(as-> x it
(assoc it :a 20)
(assoc it :b (:a it))))
In addition to akond's comment, note that using as-> can get quite confusing quickly. I recommend either extracting a top level function for these cases, or trying to use as-> in -> only:
(-> something
(process-something)
(as-> $ (do-something $ (very-complicated $)))
(finish-processing))
I'm trying to coerce a map using prismatic-schema (1.0.4)
I'm trying to coerce
{:a 1}
to
{:b 1}
Using a custom matcher with the schema:
{:b s/Int}
But this code isn't working:
(require '[schema.core :as s])
(require '[schema.coerce :as coerce])
((coerce/coercer {:b s/Int}
(fn [s]
(when (= s s/Keyword)
(fn [x]
(if (= x :a)
:b
x)))))
{:a 1})
Output:
#schema.utils.ErrorContainer{:error {:b missing-required-key, :a disallowed-key}}
I tried debugging it by running the following code which matches everything in the schema and outputs the current value and schema being matched:
((coerce/coercer {:b s/Int}
(fn [s]
(when true
(fn [x]
(println s x)
x))))
{:a 1})
Output:
{:b Int} {:a 1}
=>
#schema.utils.ErrorContainer{:error {:b missing-required-key, :a disallowed-key}}
It looks as though the matcher bombs out as soon as it gets to the map?
Schema first breaks your map up into pieces that match up to the schema, then coerces each MapEntry to the corresponding MapEntry schema, and so on down. This breakdown fails in your case, so you never get to the key.
To accomplish what you want, you'll have to attach the coercion to the map schema and use e.g. clojure.set/rename-keys in your coercion function:
(def Foo {:b s/Int})
((coerce/coercer
Foo
(fn [s]
(when (= s Foo)
#(clojure.set/rename-keys % {:a :b}))))
{:a 1})
I have this function:
(defn dissoc-all [m kv]
(let [[k & ks] kv]
(dissoc m k ks)))
Where m is the map and kv is the vector of keys. I use it like this:
(dissoc-all {:a 1 :b 2} [:a :b])
=>{:b 2}
This is not what I've expected. ks has :b but I don't know why it is not being use by dissoc. Anyone can help me with this?
Edit: Added question is that why is this not triggering the 3rd overload of dissoc, which is dissoc [map key & ks]?
Changed name from dissoc-in to dissoc-all as noisesmith have said, -in is not a proper name for this and I agree.
This won't work because ks is a collection of all the elements in kv after the first. So instead of :b it is [:b].
Instead, you can just use apply:
(defn dissoc-in [m vs]
(apply dissoc m vs))
Also, dissoc-in is an odd name for this function, because the standard functions with -in in the name all do nested access, and this does not use the keys to do any nested access of the map.
Why not something like this?
(defn dissoc-all [m ks]
(apply dissoc m ks))
(dissoc-all {:a 1 :b 2} [:a :b])
=> {}
The reason the third overlod of dissoc is not getting called is because it does not expect a collection of keys like [:a :b] - it expects just the keys.
For example:
(dissoc {:a "a" :b "b" :c "c" :d "d"} :a :b :c)
=> {:d "d"}
Further to noisesmith's answer:
You're being confused by the overloads/arities of dissoc, which have this simple effect:
[m & ks]
"Returns a new map of the same (hashed/sorted) type,
that does not contain a mapping for any of ks. "
The explicit arities for no keys and one key are for performance. Many clojure functions are so organised, and the documentation follows the organisation, not the underlying idea.
Now, the action of
(dissoc-all {:a 1 :b 2} [:a :b])
;{:b 2}
is to bind
k to :a
ks to [:b]
Note the latter. The example removes the :a but fails to remove the [:b], which isn't there.
You can use apply to crack open ks:
(defn dissoc-all [m kk]
(let [[k & ks] kk]
(apply dissoc m k ks)))
(dissoc-all {:a 1 :b 2} [:a :b])
;{}
... or, better, do as #noisesmith does and short-circuit the destructuring, using apply at once.
I'm very new to Clojure and learning Clojure by reading
good open source code. So I choose Ring and starting to read
the code but got stuck in assoc-query-params function.
(which is located in ring.middleware/params.clj)
And I could not understand why "merge-with" is used.
Can anybody help me to understand this code snippet?
(defn- assoc-query-params
"Parse and assoc parameters from the query string with the request."
[request encoding]
; I think (merge request (some-form)) is enough
; but the author used merge-with with merge function.
(merge-with merge request
(if-let [query-string (:query-string request)]
(let [params (parse-params query-string encoding)]
{:query-params params, :params params})
{:query-params {}, :params {}})))
Here's the description of the merge function: reworded it says that if a key is met more than once than value in the latest map will be selected. In the example that you posted that would mean that values of :query-params :params will be taken as is from the tail of the function instead of combining them with what's in the request.
Let's look at the example:
(def m {:a {:a-key1 "value1"} :b {:b-key1 "value3"} :c {}})
(def m2 {:a {:a-key2 "value2"} :b {}})
(merge m m2)
;-> {:a {:a-key2 "value2"}, :b {}, :c {}}
(merge-with merge m m2)
;-> {:a {:a-key1 "value1", :a-key2 "value2"}, :b {:b-key1 "value3"} :c {}}
So (merge-with merge ...) construct gives us a way to merge maps in the map. You can look at it this way: merge-with will group all key/value pairs by the key (:a :b :c in our example) and apply merge to their values.
{:a (merge {:a-key1 "value1"} {:a-key2 "value2"})
:b (merge {:b-key1 "value3"} {})
:c (merge {})}
Having handled that I think that the original intention of the assoc-query-params author is to extend :query-params and :params instead of completely replacing them.
I have a map in Clojure something like this:
(def stuff #{
{:a "help" :b "goodbye"}
{:c "help2" :b "goodbye"}
{:a "steve" :b "goodbye"}
{:c "hello2" :b "sue"}
})
: and I want to provide a search so that:
(search stuff "help")
: would return :
#{
{:a "help" :b "goodbye"}
{:c "help2" :b "goodbye"}
}
: What is the simplest way to do this?
user=> (defn search [s q] (select #(some (partial re-find (re-pattern q)) (vals %)) s))
#'user/search
user=> (search stuff "help")
#{{:a "help", :b "goodbye"} {:c "help2", :b "goodbye"}}
This does the trick.
Full text search is a different topic, but if you can live with regexps I would use something like:
(defn match [re e]
(re-find re (:a e))
(defn search [re m]
(into #{} (filter (partial match re) m)))
(filter (comp #{"help"} :a) stuff): the freshly-composed function first calls :a on the target, then calls #{"help"} on the result: this returns a truthy value iff the :a attribute is exactly "help".
Converting this into a set, and encapsulating it in a function with the arguments you want to tweak, is left as an exercise for the reader. Frankly, though, the code is so simple that it might well be shorter and more readable to rewrite it each time you want to do a "search".