I am completely new to Clojure. I still struggle with reading functions sometimes.
I am trying to change this function to use checkshire.
Here is my attempt :
defn- json->messages [json]
(let [records (:amazon.aws.sqs/records (cheshire/decode
json
:key-fn key-reader
:value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
The function key-reader function:
(def ^:private
key-reader
(memoize (fn [key]
(let [kebab-key (if (= "md5OfBody" key)
"md5-of-body"
(csk/->kebab-case key))]
(keyword "amazon.aws.sqs" kebab-key)))))
The function :
(def ^:private
value-reader
(memoize (fn [key value]
(if (= key :amazon.aws.sqs/receipt-handle)
value-reader
value))))
I than call the function like so :
(json->messages msg)
msg is a json string.
However I am getting the error below with that attempt :
Execution error (ArityException) at tech.matterindustries.titan.ion.lambda.sqs-receive/json->messages (sqs_receive.clj:36).
Wrong number of args (5) passed to: cheshire.core/parse-smile
You are sending the wrong number of args to cheshire.core/parse-smile. Do you have a piece of sample data?
Please also keep your code clean & formatted, like this:
(defn- json->messages
[json]
(let [records (:amazon.aws.sqs/records (cheshire/decode json :key-fn key-reader :value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
I could not find decode in the Cheshire docs, but in the source it has this:
(def decode "Alias to parse-string for clojure-json users" parse-string)
I am disappointed in their incomplete docs.
A quick google shows the docs:
(parse-string string & [key-fn array-coerce-fn])
Returns the Clojure object corresponding to the given JSON-encoded string.
An optional key-fn argument can be either true (to coerce keys to keywords),
false to leave them as strings, or a function to provide custom coercion.
The array-coerce-fn is an optional function taking the name of an array field,
and returning the collection to be used for array values.
This may not be clear. What it means is there are 3 legal ways to call parse-string:
(parse-string <json-str>)
(parse-string <json-str> <key-fn>)
(parse-string <json-str> <key-fn> <array-coerce-fn>)
So you can call it with 1, 2, or 3 args. You cannot add in :key-fn or :value-fn map keys, as in your example.
Please also note that your key-reader and value-reader look like they do not match what cheshire/read-string is expecting.
Related
I'm new to Clojure and as a learning exercise I'm trying to write a function that validates the present of keys in a map.
when I try to run the code below I get an error saying
java.lang.UnsupportedOperationException: nth not supported on this type: Keyword
(def record {:name "Foobar"})
(def validations [:name (complement nil?)])
(defn validate
[candidate [key val]]
(val (get candidate key))
(def actual (every? (partial validate record) validations))
(= true actual)
As I understand I'm partial applying the validate function and asserting every validations function on the map - but it doesn't seem to work - so I must be misunderstanding something?
The error is coming from the destructuring that you are using in validate: [key val]. Under the hood, destructuring uses the function nth, and that's what's failing.
Your issue is that you are passing to every? a list of [keyword validation-function]. And every? is iterating over each element of that list and calling the partially applied validate function with it. That means that your validate is called first with the keyword :name and that throws an exception, because you can not extract a [key val] pair out of the keyword :name, causing the exception.
To fix it, you need to make your validations list a list of lists as so:
(def record {:name "Foobar"})
(def validations [[:name (complement nil?)]])
(defn validate
[candidate [key val]]
(val (get candidate key)))
(def actual (every? (partial validate record) validations))
(= true actual)
;; => true
That way, every? is now iterating over each pair of [keyword validation-function], one at a time, and calling validate with that. Since this is a pair, it can be destructured into a [key val] and everything works.
And just so you know, in newer Clojure versions (1.6 and above), there is now a function called some? which is equivalent to (complement nil?).
every? takes a collection as the second argument, and so does your validate function. Since you're passing a vector to every?, validate is being called on the contents of the vector (that is, :name and (complement nil?)). You're also missing a closing paren in the definition of validate. Try the following:
(def record {:name "Foobar"})
(def validations [:name (complement nil?)])
(defn validate
[candidate [key val]]
(val (get candidate key)))
(def actual (every? (partial validate record) [validations]))
(= true actual)
BTW, you could use some? instead of (complement nil?)
I'm developing a mini-social media API where the user is allowed to insert a new profile, connect two profiles together (like friends) and then receive recommendations based on the "friends of my friends" rule.
Right now I'm trying to create the API for Profile.
I have an atom that holds a list of maps, one for each profile.
(def profiles (atom ()))
(defn create [request]
(swap! profiles conj {:id (get-in request [:body "id"])
:name (get-in request [:body "name"])
:age (get-in request [:body "age"])
:recommendable (get-in request [:body "recommendable"])
:friends (list)
})
(created "")
)
I was trying to develop the find-by-id for the GET http verb for the API when I stumbled into a problem. How can I get the values from the maps within said list so I can apply functions to it?
For instance, here I was trying to use the filter function to return me only the maps that contained a given id. But I keep getting an error:
(defn find-by-id [id]
(filter #(= (:id %) id) profiles)
)
Dont know how to create ISeq from: clojure.lang.Atom
It seems to me that filter is not applicable to an Atom.
Same thing happens to remove:
(defn delete-by-id [id]
(swap! profiles (remove #(= (:id %) id) profiles))
)
When I try with #profiles I get an empty array as a result. And to make things worst when I tried the filter function using REPL it worked just fine.
Which leaves me wondering what am I missing here.
Could anyone please tell me what's going on?
Thanks in advance.
The first one fails because, as it says, atoms aren't a sequence, which filter is expecting.
You need to get the sequence out of the atom before you can filter it:
; I'm dereferencing the atom using # to get the list of profiles that it holds
(defn find-by-id [id]
(filter #(= (:id %) id) #profiles))
Note though, this isn't optimal. You're relying on the state of profiles that can change at seemingly random times (if you have asynchronous processes swap!ping it). It may complicate debugging since you can't get a good handle on the data before it's passed to filter. It also isn't good for the function to rely on profiles being an atom, since that's irrelevant to its function, and you may change your design later. It would be more future proof to make this function rely purely on its parameters and have no knowledge of the atom:
(defn find-by-id [id profiles]
(filter #(= (:id %) id) profiles))
; Then call it like this. I renamed your atom here
(find-by-id some-id #profile-atom)
Your second example fails because swap! accepts a function as its second argument. I think you meant to use reset!, which changes the value of the atom regardless of what it was before:
(defn delete-by-id [id]
(reset! profiles (remove #(= (:id %) id) #profiles)))
Although, this isn't optimal either. If you want to update an atom based on a previous state, use swap! instead and supply an updating function:
(defn delete-by-id [id]
(swap! profile-atom (fn [profiles] (remove #(= (:id %) id)) profiles)))
Or, slightly more succinctly:
(defn delete-by-id [id]
(swap! profile-atom (partial remove #(= (:id %) id))))
I'm partially applying remove to make a function. The old state of the atom is passed as the last argument to remove.
I am trying to convert a Javascript object to a Clojure. However, I get the following error :
(js/console.log (js->clj e)) ;; has no effect
(pprint (js->clj e)) ;; No protocol method IWriter.-write defined for type object: [object Geoposition]
Yes, this object comes from the Geolocation API. I suppose that I have to extend IEncodeClojure and IWriter, but I have no clue how.
For instance adding the following :
(extend-protocol IEncodeClojure
Coordinates
(-js->clj [x options]
(println "HERE " x options)))
Yields an error when loading my code : Uncaught TypeError: Cannot read property 'prototype' of undefined
The accepted answer wasn't working for me with the javascript object window.performance.timing. This is because Object.keys() doesn't actually return the props for the PerformanceTiming object.
(.keys js/Object (.-timing (.-performance js/window))
; => #js[]
This is despite the fact that the props of PerformanceTiming are indeed iterable with a vanilla JavaScript loop:
for (a in window.performance.timing) {
console.log(a);
}
// navigationStart
// unloadEventStart
// unloadEventEnd
// ...
The following is what I came up with to convert an arbitrary JavaScript object to a ClojureScript map. Note the use of two simple Google Closure functions.
goog.typeOf wraps typeof, which isn't normally accessible to us in ClojureScript. I use this to filter out props which are functions.
goog.object.getKeys wraps for (prop in obj) {...}, building up an array result which we can reduce into a map.
Solution (flat)
(defn obj->clj
[obj]
(-> (fn [result key]
(let [v (goog.object/get obj key)]
(if (= "function" (goog/typeOf v))
result
(assoc result key v))))
(reduce {} (.getKeys goog/object obj))))
Solution (recursive)
Update: This solution will work for nested maps.
(defn obj->clj
[obj]
(if (goog.isObject obj)
(-> (fn [result key]
(let [v (goog.object/get obj key)]
(if (= "function" (goog/typeOf v))
result
(assoc result key (obj->clj v)))))
(reduce {} (.getKeys goog/object obj)))
obj))
js->clj only works for Object, anything with custom constructor (see type) will be returned as is.
see: https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/core.cljs#L9319
I suggest doing this instead:
(defn jsx->clj
[x]
(into {} (for [k (.keys js/Object x)] [k (aget x k)])))
UPDATE for correct solution see Aaron's answer, gotta use goog.object
Two approaches that do not require writing custom conversion functions - they both employ standard JavaScript functions to loose the custom prototype and thus enable clj->js to work correctly.
Using JSON serialization
This approach just serializes to JSON and immediately parses it:
(js->clj (-> e js/JSON.stringify js/JSON.parse))
Advantages:
does not require any helper function
works for nested objects, with/without prototype
supported in every browser
Disadvantages:
performance might be a problem in critical pieces of codebase
will strip any non-serializable values, like functions.
Using Object.assign()
This approach is based on Object.assign() and it works by copying all the properties from e onto a fresh, plain (no custom prototype) #js {}.
(js->clj (js/Object.assign #js {} e))
Advantages:
does not require any helper function
Disadvantages:
works on flat objects, if there is another nested object withing e, it won't be converted by clj->js.
Object.assign() is not supported by old browsers, most notably - IE.
(defn obj->clj
([obj]
(obj->clj obj :keywordize-keys false))
([obj & opts]
(let [{:keys [keywordize-keys]} opts
keyfn (if keywordize-keys keyword str)]
(if (and (not-any? #(% obj) [inst? uuid?])
(goog.isObject obj))
(-> (fn [result k]
(let [v (goog.object/get obj k)]
(if (= "function" (goog/typeOf v))
result
(assoc result (keyfn k) (apply obj->clj v opts)))))
(reduce {} (.getKeys goog/object obj)))
obj))))
Small problem with the original above is that JS treats #inst and #uuid as objects. Seems like those are the only tagged literals in clojure
I also added the option to keywordize keys by looking at js->clj source
I am working in clojure with a java class which provides a retrieval API for a domain specific binary file holding a series of records.
The java class is initialized with a file and then provides a .query method which returns an instance of an inner class which has only one method .next, thus not playing nicely with the usual java collections API. Neither the outer nor inner class implements any interface.
The .query method may return null instead of the inner class. The .next method returns a record string or null if no further records are found, it may return null immediately upon the first call.
How do I make this java API work well from within clojure without writing further java classes?
The best I could come up with is:
(defn get-records
[file query-params]
(let [tr (JavaCustomFileReader. file)]
(if-let [inner-iter (.query tr query-params)] ; .query may return null
(loop [it inner-iter
results []]
(if-let [record (.next it)]
(recur it (conj results record))
results))
[])))
This gives me a vector of results to work with the clojure seq abstractions. Are there other ways to expose a seq from the java API, either with lazy-seq or using protocols?
Without dropping to lazy-seq:
(defn record-seq
[q]
(take-while (complement nil?) (repeatedly #(.next q))))
Instead of (complement nil?) you could also just use identity if .next does not return boolean false.
(defn record-seq
[q]
(take-while identity (repeatedly #(.next q))))
I would also restructure a little bit the entry points.
(defn query
[rdr params]
(when-let [q (.query rdr params)]
(record-seq q)))
(defn query-file
[file params]
(with-open [rdr (JavaCustomFileReader. file)]
(doall (query rdr params))))
Seems like a good fit for lazy-seq:
(defn query [file query]
(.query (JavaCustomFileReader. file) query))
(defn record-seq [query]
(when query
(when-let [v (.next query)]
(cons v (lazy-seq (record-seq query))))))
;; usage:
(record-seq (query "filename" "query params"))
Your code is not lazy as it would be if you were using Iterable but you can fill the gap with lazy-seq as follows.
(defn query-seq [q]
(lazy-seq
(when-let [val (.next q)]
(cons val (query-seq q)))))
Maybe you shoul wrap the query method to protect yourself from the first null value as well.
If I have the request "size=3&mean=1&sd=3&type=pdf&distr=normal" what's the idiomatic way of writing the function (defn request->map [request] ...) that takes this request and
returns a map {:size 3, :mean 1, :sd 3, :type pdf, :distr normal}
Here is my attempt (using clojure.walk and clojure.string):
(defn request-to-map
[request]
(keywordize-keys
(apply hash-map
(split request #"(&|=)"))))
I am interested in how others would solve this problem.
Using form-decode and keywordize-keys:
(use 'ring.util.codec)
(use 'clojure.walk)
(keywordize-keys (form-decode "hello=world&foo=bar"))
{:foo "bar", :hello "world"}
Assuming you want to parse HTTP request query parameters, why not use ring? ring.middleware.params contains what you want.
The function for parameter extraction goes like this:
(defn- parse-params
"Parse parameters from a string into a map."
[^String param-string encoding]
(reduce
(fn [param-map encoded-param]
(if-let [[_ key val] (re-matches #"([^=]+)=(.*)" encoded-param)]
(assoc-param param-map
(codec/url-decode key encoding)
(codec/url-decode (or val "") encoding))
param-map))
{}
(string/split param-string #"&")))
You can do this easily with a number of Java libraries. I'd be hesitant to try to roll my own parser unless I read the URI specs carefully and made sure I wasn't missing any edge cases (e.g. params appearing in the query twice with different values). This uses jetty-util:
(import '[org.eclipse.jetty.util UrlEncoded MultiMap])
(defn parse-query-string [query]
(let [params (MultiMap.)]
(UrlEncoded/decodeTo query params "UTF-8")
(into {} params)))
user> (parse-query-string "size=3&mean=1&sd=3&type=pdf&distr=normal")
{"sd" "3", "mean" "1", "distr" "normal", "type" "pdf", "size" "3"}
Can also use this library for both clojure and clojurescript: https://github.com/cemerick/url
user=> (-> "a=1&b=2&c=3" cemerick.url/query->map clojure.walk/keywordize-keys)
{:a "1", :b "2", :c "3"}
Yours looks fine. I tend to overuse regexes, so I would have solved it as
(defn request-to-keywords [req]
(into {} (for [[_ k v] (re-seq #"([^&=]+)=([^&]+)" req)]
[(keyword k) v])))
(request-to-keywords "size=1&test=3NA=G")
{:size "1", :test "3NA=G"}
Edit: try to stay away from clojure.walk though. I don't think it's officially deprecated, but it's not very well maintained. (I use it plenty too, though, so don't feel too bad).
I came across this question when constructing my own site and the answer can be a bit different, and easier, if you are passing parameters internally.
Using Secretary to handle routing: https://github.com/gf3/secretary
Parameters are automatically extracted to a map in :query-params when a route match is found. The example given in the documentation:
(defroute "/users/:id" [id query-params]
(js/console.log (str "User: " id))
(js/console.log (pr-str query-params)))
(defroute #"/users/(\d+)" [id {:keys [query-params]}]
(js/console.log (str "User: " id))
(js/console.log (pr-str query-params)))
;; In both instances...
(secretary/dispach! "/users/10?action=delete")
;; ... will log
;; User: 10
;; "{:action \"delete\"}"
You can use ring.middleware.params. Here's an example with aleph:
user=> (require '[aleph.http :as http])
user=> (defn my-handler [req] (println "params:" (:params req)))
user=> (def server (http/start-server (wrap-params my-handler)))
wrap-params creates an entry in the request object called :params. If you want the query parameters as keywords, you can use ring.middleware.keyword-params. Be sure to wrap with wrap-params first:
user=> (require '[ring.middleware.params :refer [wrap-params]])
user=> (require '[ring.middleware.keyword-params :refer [wrap-keyword-params])
user=> (def server
(http/start-server (wrap-keyword-params (wrap-params my-handler))))
However, be mindful that this includes a dependency on ring.