ClojureScript - convert arbitrary JavaScript object to Clojure Script map - clojure

I am trying to convert a Javascript object to a Clojure. However, I get the following error :
(js/console.log (js->clj e)) ;; has no effect
(pprint (js->clj e)) ;; No protocol method IWriter.-write defined for type object: [object Geoposition]
Yes, this object comes from the Geolocation API. I suppose that I have to extend IEncodeClojure and IWriter, but I have no clue how.
For instance adding the following :
(extend-protocol IEncodeClojure
Coordinates
(-js->clj [x options]
(println "HERE " x options)))
Yields an error when loading my code : Uncaught TypeError: Cannot read property 'prototype' of undefined

The accepted answer wasn't working for me with the javascript object window.performance.timing. This is because Object.keys() doesn't actually return the props for the PerformanceTiming object.
(.keys js/Object (.-timing (.-performance js/window))
; => #js[]
This is despite the fact that the props of PerformanceTiming are indeed iterable with a vanilla JavaScript loop:
for (a in window.performance.timing) {
console.log(a);
}
// navigationStart
// unloadEventStart
// unloadEventEnd
// ...
The following is what I came up with to convert an arbitrary JavaScript object to a ClojureScript map. Note the use of two simple Google Closure functions.
goog.typeOf wraps typeof, which isn't normally accessible to us in ClojureScript. I use this to filter out props which are functions.
goog.object.getKeys wraps for (prop in obj) {...}, building up an array result which we can reduce into a map.
Solution (flat)
(defn obj->clj
[obj]
(-> (fn [result key]
(let [v (goog.object/get obj key)]
(if (= "function" (goog/typeOf v))
result
(assoc result key v))))
(reduce {} (.getKeys goog/object obj))))
Solution (recursive)
Update: This solution will work for nested maps.
(defn obj->clj
[obj]
(if (goog.isObject obj)
(-> (fn [result key]
(let [v (goog.object/get obj key)]
(if (= "function" (goog/typeOf v))
result
(assoc result key (obj->clj v)))))
(reduce {} (.getKeys goog/object obj)))
obj))

js->clj only works for Object, anything with custom constructor (see type) will be returned as is.
see: https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/core.cljs#L9319
I suggest doing this instead:
(defn jsx->clj
[x]
(into {} (for [k (.keys js/Object x)] [k (aget x k)])))
UPDATE for correct solution see Aaron's answer, gotta use goog.object

Two approaches that do not require writing custom conversion functions - they both employ standard JavaScript functions to loose the custom prototype and thus enable clj->js to work correctly.
Using JSON serialization
This approach just serializes to JSON and immediately parses it:
(js->clj (-> e js/JSON.stringify js/JSON.parse))
Advantages:
does not require any helper function
works for nested objects, with/without prototype
supported in every browser
Disadvantages:
performance might be a problem in critical pieces of codebase
will strip any non-serializable values, like functions.
Using Object.assign()
This approach is based on Object.assign() and it works by copying all the properties from e onto a fresh, plain (no custom prototype) #js {}.
(js->clj (js/Object.assign #js {} e))
Advantages:
does not require any helper function
Disadvantages:
works on flat objects, if there is another nested object withing e, it won't be converted by clj->js.
Object.assign() is not supported by old browsers, most notably - IE.

(defn obj->clj
([obj]
(obj->clj obj :keywordize-keys false))
([obj & opts]
(let [{:keys [keywordize-keys]} opts
keyfn (if keywordize-keys keyword str)]
(if (and (not-any? #(% obj) [inst? uuid?])
(goog.isObject obj))
(-> (fn [result k]
(let [v (goog.object/get obj k)]
(if (= "function" (goog/typeOf v))
result
(assoc result (keyfn k) (apply obj->clj v opts)))))
(reduce {} (.getKeys goog/object obj)))
obj))))
Small problem with the original above is that JS treats #inst and #uuid as objects. Seems like those are the only tagged literals in clojure
I also added the option to keywordize keys by looking at js->clj source

Related

Convert from data.json to cheshire

I am completely new to Clojure. I still struggle with reading functions sometimes.
I am trying to change this function to use checkshire.
Here is my attempt :
defn- json->messages [json]
(let [records (:amazon.aws.sqs/records (cheshire/decode
json
:key-fn key-reader
:value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
The function key-reader function:
(def ^:private
key-reader
(memoize (fn [key]
(let [kebab-key (if (= "md5OfBody" key)
"md5-of-body"
(csk/->kebab-case key))]
(keyword "amazon.aws.sqs" kebab-key)))))
The function :
(def ^:private
value-reader
(memoize (fn [key value]
(if (= key :amazon.aws.sqs/receipt-handle)
value-reader
value))))
I than call the function like so :
(json->messages msg)
msg is a json string.
However I am getting the error below with that attempt :
Execution error (ArityException) at tech.matterindustries.titan.ion.lambda.sqs-receive/json->messages (sqs_receive.clj:36).
Wrong number of args (5) passed to: cheshire.core/parse-smile
You are sending the wrong number of args to cheshire.core/parse-smile. Do you have a piece of sample data?
Please also keep your code clean & formatted, like this:
(defn- json->messages
[json]
(let [records (:amazon.aws.sqs/records (cheshire/decode json :key-fn key-reader :value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
I could not find decode in the Cheshire docs, but in the source it has this:
(def decode "Alias to parse-string for clojure-json users" parse-string)
I am disappointed in their incomplete docs.
A quick google shows the docs:
(parse-string string & [key-fn array-coerce-fn])
Returns the Clojure object corresponding to the given JSON-encoded string.
An optional key-fn argument can be either true (to coerce keys to keywords),
false to leave them as strings, or a function to provide custom coercion.
The array-coerce-fn is an optional function taking the name of an array field,
and returning the collection to be used for array values.
This may not be clear. What it means is there are 3 legal ways to call parse-string:
(parse-string <json-str>)
(parse-string <json-str> <key-fn>)
(parse-string <json-str> <key-fn> <array-coerce-fn>)
So you can call it with 1, 2, or 3 args. You cannot add in :key-fn or :value-fn map keys, as in your example.
Please also note that your key-reader and value-reader look like they do not match what cheshire/read-string is expecting.

Keywordize with Java Maps

I'm using Nashorn in my Clojure application. When evaluating a javascript code it returns a ScriptObjectMirror which in turn implements the java.util.Map interface.
I'm trying to keywordize the maps keys using clojure.walk/keywordize-keys, however it seems to have no effect in maps other than IPersistentMap.
So I tried the follwing code:
(into {}
(for [[k v] eval-result]
[(keyword k) v]))
However it doesn't work recursively. Any idea on how to keywordize java maps? Shouldn't clojure.walk/keywordize-keys work with the Map interface?
Thanks.
I've found a way
(defn keywordize [object]
(if (instance? java.util.Map object)
(into {}
(for [[k v] object]
[(keyword k) (keywordize v)]))
object
)
)

Why in this example calling (f arg) and calling the body of f explicitly yields different results?

First, I have no experience with CS and Clojure is my first language, so pardon if the following problem has a solution, that is immediately apparent for a programmer.
The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. My approach revolves around a) storing temporarily the names of the atoms as strings in an atom itself; b) changing those strings to symbols with a function; c) using a function to add and create new atoms. The problem pertains to step "c": calling the function does not create new atoms, but using its body does create them.
All steps taken in the REPL are below (comments follow code blocks):
user=> (def atom-pool
#_=> (atom ["a1" "a2"]))
#'user/atom-pool
'atom-pool is the atom that stores intermediate to-be atoms as strings.
user=> (defn atom-symbols []
#_=> (mapv symbol (deref atom-pool)))
#'user/atom-symbols
user=> (defmacro populate-atoms []
#_=> (let [qs (vec (remove #(resolve %) (atom-symbols)))]
#_=> `(do ~#(for [s qs]
#_=> `(def ~s (atom #{}))))))
#'user/populate-atoms
'populate-atoms is the macro, that defines those atoms. Note, the purpose of (remove #(resolve %) (atom-symbols)) is to create only yet non-existing atoms. 'atom-symbols reads 'atom-pool and turns its content to symbols.
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(nil nil nil)
Here it is confirmed that there are no 'a1', 'a2', 'a-new' atoms as of yet.
user=> (defn new-atom [a]
#_=> (do
#_=> (swap! atom-pool conj a)
#_=> (populate-atoms)))
#'user/new-atom
'new-atom is the function, that first adds new to-be atom as string to `atom-pool. Then 'populate-atoms creates all the atoms from 'atom-symbols function.
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 nil)
Here we see that 'a1 'a2 were created as clojure.lang.Var$Unbound just by defining a function, why?
user=> (new-atom "a-new")
#'user/a2
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 nil)
Calling (new-atom "a-new") did not create the 'a-new atom!
user=> (do
#_=> (swap! atom-pool conj "a-new")
#_=> (populate-atoms))
#'user/a-new
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 #'user/a-new)
user=>
Here we see that resorting explicitly to 'new-atom's body did create the 'a-new atom. 'a-new is a type of clojure.lang.Atom, but 'a1 and 'a2 were skipped due to already being present in the namespace as clojure.lang.Var$Unbound.
Appreciate any help how to make it work!
EDIT: Note, this is an example. In my project the 'atom-pool is actually a collection of maps (atom with maps). Those maps have keys {:name val}. If a new map is added, then I create a corresponding atom for this map by parsing its :name key.
"The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. "
This sounds like a solution looking for a problem. I would generally suggest you try another way of achieving whatever the actual functionality is without generating vars at runtime, but if you must, you should use intern and leave out the macro stuff.
You cannot solve this with macros since macros are expanded at compile time, meaning that in
(defn new-atom [a]
(do
(swap! atom-pool conj a)
(populate-atoms)))
populate-atoms is expanded only once; when the (defn new-atom ...) form is compiled, but you're attempting to change its expansion when new-atom is called (which necessarily happens later).
#JoostDiepenmaat is right about why populate-atoms is not behaving as expected. You simply cannot do this using macros, and it is generally best to avoid generating vars at runtime. A better solution would be to define your atom-pool as a map of keywords to atoms:
(def atom-pool
(atom {:a1 (atom #{}) :a2 (atom #{})}))
Then you don't need atom-symbols or populate-atoms because you're not dealing with vars at compile-time, but typical data structures at run-time. Your new-atom function could look like this:
(defn new-atom [kw]
(swap! atom-pool assoc kw (atom #{})))
EDIT: If you don't want your new-atom function to override existing atoms which might contain actual data instead of just #{}, you can check first to see if the atom exists in the atom-pool:
(defn new-atom [kw]
(when-not (kw #atom-pool)
(swap! atom-pool assoc kw (atom #{}))))
I've already submitted one answer to this question, and I think that that answer is better, but here is a radically different approach based on eval:
(def atom-pool (atom ["a1" "a2"]))
(defn new-atom! [name]
(load-string (format "(def %s (atom #{}))" name)))
(defn populate-atoms! []
(doseq [x atom-pool]
(new-atom x)))
format builds up a string where %s is substituted with the name you're passing in. load-string reads the resulting string (def "name" (atom #{})) in as a data structure and evals it (this is equivalent to (eval (read-string "(def ...)
Of course, then we're stuck with the problem of only defining atoms that don't already exist. We could change the our new-atom! function to make it so that we only create an atom if it doesn't already exist:
(defn new-atom! [name]
(when-not (resolve (symbol name))
(load-string (format "(def %s (atom #{}))" name name))))
The Clojure community seems to be against using eval in most cases, as it is usually not needed (macros or functions will do what you want in 99% of cases*), and eval can be potentially unsafe, especially if user input is involved -- see Brian Carper's answer to this question.
*After attempting to solve this particular problem using macros, I came to the conclusion that it either cannot be done without relying on eval, or my macro-writing skills just aren't good enough to get the job done with a macro!
At any rate, I still think my other answer is a better solution here -- generally when you're getting way down into the nuts & bolts of writing macros or using eval, there is probably a simpler approach that doesn't involve metaprogramming.

Handling user input validation with preconditions functionally in Clojure

I write a game server and have to check that messages arriving from users are correct and valid. That means they have to be of correct syntax, comply to parameter formatting and are semantically correct, i.e. complying to the game's rules.
My goal is to have an expressive, functional way without throwing exceptions that allows composability as good as possible.
I am aware of other similar questions but they either refer to {:pre ..., :post ...} which I dislike as only stringified information can be processed once the exception is thrown, or refer exception handling in general which I dislike because Clojure should be able to do this kind of task, or they refer to Haskell's monadic style with e.g. a maybe Monad à la (-> [err succ]) which I also dislike because Clojure should be able to handle this kind of task without needing a Monad.
So far I do the ugly way using cond as pre-condition checker and error codes which I then send back to the client who send the request:
(defn msg-handler [client {:keys [version step game args] :as msg}]
(cond
(nil? msg) :4001
(not (valid-message? msg)) :4002
(not (valid-version? version)) :5050
(not (valid-step? step)) :4003
(not (valid-game-id? game)) :4004
(not (valid-args? args)) :4007
:else (step-handler client step game args)))
and similar...
(defn start-game [game-id client]
(let [games #*games*
game (get games game-id)
state (:state game)
players (:players game)]
(cond
(< (count players) 2) :4120
(= state :started) :4093
(= state :finished) :4100
:else ...)))
Another way would be to write a macro, similar to defn and {:pre} but instead of throwing an AssertionError throw an ex-info with a map, but again: opposed to exception throwing.
The heart of your question seems to be in your comment:
Yeah, maybe I should explain what about this is "ugly": It's the not-well composability of just return an actual result, which can be represented almost arbitrarily and a numbered keyword as error. My callstack looks like this: (-> msg msg-handler step-handler step-N sub-function), and all of them could return this one error type, but none of them returns the same success type and another thing is, that I have to manually design if en error return type should short-circuit or has to be expected to do some intermediate work (undo data changes or notify other clients in parallel)
Clojure 1.5+ has the some-> threading macro to get rid of nil? check boilerplate. We just need to gently adjust the code to substitute the nil? check for a check of our choice.
(defmacro pred->
"When predicate is not satisfied, threads expression into the first form
(via ->), and when that result does not satisfy the predicate, through the
next, etc. If an expression satisfies the predicate, that expression is
returned without evaluating additional forms."
[expr pred & forms]
(let [g (gensym)
pstep (fn [step] `(if (~pred ~g) ~g (-> ~g ~step)))]
`(let [~g ~expr
~#(interleave (repeat g) (map pstep forms))]
~g)))
Note with this definition, (pred-> expr nil? form1 form2 ...) is (some-> expr form1 form2...). But now we can use other predicates.
Example
(defn foo [x] (if (even? x) :error-even (inc x)))
(defn bar [x] (if (zero? (mod x 3)) :error-multiple-of-three (inc x)))
(pred-> 1 keyword? foo) ;=> 2
(pred-> 1 keyword? foo foo) ;=> :error-even
(pred-> 1 keyword? foo foo foo) ;=> :error-even
(pred-> 1 keyword? foo bar foo bar) ;=> 5
(pred-> 1 keyword? foo bar foo bar foo bar foo bar) ;=> :error-multiple-of-three
Your use case
A flexible choice would be to make a wrapper for validation errors
(deftype ValidationError [msg])
Then you can wrap your error code/messages as in (->ValidationError 4002) and change your threading to
(pred-> msg #(instance? ValidationError %)
msg-handler step-handler step-N sub-function)

Interop from clojure a with non-standard iterative java API

I am working in clojure with a java class which provides a retrieval API for a domain specific binary file holding a series of records.
The java class is initialized with a file and then provides a .query method which returns an instance of an inner class which has only one method .next, thus not playing nicely with the usual java collections API. Neither the outer nor inner class implements any interface.
The .query method may return null instead of the inner class. The .next method returns a record string or null if no further records are found, it may return null immediately upon the first call.
How do I make this java API work well from within clojure without writing further java classes?
The best I could come up with is:
(defn get-records
[file query-params]
(let [tr (JavaCustomFileReader. file)]
(if-let [inner-iter (.query tr query-params)] ; .query may return null
(loop [it inner-iter
results []]
(if-let [record (.next it)]
(recur it (conj results record))
results))
[])))
This gives me a vector of results to work with the clojure seq abstractions. Are there other ways to expose a seq from the java API, either with lazy-seq or using protocols?
Without dropping to lazy-seq:
(defn record-seq
[q]
(take-while (complement nil?) (repeatedly #(.next q))))
Instead of (complement nil?) you could also just use identity if .next does not return boolean false.
(defn record-seq
[q]
(take-while identity (repeatedly #(.next q))))
I would also restructure a little bit the entry points.
(defn query
[rdr params]
(when-let [q (.query rdr params)]
(record-seq q)))
(defn query-file
[file params]
(with-open [rdr (JavaCustomFileReader. file)]
(doall (query rdr params))))
Seems like a good fit for lazy-seq:
(defn query [file query]
(.query (JavaCustomFileReader. file) query))
(defn record-seq [query]
(when query
(when-let [v (.next query)]
(cons v (lazy-seq (record-seq query))))))
;; usage:
(record-seq (query "filename" "query params"))
Your code is not lazy as it would be if you were using Iterable but you can fill the gap with lazy-seq as follows.
(defn query-seq [q]
(lazy-seq
(when-let [val (.next q)]
(cons val (query-seq q)))))
Maybe you shoul wrap the query method to protect yourself from the first null value as well.