I'm new to Clojure and I'm stuck on how to 'expand' a function's optional args so they can be sent to another function that uses optional args (but wants those args as keywords not a seq of keywords).
I'm parsing xml and if I hard code values as below my function works, it walks the xml and finds the value of 'title':
; zd was required like this
[clojure.data.zip.xml :as zd]
; ...
(defn get-node-value [parsed-xml & node-path]
(zd/xml-> (zip/xml-zip parsed-xml) :item :title zd/text))
(get-node-value parsed-xml)
What I want to do is use 'node-path' to pass in any number of keywords, but when written as below it comes in as a sequence of keywords so it throws an exception:
(defn get-node-value [parsed-xml & node-path]
(zd/xml-> (zip/xml-zip parsed-xml) node-path zd/text))
(get-node-value parsed-xml :item :title)
; ClassCastException clojure.lang.ArraySeq cannot be cast to clojure.lang.IFn clojure.data.zip/fixup-apply (zip.clj:73)
thanks!
I think you are looking for apply (http://clojuredocs.org/clojure_core/clojure.core/apply)
(defn get-node-value [parsed-xml & node-path]
(let [params (concat node-path [zd/text])]
(apply zd/xml-> (zip/xml-zip parsed-xml) params)))
Maybe you want:
(defn get-node-value [parsed-xml & node-path]
(zd/xml-> (zip/xml-zip parsed-xml) ((apply comp (reverse node-path))) zd/text))
I can't test the above, so am working from analogy:
(-> {:a {:c 1}, :b 2} ((apply comp (reverse [:a :c]))))
1
However, if dAni's solution works, ignore me!
Related
Let's say I have several vectors
(def coll-a [{:name "foo"} ...])
(def coll-b [{:name "foo"} ...])
(def coll-c [{:name "foo"} ...])
and that I would like to see if the names of the first elements are equal.
I could
(= (:name (first coll-a)) (:name (first coll-b)) (:name (first coll-c)))
but this quickly gets tiring and overly verbose as more functions are composed. (Maybe I want to compare the last letter of the first element's name?)
To directly express the essence of the computation it seems intuitive to
(apply = (map (comp :name first) [coll-a coll-b coll-c]))
but it leaves me wondering if there's a higher level abstraction for this sort of thing.
I often find myself comparing / otherwise operating on things which are to be computed via a single composition applied to multiple elements, but the map syntax looks a little off to me.
If I were to home brew some sort of operator, I would want syntax like
(-op- (= :name first) coll-a coll-b coll-c)
because the majority of the computation is expressed in (= :name first).
I'd like an abstraction to apply to both the operator & the functions applied to each argument. That is, it should be just as easy to sum as compare.
(def coll-a [{:name "foo" :age 43}])
(def coll-b [{:name "foo" :age 35}])
(def coll-c [{:name "foo" :age 28}])
(-op- (+ :age first) coll-a coll-b coll-c)
; => 106
(-op- (= :name first) coll-a coll-b coll-c)
; => true
Something like
(defmacro -op-
[[op & to-comp] & args]
(let [args' (map (fn [a] `((comp ~#to-comp) ~a)) args)]
`(~op ~#args')))
Is there an idiomatic way to do this in clojure, some standard library function I could be using?
Is there a name for this type of expression?
For your addition example, I often use transduce:
(transduce
(map (comp :age first))
+
[coll-a coll-b coll-c])
Your equality use case is trickier, but you could create a custom reducing function to maintain a similar pattern. Here's one such function:
(defn all? [f]
(let [prev (volatile! ::no-value)]
(fn
([] true)
([result] result)
([result item]
(if (or (= ::no-value #prev)
(f #prev item))
(do
(vreset! prev item)
true)
(reduced false))))))
Then use it as
(transduce
(map (comp :name first))
(all? =)
[coll-a coll-b coll-c])
The semantics are fairly similar to your -op- macro, while being both more idiomatic Clojure and more extensible. Other Clojure developers will immediately understand your usage of transduce. They may have to investigate the custom reducing function, but such functions are common enough in Clojure that readers can see how it fits an existing pattern. Also, it should be fairly transparent how to create new reducing functions for use cases where a simple map-and-apply wouldn't work. The transducing function can also be composed with other transformations such as filter and mapcat, for cases when you have a more complex initial data structure.
You may be looking for the every? function, but I would enhance clarity by breaking it down and naming the sub-elements:
(let [colls [coll-a coll-b coll-c]
first-name (fn [coll] (:name (first coll)))
names (map first-name colls)
tgt-name (first-name coll-a)
all-names-equal (every? #(= tgt-name %) names)]
all-names-equal => true
I would avoid the DSL, as there is no need and it makes it much harder for others to read (since they don't know the DSL). Keep it simple:
(let [colls [coll-a coll-b coll-c]
vals (map #(:age (first %)) colls)
result (apply + vals)]
result => 106
I don't think you need a macro, you just need to parameterize your op function and compare functions. To me, you are pretty close with your (apply = (map (comp :name first) [coll-a coll-b coll-c])) version.
Here is one way you could make it more generic:
(defn compare-in [op to-compare & args]
(apply op (map #(get-in % to-compare) args)))
(compare-in + [0 :age] coll-a coll-b coll-c)
(compare-in = [0 :name] coll-a coll-b coll-c)
;; compares last element of "foo"
(compare-in = [0 :name 2] coll-a coll-b coll-c)
I actually did not know you can use get on strings, but in the third case you can see we compare the last element of each foo.
This approach doesn't allow the to-compare arguments to be arbitrary functions, but it seems like your use case mainly deals with digging out what elements you want to compare, and then applying an arbitrary function to those values.
I'm not sure this approach is better than the transducer version supplied above (certainly not as efficient), but I think it provides a simpler alternative when that efficiency is not needed.
I would split this process into three stages:
transform items in collections into the data in collections you want to operate
on - (map :name coll);
Operate on transformed items in collections, returning collection of results - (map = transf-coll-a transf-coll-b transf-coll-c)
Finally, selecting which result in resulting collection to return - (first calculated-coll)
When playing with collections, I try to put more than one item into collection:
(def coll-a [{:name "foo" :age 43} {:name "bar" :age 45}])
(def coll-b [{:name "foo" :age 35} {:name "bar" :age 37}])
(def coll-c [{:name "foo" :age 28} {:name "bra" :age 30}])
For example, matching items by second char in :name and returning result for items in second place:
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp #(nth % 1) :name)
op =
fetch second]
(fetch (apply map op (map #(map transf-fn %) colls))))
;; => false
In transducers world you can use sequence function which also works on multiple collections:
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp (map :name) (map #(nth % 1)))
op =
fetch second]
(fetch (apply sequence (map op) (map #(sequence transf-fn %) colls))))
Calculate sum of ages (for all items at the same level):
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp (map :age))
op +
fetch identity]
(fetch (apply sequence (map op) (map #(sequence transf-fn %) colls))))
;; => (106 112)
For the most part I understand what Clojure is telling me with it's error messages. But I am still clueless as to find out where the error happened.
Here is an example of what I mean
(defn extract [m]
(keys m))
(defn multiple [xs]
(map #(* 2 %) xs))
(defn process [xs]
(-> xs
(multiple) ; seq -> seq
(extract))) ; map -> seq ... fails
(process [1 2 3])
Statically typed languages would now tell me that I tried to pass a sequence to a function that expects a map on line X. And Clojure does this in a way:
ClassCastException java.lang.Long cannot be cast to java.util.Map$Entry
But I still have no idea where the error happened. Obviously for this instance it's easy because there are just 3 functions involved, you can easily just read through all of them but as programs grow bigger this gets old very quickly.
Is there a way find out where the errors happened other than just proof reading the code from top to bottom? (which is my current approach)
You can use clojure.spec. It is still in alpha, and there's still a bunch of tooling support coming (hopefully), but instrumenting functions works well.
(ns foo.core
(:require
;; For clojure 1.9.0-alpha16 and higher, it is called spec.alpha
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest]))
;; Extract takes a map and returns a seq
(s/fdef extract
:args (s/cat :m map?)
:ret seq?)
(defn extract [m]
(keys m))
;; multiple takes a coll of numbers and returns a coll of numbers
(s/fdef multiple
:args (s/cat :xs (s/coll-of number?))
:ret (s/coll-of number?))
(defn multiple [xs]
(map #(* 2 %) xs))
(defn process [xs]
(-> xs
(multiple) ; seq -> seq
(extract))) ; map -> seq ... fails
;; This needs to come after the definition of the specs,
;; but before the call to process.
;; This is something I imagine can be handled automatically
;; by tooling at some point.
(stest/instrument)
;; The println is to force evaluation.
;; If not it wouldn't run because it's lazy and
;; not used for anything.
(println (process [1 2 3]))
Running this file prints (among other info):
Call to #'foo.core/extract did not conform to spec: In: [0] val: (2
4 6) fails at: [:args :m] predicate: map? :clojure.spec.alpha/spec
#object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x2b935f0d
"clojure.spec.alpha$regex_spec_impl$reify__1200#2b935f0d"]
:clojure.spec.alpha/value ((2 4 6)) :clojure.spec.alpha/args ((2 4
6)) :clojure.spec.alpha/failure :instrument
:clojure.spec.test.alpha/caller {:file "core.clj", :line 29,
:var-scope foo.core/process}
Which can be read as: A call to exctract failed because the value passed in (2 4 6) failed the predicate map?. That call happened in the file "core.clj" at line 29.
A caveat that trips people up is that instrument only checks function arguments and not return values. This is a (strange if you ask me) design decision from Rich Hickey. There's a library for that, though.
If you have a REPL session you can print a stack trace:
(clojure.stacktrace/print-stack-trace *e 30)
See http://puredanger.github.io/tech.puredanger.com/2010/02/17/clojure-stack-trace-repl/ for various different ways of printing the stack trace. You will need to have a dependency such as this in your project.clj:
[org.clojure/tools.namespace "0.2.11"]
I didn't get a stack trace using the above method, however just typing *e at the REPL will give you all the available information about the error, which to be honest didn't seem very helpful.
For the rare cases where the stack trace is not helpful I usually debug using a call to a function that returns the single argument it is given, yet has the side effect of printing that argument. I happen to call this function probe. In your case it can be put at multiple places in the threading macro.
Re-typing your example I have:
(defn extract [m]
(keys m))
(defn multiply [xs]
(mapv #(* 2 %) xs))
(defn process [xs]
(-> xs
(multiply) ; seq -> seq
(extract))) ; map -> seq ... fails ***line 21***
(println (process [1 2 3]))
;=> java.lang.ClassCastException: java.lang.Long cannot be cast
to java.util.Map$Entry, compiling:(tst/clj/core.clj:21:21)
So we get a good clue in the exception where is says the file and line/col number tst.clj.core.clj:21:21 that the extract method is the problem.
Another indispensible tool I use is Plumatic Schema to inject "gradual" type checking into clojure. The code becomes:
(ns tst.clj.core
(:use clj.core tupelo.test)
(:require
[tupelo.core :as t]
[tupelo.schema :as tsk]
[schema.core :as s]))
(t/refer-tupelo)
(t/print-versions)
(s/defn extract :- [s/Any]
[m :- tsk/Map]
(keys m))
(s/defn multiply :- [s/Num]
[xs :- [s/Num]]
(mapv #(* 2 %) xs))
(s/defn process :- s/Any
[xs :- [s/Num]]
(-> xs
(multiply) ; seq -> seq
(extract))) ; map -> seq ... fails
(println (process [1 2 3]))
clojure.lang.ExceptionInfo: Input to extract does not match schema:
[(named (not (map? [2 4 6])) m)] {:type :schema.core/error, :schema [#schema.core.One{:schema {Any Any},
:optional? false, :name m}],
:value [[2 4 6]], :error [(named (not (map? [2 4 6])) m)]},
compiling:(tst/clj/core.clj:23:17)
So, while the format of the error message is a bit lengthy, it tells right away that we passed a parameter of the wrong type and/or shape into the method extract.
Note that you need a line like this:
(s/set-fn-validation! true) ; enforce fn schemas
I create a special file test/tst/clj/_bootstrap.clj so it is always in the same place.
For more information on Plumatic Schema please see:
https://github.com/plumatic/schema
https://youtu.be/o_jtwIs2Ot8
https://github.com/plumatic/schema/wiki/Basics-Examples
https://github.com/plumatic/schema/wiki/Defining-New-Schema-Types-1.0
I have a question regarding two functions, one taking a complete map and the other specific keywords like so:
(def mapVal
{:test "n"
:anotherKey "n"})
(defn functionTest
[& {:keys [test anotherKey] :or {:test "d" :anotherKey "d"}}]
(println :test :anotherKey))
(defn mapFunc
[map]
(functionTest (get-in map [:test :anotherKey])))
The goal would be that all the keys in the parameter map will be passed correctly to the functionTest. Is there anyway this could work? I tried a few things but i just cant get all the keywords and values passed to the functionTest. What i dont want is just the values of the map, it should be passed to the other function with keyword and value.
You're pretty close. A few things should clear it up.
First, when you declare parameters with [& varname] that means varname will be a list containing all the extra parameters. So you don't need to use that '&' here to destructure the input. Instead, you just name which keys you want to have become variables.
Try this:
(defn functionTest
[{:keys [test anotherKey]}]
(println test anotherKey))
And the other problem is using get-in. With get-in you're defining a "path" through nested data structures with that vector. For example, given:
{:first {:a 1 :b 2} :second {:c 3 :d 4}}
You could use get-in to get the value at :second :c with this:
(get-in {:first {:a 1 :b 2} :second {:c 3 :d 4}} [:second :c])
In your case, you don't need to use get-in at all. You just need to pass the entire map. The destructuring you defined in functionTest will handle the rest. Here is what I did that worked:
(defn mapFunc
[map]
(functionTest map))
I would also suggest you not name the that variable 'map' since it conflicts with the map function.
get-in is for accessing nested associative data structures.
(def m {:a {:x 1}})
(get-in m [:a :x]) ;;=> 1
After you destructure a map those values are in scope and are accessed via symbols. Your example should look like this:
(def mapVal
{:test "n"
:anotherKey "n"})
(defn functionTest
[& {:keys [test anotherKey]
:or {:test "d" :anotherKey "d"}}]
(println test anotherKey))
(defn mapFunc
[m]
(apply functionTest (apply concat (select-keys m [:test :anotherKey]))))
(mapFunc mapVal) ;;=> prints: n n
You have to go through this because functionTest is accepting bare key value pairs as optional parameters (the ones to the right of the &), as in:
(functionTest :test "n"
:anotherKey "n" )
;;=> Also prints: n n
select-keys returns a map with only the specified keys:
(select-keys mapVal [:test])
;; => {:test "n"}
applying concat to a map returns a flat seq of keys and values:
(apply concat (select-keys mapVal [:test :anotherKey]))
;; => (:test "n" :anotherKey "n")
apply applies a function to a seq, as though the seq were its argument list:
(+ [1 2 3]) ;;=> Error.
(apply + [1 2 3]) ;; => 6
As a side note, conventionally in Clojure code, snake-case is preferred to camelCase for most names.
I'm generating json as literally as I can in clojure. My problem is that certain branches of the json are only present if given parameters are given. Here is a sample of such a condition
(defn message-for
[name uuid & [generated-uuids]]
{:message {:id (generate-uuid)
:details {:name name}
:metadata {:batch (merge {:id uuid}
(when generated-uuids (let [batches (map #(array-map :id %) generated-uuids)]
{:generatedBatches batches})))}}})
Unfortunately the when/let part is quite ugly. This same could be achieved using when-let as following but it doesn't work because my map returns [] instead of a nil.
(defn message-for
[name uuid & [generated-uuids]]
{:message {:id (generate-uuid)
:details {:name name}
:metadata {:batch (merge {:id uuid}
(when-let [batches (map #(array-map :id %) generated-uuids)]
{:generatedBatches batches}))}}})
Any ideas if I could somehow make when-let consider an empty list/array/seq as false so I could clean up my code a bit?
not-empty returns its argument if it is not empty.
When using when-let with a collection, always use not-empty
to retain the collection type
make refactoring easier
expressivenes
(when-let [batches (not-empty (map ...))]
...)
In your case I'd however prefer something like this:
...
:metadata {:batch (cond-> {:id uuid}
(seq generated-uuids)
(assoc :generatedBatches (map ...)))}
...
Notice that all three of the advantages listed above where met, without a nested let.
Also notice a new advantage
easier to extend with more conditions lateron
seq returns nil on an empty input sequence so you could do:
(when-let [batches (seq (map #(array-map :id %) generated-uuids))]
{:generatedBatches batches}))}}})
I'm trying to handle following DSL:
(simple-query
(is :category "car/audi/80")
(is :price 15000))
that went quite smooth, so I added one more thing - options passed to the query:
(simple-query {:page 1 :limit 100}
(is :category "car/audi/80")
(is :price 15000))
and now I have a problem how to handle this case in most civilized way. as you can see simple-query may get hash-map as a first element (followed by long list of criteria) or may have no hash-mapped options at all. moreover, I would like to have defaults as a default set of options in case when some (or all) of them are not provided explicite in query.
this is what I figured out:
(def ^{:dynamic true} *defaults* {:page 1
:limit 50})
(defn simple-query [& body]
(let [opts (first body)
[params criteria] (if (map? opts)
[(merge *defaults* opts) (rest body)]
[*defaults* body])]
(execute-query params criteria)))
I feel it's kind of messy. any idea how to simplify this construction?
To solve this problem in my own code, I have a handy function I'd like you to meet... take-when.
user> (defn take-when [pred [x & more :as fail]]
(if (pred x) [x more] [nil fail]))
#'user/take-when
user> (take-when map? [{:foo :bar} 1 2 3])
[{:foo :bar} (1 2 3)]
user> (take-when map? [1 2 3])
[nil [1 2 3]]
So we can use this to implement a parser for your optional map first argument...
user> (defn maybe-first-map [& args]
(let [defaults {:foo :bar}
[maybe-map args] (take-when map? args)
options (merge defaults maybe-map)]
... ;; do work
))
So as far as I'm concerned, your proposed solution is more or less spot on, I would just clean it up by factoring out parser for grabbing the options map (here into my take-when helper) and by factoring out the merging of defaults into its own binding statement.
As a general matter, using a dynamic var for storing configurations is an antipattern due to potential missbehavior when evaluated lazily.
What about something like this?
(defn simple-query
[& body]
(if (map? (first body))
(execute-query (merge *defaults* (first body)) (rest body))
(execute-query *defaults* body)))