clojure 1.9.0
A simple test of sort-by on the character array works below,
user=> (sort-by identity [[\B] [\a]])
([\B] [\a])
but why did another test fail to sort-by case-insensitively?
user=> (sort-by (partial map #(Character/toLowerCase %)) [[\B] [\a]])
java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.Comparable
Solution
Using mapv instead of map makes it.
user=> (instance? clojure.lang.LazySeq (map identity []))
true
user=> (instance? clojure.lang.PersistentVector (mapv identity (map identity [])))
true
You don't need map:
(ns tst.demo.core
(:require
[clojure.string :as str] ))
(sort-by #(.toLowerCase (str/join %)) [[\a \b] [\B] [\a]])
;=> ([\a] [\a \b] [\B])
but why did another test fail to sort-by case-insensitively?
map returns a lazy sequence, which doesn't implement Comparable. mapv works because vectors support Comparable and that's what sort-by is using to sort.
(supers (type []))
=> #{... java.lang.Comparable ...}
(supers clojure.lang.LazySeq)
=> #{clojure.lang.IObj clojure.lang.ISeq clojure.lang.Seqable clojure.lang.IMeta java.lang.Iterable java.util.List clojure.lang.IHashEq java.lang.Object clojure.lang.Obj java.util.Collection clojure.lang.IPending clojure.lang.Sequential clojure.lang.IPersistentCollection java.io.Serializable}
Related
Let's say I have several vectors
(def coll-a [{:name "foo"} ...])
(def coll-b [{:name "foo"} ...])
(def coll-c [{:name "foo"} ...])
and that I would like to see if the names of the first elements are equal.
I could
(= (:name (first coll-a)) (:name (first coll-b)) (:name (first coll-c)))
but this quickly gets tiring and overly verbose as more functions are composed. (Maybe I want to compare the last letter of the first element's name?)
To directly express the essence of the computation it seems intuitive to
(apply = (map (comp :name first) [coll-a coll-b coll-c]))
but it leaves me wondering if there's a higher level abstraction for this sort of thing.
I often find myself comparing / otherwise operating on things which are to be computed via a single composition applied to multiple elements, but the map syntax looks a little off to me.
If I were to home brew some sort of operator, I would want syntax like
(-op- (= :name first) coll-a coll-b coll-c)
because the majority of the computation is expressed in (= :name first).
I'd like an abstraction to apply to both the operator & the functions applied to each argument. That is, it should be just as easy to sum as compare.
(def coll-a [{:name "foo" :age 43}])
(def coll-b [{:name "foo" :age 35}])
(def coll-c [{:name "foo" :age 28}])
(-op- (+ :age first) coll-a coll-b coll-c)
; => 106
(-op- (= :name first) coll-a coll-b coll-c)
; => true
Something like
(defmacro -op-
[[op & to-comp] & args]
(let [args' (map (fn [a] `((comp ~#to-comp) ~a)) args)]
`(~op ~#args')))
Is there an idiomatic way to do this in clojure, some standard library function I could be using?
Is there a name for this type of expression?
For your addition example, I often use transduce:
(transduce
(map (comp :age first))
+
[coll-a coll-b coll-c])
Your equality use case is trickier, but you could create a custom reducing function to maintain a similar pattern. Here's one such function:
(defn all? [f]
(let [prev (volatile! ::no-value)]
(fn
([] true)
([result] result)
([result item]
(if (or (= ::no-value #prev)
(f #prev item))
(do
(vreset! prev item)
true)
(reduced false))))))
Then use it as
(transduce
(map (comp :name first))
(all? =)
[coll-a coll-b coll-c])
The semantics are fairly similar to your -op- macro, while being both more idiomatic Clojure and more extensible. Other Clojure developers will immediately understand your usage of transduce. They may have to investigate the custom reducing function, but such functions are common enough in Clojure that readers can see how it fits an existing pattern. Also, it should be fairly transparent how to create new reducing functions for use cases where a simple map-and-apply wouldn't work. The transducing function can also be composed with other transformations such as filter and mapcat, for cases when you have a more complex initial data structure.
You may be looking for the every? function, but I would enhance clarity by breaking it down and naming the sub-elements:
(let [colls [coll-a coll-b coll-c]
first-name (fn [coll] (:name (first coll)))
names (map first-name colls)
tgt-name (first-name coll-a)
all-names-equal (every? #(= tgt-name %) names)]
all-names-equal => true
I would avoid the DSL, as there is no need and it makes it much harder for others to read (since they don't know the DSL). Keep it simple:
(let [colls [coll-a coll-b coll-c]
vals (map #(:age (first %)) colls)
result (apply + vals)]
result => 106
I don't think you need a macro, you just need to parameterize your op function and compare functions. To me, you are pretty close with your (apply = (map (comp :name first) [coll-a coll-b coll-c])) version.
Here is one way you could make it more generic:
(defn compare-in [op to-compare & args]
(apply op (map #(get-in % to-compare) args)))
(compare-in + [0 :age] coll-a coll-b coll-c)
(compare-in = [0 :name] coll-a coll-b coll-c)
;; compares last element of "foo"
(compare-in = [0 :name 2] coll-a coll-b coll-c)
I actually did not know you can use get on strings, but in the third case you can see we compare the last element of each foo.
This approach doesn't allow the to-compare arguments to be arbitrary functions, but it seems like your use case mainly deals with digging out what elements you want to compare, and then applying an arbitrary function to those values.
I'm not sure this approach is better than the transducer version supplied above (certainly not as efficient), but I think it provides a simpler alternative when that efficiency is not needed.
I would split this process into three stages:
transform items in collections into the data in collections you want to operate
on - (map :name coll);
Operate on transformed items in collections, returning collection of results - (map = transf-coll-a transf-coll-b transf-coll-c)
Finally, selecting which result in resulting collection to return - (first calculated-coll)
When playing with collections, I try to put more than one item into collection:
(def coll-a [{:name "foo" :age 43} {:name "bar" :age 45}])
(def coll-b [{:name "foo" :age 35} {:name "bar" :age 37}])
(def coll-c [{:name "foo" :age 28} {:name "bra" :age 30}])
For example, matching items by second char in :name and returning result for items in second place:
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp #(nth % 1) :name)
op =
fetch second]
(fetch (apply map op (map #(map transf-fn %) colls))))
;; => false
In transducers world you can use sequence function which also works on multiple collections:
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp (map :name) (map #(nth % 1)))
op =
fetch second]
(fetch (apply sequence (map op) (map #(sequence transf-fn %) colls))))
Calculate sum of ages (for all items at the same level):
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp (map :age))
op +
fetch identity]
(fetch (apply sequence (map op) (map #(sequence transf-fn %) colls))))
;; => (106 112)
Given I have the following form:
(def data-points [[1483249740 "ONE"]
[1483249680 "TWO"]
[1483249620 "THREE"]
[1483249560 "FOUR"]])
How can I transform this data into this?
{:data [1483249740 1483249680 1483249620 1483249560]
:value ["ONE" "TWO" "THREE" "FOUR"]}
I would also like to know how to approach similar problems.
What is your way to break this down and what functions do I need to know to transform any data.
I'm new to clojure and haven't found a satisfying solution for this.
Thank you
i would use this:
(zipmap [:data :value] (apply map vector data-points))
;;=> {:data [1483249740 1483249680 1483249620 1483249560],
;; :value ["ONE" "TWO" "THREE" "FOUR"]}
it uses a single pass over the data collections, but more concise than the reduction, yet shouldn't differ in terms of performance
the snippet (apply map vector data) is quite an idiomatic way to transpose matrix in clojure (in your case it is what you need, since it turns columns into rows)
user> (apply map vector [[:a 1] [:b 2] [:c 3]])
;;=> ([:a :b :c] [1 2 3])
I'd probably write this as a reduction. This approach only requires a single pass over 'data-points' which may be preferable.
(reduce
(fn [m [data value]]
(-> m
(update :data conj data)
(update :values conj value)))
{:data [] :values []}
data-points)
Another representation which can be both efficient and easier to work with:
(def data-points-map
(into {} data-points))
Then you can do
(get data-points-map 1483249740)
to get "ONE". Otherwise you would need to
(aget (:value m) (.indexOf (:data m) 1483249740))
to achieve the same result.
Finally you can
{:data (keys data-points-map)
:value (values data-points-map)}
to get the "weird" representation in the original question.
Here's an obtusely functional way to do it
(->> data-points
(mapv (partial mapv vector))
(mapv (partial zipmap [:data :value]))
(reduce (partial merge-with into)))
As a function:
(def format-data-points
(comp (partial reduce (partial merge-with into))
(partial mapv (partial zipmap [:data :value]))
(partial mapv (partial mapv vector))))
(format-data-points data-points)
(I wouldn't recommend doing either of these actually, just presenting this for fun)
Is there anyway to include clojure.spec'd functions in a generalized test suite? I know we can register specs and directly spec functions.
(ns foo
(:require [clojure.spec :as s]
[clojure.spec.test :as stest]))
(defn average [list-sum list-count]
(/ list-sum list-count))
(s/fdef average
:args (s/and (s/cat :list-sum float? :list-count integer?)
#(not (zero? (:list-count %))))
:ret number?)
And later, if I want to run generative tests against that spec'd function, I can use stest/check.
=> (stest/check `average)
({:spec #object[clojure.spec$fspec_impl$reify__14282 0x68e9f37c "clojure.spec$fspec_impl$reify__14282#68e9f37c"], :clojure.spec.test.check/ret {:result true, :num-tests 1000, :seed 1479587517232}, :sym edgar.core.analysis.lagging/average})
But i) is there anyway to include these test runs in my general test suite? I'm thinking of the kind of clojure.test integration that test.check has. The closest thing that I can see ii) is the stest/instrument (see here) function. But that seems to just let us turn on checking at the repl. Not quite what I want. Also, iii) are function specs registered?
(defspec foo-test
100
;; NOT this
#_(prop/for-all [v ...]
(= v ...))
;; but THIS
(stest/some-unknown-spec-fn foo))
Ok, solved this one. Turns out there's no solution out of the box. But some people on the clojure-spec slack channel have put together a defspec-test solution for clojure.spec.test and clojure.test.
So given the code in the question. You can A) define the defspec-test macro that takes your test name and a list of spec'd functions. You can then B) use it in your test suite.
Thanks Clojure community!! And hopefully such a utility function makes it into the core library.
A)
(ns foo.test
(:require [clojure.test :as t]
[clojure.string :as str]))
(defmacro defspec-test
([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
([name sym-or-syms opts]
(when t/*load-tests*
`(def ~(vary-meta name assoc
:test `(fn []
(let [check-results# (clojure.spec.test/check ~sym-or-syms ~opts)
checks-passed?# (every? nil? (map :failure check-results#))]
(if checks-passed?#
(t/do-report {:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym check-results#)))})
(doseq [failed-check# (filter :failure check-results#)
:let [r# (clojure.spec.test/abbrev-result failed-check#)
failure# (:failure r#)]]
(t/do-report
{:type :fail
:message (with-out-str (clojure.spec/explain-out failure#))
:expected (->> r# :spec rest (apply hash-map) :ret)
:actual (if (instance? Throwable failure#)
failure#
(:clojure.spec.test/val failure#))})))
checks-passed?#)))
(fn [] (t/test-var (var ~name)))))))
B)
(ns foo-test
(:require [foo.test :refer [defspec-test]]
[foo]))
(defspec-test test-average [foo/average])
The above example can fail in the case where :failure is false due to how stest/abbrev-result tests for failure. See CLJ-2246 for more details. You can work around this by defining your own version of abbrev-result. Also, the formatting of failure data has changed.
(require
'[clojure.string :as str]
'[clojure.test :as test]
'[clojure.spec.alpha :as s]
'[clojure.spec.test.alpha :as stest])
;; extracted from clojure.spec.test.alpha
(defn failure-type [x] (::s/failure (ex-data x)))
(defn unwrap-failure [x] (if (failure-type x) (ex-data x) x))
(defn failure? [{:keys [:failure]}] (not (or (true? failure) (nil? failure))))
;; modified from clojure.spec.test.alpha
(defn abbrev-result [x]
(let [failure (:failure x)]
(if (failure? x)
(-> (dissoc x ::stc/ret)
(update :spec s/describe)
(update :failure unwrap-failure))
(dissoc x :spec ::stc/ret))))
(defn throwable? [x]
(instance? Throwable x))
(defn failure-report [failure]
(let [expected (->> (abbrev-result failure) :spec rest (apply hash-map) :ret)]
(if (throwable? failure)
{:type :error
:message "Exception thrown in check"
:expected expected
:actual failure}
(let [data (ex-data (get-in failure
[::stc/ret
:result-data
:clojure.test.check.properties/error]))]
{:type :fail
:message (with-out-str (s/explain-out data))
:expected expected
:actual (::s/value data)}))))
(defn check?
[msg [_ body :as form]]
`(let [results# ~body
failures# (filter failure? results#)]
(if (empty? failures#)
[{:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym results#)))}]
(map failure-report failures#))))
(defmethod test/assert-expr 'check?
[msg form]
`(dorun (map test/do-report ~(check? msg form))))
Here's a slightly modified version of grzm's excellent answer that works with [org.clojure/test.check "0.10.0-alpha4"]. It uses the new :pass? key that comes from this PR: https://github.com/clojure/test.check/commit/09927b64a60c8bfbffe2e4a88d76ee4046eef1bc#diff-5eb045ad9cf20dd057f8344a877abd89R1184.
(:require [clojure.test :as t]
[clojure.string :as str]
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest])
(alias 'stc 'clojure.spec.test.check)
;; extracted from clojure.spec.test.alpha
(defn failure-type [x] (::s/failure (ex-data x)))
(defn unwrap-failure [x] (if (failure-type x) (ex-data x) x))
;; modified from clojure.spec.test.alpha
(defn abbrev-result [x]
(if (-> x :stc/ret :pass?)
(dissoc x :spec ::stc/ret)
(-> (dissoc x ::stc/ret)
(update :spec s/describe)
(update :failure unwrap-failure))))
(defn throwable? [x]
(instance? Throwable x))
(defn failure-report [failure]
(let [abbrev (abbrev-result failure)
expected (->> abbrev :spec rest (apply hash-map) :ret)
reason (:failure abbrev)]
(if (throwable? reason)
{:type :error
:message "Exception thrown in check"
:expected expected
:actual reason}
(let [data (ex-data (get-in failure
[::stc/ret
:shrunk
:result-data
:clojure.test.check.properties/error]))]
{:type :fail
:message (with-out-str (s/explain-out data))
:expected expected
:actual (::s/value data)}))))
(defn check?
[msg [_ body :as form]]
`(let [results# ~body
failures# (remove (comp :pass? ::stc/ret) results#)]
(if (empty? failures#)
[{:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym results#)))}]
(map failure-report failures#))))
(defmethod t/assert-expr 'check?
[msg form]
`(dorun (map t/do-report ~(check? msg form))))
Usage:
(deftest whatever-test
(is (check? (stest/check `whatever
;; optional
{:clojure.spec.test.check/opts {:num-tests 10000}})))
I'm trying to handle following DSL:
(simple-query
(is :category "car/audi/80")
(is :price 15000))
that went quite smooth, so I added one more thing - options passed to the query:
(simple-query {:page 1 :limit 100}
(is :category "car/audi/80")
(is :price 15000))
and now I have a problem how to handle this case in most civilized way. as you can see simple-query may get hash-map as a first element (followed by long list of criteria) or may have no hash-mapped options at all. moreover, I would like to have defaults as a default set of options in case when some (or all) of them are not provided explicite in query.
this is what I figured out:
(def ^{:dynamic true} *defaults* {:page 1
:limit 50})
(defn simple-query [& body]
(let [opts (first body)
[params criteria] (if (map? opts)
[(merge *defaults* opts) (rest body)]
[*defaults* body])]
(execute-query params criteria)))
I feel it's kind of messy. any idea how to simplify this construction?
To solve this problem in my own code, I have a handy function I'd like you to meet... take-when.
user> (defn take-when [pred [x & more :as fail]]
(if (pred x) [x more] [nil fail]))
#'user/take-when
user> (take-when map? [{:foo :bar} 1 2 3])
[{:foo :bar} (1 2 3)]
user> (take-when map? [1 2 3])
[nil [1 2 3]]
So we can use this to implement a parser for your optional map first argument...
user> (defn maybe-first-map [& args]
(let [defaults {:foo :bar}
[maybe-map args] (take-when map? args)
options (merge defaults maybe-map)]
... ;; do work
))
So as far as I'm concerned, your proposed solution is more or less spot on, I would just clean it up by factoring out parser for grabbing the options map (here into my take-when helper) and by factoring out the merging of defaults into its own binding statement.
As a general matter, using a dynamic var for storing configurations is an antipattern due to potential missbehavior when evaluated lazily.
What about something like this?
(defn simple-query
[& body]
(if (map? (first body))
(execute-query (merge *defaults* (first body)) (rest body))
(execute-query *defaults* body)))
It seems to be a powerful macro, yet I'm failing to apply it to anything but silly examples. Can you show me some real use of it?
Thanks!
Compare:
user> (:baz (:bar (:foo {:foo {:bar {:baz 123}}})))
123
user> (java.io.BufferedReader. (java.io.FileReader. "foo.txt"))
#<BufferedReader java.io.BufferedReader#6e1f8f>
user> (vec (reverse (.split (.replaceAll (.toLowerCase "FOO,BAR,BAZ") "b" "x") ",")))
["xaz" "xar" "foo"]
to:
user> (-> {:foo {:bar {:baz 123}}} :foo :bar :baz)
123
user> (-> "foo.txt" java.io.FileReader. java.io.BufferedReader.)
#<BufferedReader java.io.BufferedReader#7a6c34>
user> (-> "FOO,BAR,BAZ" .toLowerCase (.replaceAll "b" "x") (.split ",") reverse vec)
["xaz" "xar" "foo"]
-> is used when you want a concise way to nest calls. It lets you list the calls in the order they'll be called rather than inside-out, which can be more readable. In the third example, notice how much distance is between some of the arguments and the function they belong to; -> lets you group arguments and function calls a bit more cleanly. Because it's a macro it also works for Java calls, which is nice.
-> isn't that powerful, it just saves you a few parens now and then. Using it or not is a question of style and readability.
Look at the bottom of clojure.zip for extreme examples of how this is helpful.
(-> dz next next next next next next next next next remove up (append-child 'e) root)
Taken from the wiki I've always found this example impressive:
user=> (import '(java.net URL) '(java.util.zip ZipInputStream))
user=> (-> "http://clojure.googlecode.com/files/clojure_20081217.zip"
URL. .openStream ZipInputStream. .getNextEntry bean :name)
As Brian said - it isn't 'useful' so much as 'different style'. I find for all java interop this form of 'start with X' then do Y and Z ... more readable than do Z to Y of X.
Basically you have 4 options:
; imperative style named steps:
(let [X something
b (Y X)
c (Z b)] c)
; nested calls
(Z (Y X))
; threaded calls
(-> X Y Z)
; functional composition
((comp Z Y) X)
I find -> really shines for java interop but avoid it elsewhere.
(defn search-tickets-for [term]
(-> term search zip-soup first :content
((partial filter #(= :body (:tag %)))) first :content
((partial filter #(= :div (:tag %))))
((partial filter #(= "content" ((comp :id :attrs) %))))
((partial map :content)) first ((partial map :content))
((partial map first)) ((partial filter #(= :ul (:tag %)))) first :content
((partial map :content))
((partial map first))
((partial mapcat :content))
((partial filter #(= :h4 (:tag %))))
((partial mapcat :content))
((partial filter #(= :a (:tag %))))
((partial mapcat :content))))
clojurebot from #clojure uses this to search assembla tickets