I find myself writing a lot of specs like this:
(s/def ::name string?)
(s/def ::logUri string?)
(s/def ::subnet (s/and string? #(> (count %) 5)))
(s/def ::instanceType string?)
...
(s/def ::key (s/and string? #(> (count %) 5)))
(s/def ::instanceCount string?)
(s/def ::bidPct string?)
I.e. Lots of s/and and s/def. This seems like a waste. So I decided to write a macro that did this for me. Something like:
(defmacro and-spec [validations]
(doseq [[keyname & funcs] validations]
`(s/def ~keyname (s/and ~#funcs))))
So I would be able to do something like:
(and-spec [[::name1 [string?]]
[::name2 [string? #(> (count %) 5)]]])
And this would just do all my s/def stuff for me. Unfortunately, the above macro doesn't work, but I'm not sure why.
(s/valid? ::name1 "asdf")
Execution error at emr-cli.utils/eval6038 (form-init17784784591561795514.clj:1).
Unable to resolve spec: :emr-cli.utils/name1
Smaller versions of this work:
(defmacro small-and-spec-works [key-name validations]
`(s/def ~key-name (s/and ~#validations)))
=> #'emr-cli.utils/tsmall
(and-spec-small-works ::mykey [string?])
=> :emr-cli.utils/mykey
(s/valid? ::mykey "asdf")
=> true
But the second I introduce a let binding things start getting weird:
(defmacro small-and-spec [validation]
(let [[key-name & valids] validation]
`(s/def ~key-name (s/and ~#valids))))
=> #'emr-cli.utils/small-and-spec
(small-and-spec [::mykey2 [string?]])
=> :emr-cli.utils/mykey2
(s/valid? ::mykey2 "asdf")
Execution error (IllegalArgumentException) at emr-cli.utils/eval6012 (form-init17784784591561795514.clj:1).
Key must be integer
How can I make the doseq macro work?
What is going wrong with the small-and-spec that creates the Key must be integer error?
(defmacro and-spec [defs]
`(do
~#(map (fn [[name rest]]
`(s/def ~name (s/and ~#rest))) defs)))
doseq is for side effects. It always returns nil.
Related
I am trying to use Clojure Spec to define a data structure containing a java.time.LocalDate element:
(s/def :ex/first-name string?)
(s/def :ex/last-name string?)
(s/def :ex/birth-date (s/valid? inst? (java.time.LocalDate/now)))
(s/def :ex/person
(s/keys :req [:ex/first-name
:ex/last-name
:ex/birth-date]))
(def p1 #:ex{:first-name "Jenny"
:last-name "Barnes"
:birth-date (java.time.LocalDate/parse "1910-03-15")})
(println p1)
produces the following output
#:ex{:first-name Jenny, :last-name Barnes, :birth-date #object[java.time.LocalDate 0x4ed4f9db 1910-03-15]}
However, when I test to see if p1 conforms to the :ex/person spec, it fails:
(s/valid? :ex/person p1)
ClassCastException java.lang.Boolean cannot be cast to clojure.lang.IFn clojure.spec.alpha/spec-impl/reify--1987 (alpha.clj:875)
Looking closer at the Clojure examples for inst?, I see:
(inst? (java.time.Instant/now))
;;=> true
(inst? (java.time.LocalDateTime/now))
;;=> false
However, I don't see an obvious reason as to why that returns false. This seems to be the root of my issue, but I have not found a solution and would like some help.
You're probably looking for instance?- and your example fails, because in:
(s/def :ex/birth-date (s/valid? inst? (java.time.LocalDate/now)))
this part (s/valid? inst? (java.time.LocalDate/now)) should be a function (predicate), not boolean. The full code:
(s/def :ex/first-name string?)
(s/def :ex/last-name string?)
(s/def :ex/birth-date #(instance? java.time.LocalDate %))
(s/def :ex/person
(s/keys :req [:ex/first-name
:ex/last-name
:ex/birth-date]))
(def p1 #:ex{:first-name "Jenny"
:last-name "Barnes"
:birth-date (java.time.LocalDate/parse "1910-03-15")})
(s/valid? :ex/person p1)
=> true
inst? won't work here, because Inst is a protocol, used to extend java.util.Date and java.time.Instant:
(defprotocol Inst
(inst-ms* [inst]))
(extend-protocol Inst
java.util.Date
(inst-ms* [inst] (.getTime ^java.util.Date inst)))
(defn inst?
"Return true if x satisfies Inst"
{:added "1.9"}
[x]
(satisfies? Inst x))
(extend-protocol clojure.core/Inst
java.time.Instant
(inst-ms* [inst] (.toEpochMilli ^java.time.Instant inst)))
And you can use satisfies? to check whether some object satisfies given protocol:
(satisfies? Inst (java.time.LocalDate/parse "1910-03-15"))
=> false
I'd like to use clojure spec to build up a set of type constraints that can be aliased or further constrained by other specs.
For example, I might have many fields that all need to be valid sanitized markdown.
The following example works for validation (s/valid?) but not for generation (gen/generate)
(s/def ::sanitized-markdown string?)
(s/def ::instruction-list #(s/valid? ::sanitized-markdown %)) ;; works
(gen/generate (s/gen ::instruction-list)) ;; fails
However (gen/generate (s/gen ::sanitized-markdown)) does work.
Is there a way to extend ::instruction-list from ::sanitized-markdown so that it preserves all behavior?
You can alias another spec by providing it directly to s/def:
(s/def ::instruction-list ::sanitized-markdown)
You can use s/merge when merging map specs and s/and in other cases.
(s/def ::sanitized-markdown string?)
(s/def ::instruction-list (s/and ::sanitized-markdown #(> (count %) 10)))
(s/valid? ::instruction-list "abcd")
;; false
(s/valid? ::instruction-list "abcdefghijkl")
;; true
(gen/generate (s/gen ::instruction-list))
;; "178wzJW3W3zx2G0GJ1931eEeO"
An example with maps
(s/def ::a string?)
(s/def ::b string?)
(s/def ::c string?)
(s/def ::d string?)
(s/def ::first-map (s/keys :opt [::a ::b]))
(s/def ::second-map (s/keys :opt [::c ::d]))
(s/def ::third-map (s/merge ::first-map ::second-map))
(s/valid? ::third-map {:a "1" :d "2"})
;; true
(gen/generate (s/gen ::third-map))
;; {::b "gvQ7DI1kQ9DxG7C4poeWhk553", ::d "9KIp77974TEqs9HCq", ::c "qeSZA8NcYr7UVpJDsA17K"}
I want to write a clojure spec for a hash-map wherein the value of one of the
keys is constrained to be equal to the sum of the values of two other keys. I
know one way to write a test generator for such a spec by hand:
(ns my-domain)
(require '[clojure.test :refer :all ]
'[clojure.spec.alpha :as s ]
'[clojure.spec.gen.alpha :as gen ]
'[clojure.pprint :refer (pprint) ])
(s/def ::station-id string?)
(s/def ::sim-time (s/double-in :infinite? true, :NaN? false))
(s/def ::reserved-counts (s/and int? #(not (neg? %))))
(s/def ::free-counts (s/and int? #(not (neg? %))))
(def counts-preimage (s/gen (s/keys :req [::station-id
::sim-time
::reserved-counts
::free-counts])))
(pprint (gen/generate
(gen/bind
counts-preimage
#(gen/return
(into % {::total-counts
(+ (::reserved-counts %)
(::free-counts %))})))))
#:my-domain{:station-id "sHN8Ce0tKWSdXmRd4e46fB",
:sim-time -3.4619293212890625,
:reserved-counts 58,
:free-counts 194,
:total-counts 252}
But I haven't figured out how to write a spec for it, let alone a spec that
produces a similar generator. The gist of the problem is that I lack, in the space of specs, a way to
get hold of the "preimage" in the spec, that is, I lack an analogue to bind
from the space of generators. Here is a failed attempt:
(s/def ::counts-partial-hash-map
(s/keys :req [::station-id
::sim-time
::reserved-counts
::free-counts]))
(s/def ::counts-attempted-hash-map
(s/and ::counts-partial-hash-map
#(into % {::total-counts (+ (::reserved-counts %)
(::free-counts %))})))
(pprint (gen/generate (s/gen ::counts-attempted-hash-map)))
#:my-domain{:station-id "ls5qBUoF",
:sim-time ##Inf,
:reserved-counts 56797960,
:free-counts 17}
The generated sample conforms to the spec because #(into % {...}) is truthy,
but the result doesn't contain the new attribute with the key ::total-counts.
I'd be grateful for any guidance.
EDIT: Today I Learned about s/with-gen, which will allow me to attach
my (working) test generator to my "preimage" or "partial" spec. Perhaps
that's the best way forward?
You could use the nat-int? predicate (for which there's a built-in spec, thanks #glts) for the count keys, and add a ::total-counts spec too:
(s/def ::reserved-counts nat-int?)
(s/def ::free-counts nat-int?)
(s/def ::total-counts nat-int?)
(s/def ::counts-partial-hash-map
(s/keys :req [::station-id
::sim-time
::reserved-counts
::free-counts]))
spec for a hash-map wherein the value of one of the keys is constrained to be equal to the sum of the values of two other keys
To add this assertion you can s/and a predicate function with the keys spec (or in this example the merge spec that merges the partial map spec with a ::total-count keys spec):
(s/def ::counts-attempted-hash-map
(s/with-gen
;; keys spec + sum-check predicate
(s/and
(s/merge ::counts-partial-hash-map (s/keys :req [::total-counts]))
#(= (::total-counts %) (+ (::reserved-counts %) (::free-counts %))))
;; custom generator
#(gen/fmap
(fn [m]
(assoc m ::total-counts (+ (::reserved-counts m) (::free-counts m))))
(s/gen ::counts-partial-hash-map))))
This also uses with-gen to associate a custom generator with the spec that sets ::total-count to the sum of the other count keys.
(gen/sample (s/gen ::counts-attempted-hash-map) 1)
=> (#:user{:station-id "", :sim-time 0.5, :reserved-counts 1, :free-counts 1, :total-counts 2})
The generated sample conforms to the spec because #(into % {...}) is truthy, but the result doesn't contain the new attribute with the key ::total-counts.
I'd recommend against using specs to calculate/add ::total-counts to the map. Specs generally shouldn't be used for data transformation.
I've been getting on quite well with clojure.spec for the most part. However, I came to a problem that I couldn't figure out when dealing with unform. Here's a loose spec for Hiccup to get us moving:
(require '[clojure.spec :as s])
(s/def ::hiccup
(s/and
vector?
(s/cat
:name keyword?
:attributes (s/? map?)
:contents (s/* ::contents))))
(s/def ::contents
(s/or
:element-seq (s/* ::hiccup)
:element ::hiccup
:text string?))
Now before we get carried away, let's see if it works with a small passing case.
(def example [:div])
(->> example
(s/conform ::hiccup))
;;=> {:name :h1}
Works like a charm. But can we then undo our conformance?
(->> example
(s/conform ::hiccup)
(s/unform ::hiccup))
;;=> (:div)
Hmm, that should be a vector. Am I missing something? Let's see what spec has to say about this.
(->> example
(s/conform ::hiccup)
(s/unform ::hiccup)
(s/explain ::hiccup))
;; val: (:div) fails spec: :user/hiccup predicate: vector?
;;=> nil
Indeed, it fails. So the question: How do I get this to work correctly?
Late response here. I had not realised how old this question was but since I had already written a reply I might as well submit it.
Taking the spec that you provide for ::hiccup:
(s/def ::hiccup
(s/and
vector?
(s/cat
:name keyword?
:attributes (s/? map?)
:contents (s/* ::contents))))
The vector? spec within and will test the input data against that predicate. Unluckily as you have experienced, it doesn't unform to a vector.
Something that you can do to remedy this is to add an intermediate spec that will act as the identity when conforming and will act as desired when unforming. E.g.
(s/def ::hiccup
(s/and vector?
(s/conformer vec vec)
(s/cat
:name keyword?
:attributes (s/? map?)
:contents (s/* ::contents))))
(s/unform ::hiccup (s/conform ::hiccup [:div]))
;;=> [:div]
(s/conformer vec vec) Will be a no-op for everything that satisfies vector? and using vec as the unforming function in the conformer will make sure that the result to unforming the whole and spec stays a vector.
I am new to spec myself and this was how I got it working as you intend. Bear in mind it might not be the way in which it was intended to be used by spec's designers.
For what is worth, I made a spec that checks for a vector and unforms to a vector:
(defn vector-spec
"Create a spec that it is a vector and other conditions and unforms to a vector.
Ex (vector-spec (s/spec ::binding-form))
(vector-spec (s/* integer?))"
[form]
(let [s (s/spec (s/and vector? form))]
(reify
s/Specize
(specize* [_] s)
(specize* [_ _] s)
s/Spec
(conform* [_ x] (s/conform* s x))
(unform* [_ x] (vec (s/unform* s x))) ;; <-- important
(explain* [_ path via in x] (s/explain s path via in x))
(gen* [_ overrides path rmap] (s/gen* s overrides path rmap))
(with-gen* [_ gfn] (s/with-gen s gfn))
(describe* [_] (s/describe* s)))))
In your example, you would use it like this:
(s/def ::hiccup
(vector-spec
(s/cat
:name keyword?
:attributes (s/? map?)
:contents (s/* ::contents))))
After writing this answer, I was inspired to try to specify Clojure's destructuring language using spec:
(require '[clojure.spec :as s])
(s/def ::binding (s/or :sym ::sym :assoc ::assoc :seq ::seq))
(s/def ::sym (s/and simple-symbol? (complement #{'&})))
The sequential destructuring part is easy to spec with a regex (so I'm ignoring it here), but I got stuck at associative destructuring. The most basic case is a map from binding forms to key expressions:
(s/def ::mappings (s/map-of ::binding ::s/any :conform-keys true))
But Clojure provides several special keys as well:
(s/def ::as ::sym)
(s/def ::or ::mappings)
(s/def ::ident-vec (s/coll-of ident? :kind vector?))
(s/def ::keys ::ident-vec)
(s/def ::strs ::ident-vec)
(s/def ::syms ::ident-vec)
(s/def ::opts (s/keys :opt-un [::as ::or ::keys ::strs ::syms]))
How can I create an ::assoc spec for maps that could be created by merging together a map that conforms to ::mappings and a map that conforms to ::opts? I know that there's merge:
(s/def ::assoc (s/merge ::opts ::mappings))
But this doesn't work, because merge is basically an analogue of and. I'm looking for something that's analogous to or, but for maps.
You can spec hybrid maps using an s/merge of s/keys and s/every of the map as tuples. Here's a simpler example:
(s/def ::a keyword?)
(s/def ::b string?)
(s/def ::m
(s/merge (s/keys :opt-un [::a ::b])
(s/every (s/or :int (s/tuple int? int?)
:option (s/tuple keyword? any?))
:into {})))
(s/valid? ::m {1 2, 3 4, :a :foo, :b "abc"}) ;; true
This simpler formulation has several benefits over a conformer approach. Most importantly, it states the truth. Additionally, it should generate, conform, and unform without further effort.
You can use s/conformer as an intermediate step in s/and to transform your map to the form that’s easy to validate:
(s/def ::assoc
(s/and
map?
(s/conformer #(array-map
::mappings (dissoc % :as :or :keys :strs :syms)
::opts (select-keys % [:as :or :keys :strs :syms])))
(s/keys :opt [::mappings ::opts])))
That will get you from e.g.
{ key :key
:as name }
to
{ ::mappings { key :key }
::opts { :as name } }