I've recently been trying out Clojure Spec and ran into an unexpected error message. I've figured out that if you have a spec/or nested in a spec/and then the spec functions, after the spec/or branch, get passed a conformed value rather than the top level value.
You can see this in the printed value of "v" here (contrived example):
(spec/valid? (spec/and (spec/or :always-true (constantly true))
(fn [v]
(println "v:" v)
(constantly true)))
nil)
v: [:always-true nil]
=> true
I think this may be intentional from the doc string of spec/and:
Takes predicate/spec-forms, e.g.
(s/and even? #(< % 42))
Returns a spec that returns the conformed value. Successive
conformed values propagate through rest of predicates.
But this seems counterintuitive to me as it would hamper reuse of spec predicates, because they'd need to be written to accept "[<or branch> <actual value>]".
Things get even worse if you have multiple spec/or branches:
(spec/valid? (spec/and (spec/or :always-true (constantly true))
(spec/or :also-always-true (constantly true))
(fn [v]
(println "v:" v)
(constantly true)))
nil)
v: [:also-always-true [:always-true nil]]
=> true
Have I missed something fundamental here?
But this seems counterintuitive to me as it would hamper reuse of spec predicates
IMO the alternatives to these behaviors are less appealing:
Discard s/or's conformed tags by default. We can always discard it if we want, but we wouldn't want clojure.spec to make that decision for us. Spec assumes we want to know which s/or branch matched.
Don't flow conformed values in s/and, at the expense of spec/predicate composability.
Luckily we can discard the s/or tags if necessary. Here are two options:
Wrap the s/or in s/noncomforming. Thanks to glts' comment below reminding me about this (undocumented) function!
(s/valid?
(s/and
(s/nonconforming (s/or :s string? :v vector?))
empty?)
"")
=> true
s/and the s/or specs with a s/conformer that discards the tag.
(s/valid?
(s/and
(s/and (s/or :s string? :v vector?)
;; discard `s/or` tag here
(s/conformer second))
empty?)
[])
=> true
If you often needed this, you could reduce boilerplate with a macro:
(defmacro dkdc-or [& key-pred-forms]
`(s/and (s/or ~#key-pred-forms) (s/conformer second)))
Things get even worse if you have multiple spec/or branches
If you're writing a spec for data that allows for alternatives (e.g. s/or, s/alt), and you're "flowing" valid alternatives into subsequent predicates (s/and), IMO it's more generally useful to have that knowledge in subsequent predicates. I'd be interested in seeing a more realistic use case for this type of spec, because there might be a better way to spec it.
Complementary comment: the behavior of Spec spotted by the question means that you can very well have (and (s/valid? ::spec-1 v) (s/valid? ::spec2) v) but not (s/valid? (s/and ::spec-1 ::spec-2) v), due to conforming from ::spec-1.
You might understandably find this behavior of s/and surprising (not to mention insane). Note however that you will have (s/valid? (s/and (s/nonconforming ::spec-1) ::spec-2) v), as mentioned by the accepted answer.
The corollary is that when using s/and, ask yourself whether you need s/conforming. This piece of advice might be a worthwhile addition to the docstring of s/and.
Related
In Clojure Spec, is there a way to assert that something must be a predicate?
By predicate I mean a function which returns a true/false value.
I'd like to store some predicates in a collection. But is there a way to Spec this collection?
This is possible using fspec, with some caveats:
You must specify an :args spec; not just a :ret spec. This could be an issue if the predicates have a variety of fixed arities, or might throw on invalid inputs e.g. (pos? "1").
Each predicate in the collection will be invoked several times each to ensure they conform to the spec.
If your predicates all take a single input, you could do something like this:
(s/def ::pred (s/fspec :args (s/tuple any?) :ret boolean?))
(s/def ::pred-coll (s/coll-of ::pred))
(s/valid? ::pred-coll [boolean? number?]) => true
To illustrate that the predicates will be invoked:
(defn one? [x]
(prn x)
(= 1 x))
(s/valid? ::pred-coll [one?])
nil
()
:tL
(#uuid "9023252f-d4fe-4ee5-b526-13835cd52187")
...
=> true
I want to create a clojure spec for a map that has rules about the presence of particular keys.
The map must have a :type and can have either :default or :value but not both. I tried:
(s/def ::propertyDef
(s/keys :req [::type (s/or ::default ::value) ] :opt [::description ::required]))
but I got
CompilerException java.lang.AssertionError: Assert failed:
spec/or expects k1 p1 k2 p2..., where ks are keywords
(c/and (even? (count key-pred-forms)) (every? keyword? keys)),
compiling:(C:\Users\MartinRoberts\AppData\Local\Temp\form-init4830956164341520551.clj:1:22)
but the or gave me an error as it is in the wrong format. I have to admit to not really understanding in the documentation for s/or.
First: you are using s/or to specify either a ::default or a ::value in your list of required keys. s/or requires :label spec pairs, and you are giving only the specs themselves, which is the cause of the error.
To solve, simply use or instead:
(s/def ::propertyDef (s/keys :req [::type (or ::default ::value)]
:opt [::description ::required]))
This allows both ::default and ::value to be present in the map, but this is almost always okay. The code which actually uses the map can simply check for the presence of ::value and use that, and if it's not there, then use ::default (or whatever your logic happens to be). This is usually done as such:
(let [myvalue (or (::value mymap) (::default mymap))] ...)
There could be thousands of keys in the map, and it would not affect your ability to extract the keys you need. This is why spec does not provide a built-in way to specify keys that should not be in the map, only ways to specify which keys should be present (namely, :req and :req-un in s/keys). Think of how most http servers work: you can give them nonsensical header keys and values, but they don't refuse to service the request; they just ignore them and return a response.
So, you likely don't need to enforce that only one or the other be present, but if you must, you can define an exclusive or function:
(defn xor
[p q]
(and (or p q)
(not (and p q))))
and then add this as an additional predicate on the spec:
(s/def ::propertyDef (s/and (s/keys :req [::type (or ::default ::value)]
:opt [::description ::required])
#(xor (::default %) (::value %))))
(s/valid? ::propertyDef {::type "type" ::default "default"})
=> true
(s/valid? ::propertyDef {::type "type" ::value "value"})
=> true
(s/valid? ::propertyDef {::type "type" ::default "default" ::value "value"})
=> false
I have a DSL specification which is a sequence as usual (cat). I want to take advantage of spec's parsing (i.e. conforming) to get the AST of an expression that conforms with my DSL. E.g.
user> (s/def ::person (s/cat :person-sym '#{person} :name string? :age number?))
=> :user/person
user> (s/conform ::person '(person "Henry The Sloth" 55))
=> {:person-sym person, :name "Henry The Sloth", :age 55}
Now that it's parsed and I have my AST, I would want to do interesting things with it, so I would want to test it and whatnot. So now I need to write a spec for that AST, and that's basically duplicating everything. Actually it's worse than that because now I have to s/def specs for predicates that I didn't have to before, because as the docs for keys says: "there is no support for inline value specification, by design." / "It is the (enforced) opinion of spec that the specification of values associated with a namespaced keyword, like :my.ns/k, should be registered under that keyword itself..". So duplicating (with omitting the person-sym part):
user> (s/def ::name string?)
=> :user/name
user> (s/def ::age number?)
=> :user/age
user> (s/def ::person-ast (s/keys :req-un [::name ::age]))
:user/person-ast
And now it seems to be compatible:
user> (s/conform ::person-ast (s/conform ::person '(person "Henry The Sloth" 55)))
=> {:person-sym person, :name "Henry The Sloth", :age 55}
In practice, I have more complicated data of course, and I wonder what should I do? AFAIK spec doesn't give me the spec for the AST that it creates (actually personally I would figure that this is something it should do). Any suggestions?
I'd say right now you have two options - one is to do what you're doing and create two sets of specs for the before/after.
The other option is to create a model of your domain in data and generate both specs (I've seen many people are doing something like this).
I have not heard Rich talk about generating the output spec of conformed results so I don't think that is likely in the current roadmap.
I am following the clojure.spec guide. I understand it is possible to declare required and optional attributes when using clojure.spec/keys.
I don't understand what is meant by optional. To me :opt doesn't do anything.
(s/valid? (s/keys :req [:my/a]) {:my/a 1 :my/b 2}) ;=> true
(s/valid? (s/keys :req [:my/a] :opt []) {:my/a 1 :my/b 2}) ;=> true
The guide promises to explain this to me, "We’ll see later where optional attributes can be useful", but I fail to find the explanation. Can I declare forbidden keys? Or somehow declare the set of valid keys to equal the keys in :req and :opt?
This is a very good question, and the clojure.spec API gives the (granted, short and unsatisfying) answer:
The :opt keys serve as documentation and
may be used by the generator.
I do not think you can invalidate a map if it contains an extra (this is what you mean by "forbidden" I think) key using this method. However, you could use this spec to make sure ::bad-key is not present:
(s/def ::m (s/and (s/keys :req [::a]) #(not (contains? % ::bad-key))))
(s/valid? ::m {::a "required!"}) ; => true
(s/valid? ::m {::a "required!" ::b "optional!"}) ; => true
(s/valid? ::m {::a "required!" ::bad-key "no good!"}) ; => false
You could limit the number of keys to exactly the set you want by using this spec:
(s/def ::r (s/and (s/keys :req [::reqd1 ::reqd2]) #(= (count %) 2)))
(s/valid? ::r {::reqd1 "abc" ::reqd2 "xyz"}) ; => true
(s/valid? ::r {::reqd1 "abc" ::reqd2 "xyz" ::extra 123}) ; => false
Still, the best way to handle this IMO, would be to simply ignore that there is a key present that you don't care about.
Hopefully as spec matures, these nice things will be added. Or, maybe they are already there (it is changing rapidly) and I simply don't know about it. This is a very new concept in clojure, so most of us have a lot to learn about it.
UPDATE - December 2016
I just wanted to revisit this 6 months since writing it. It looks like my initial comment about ignoring keys you don't care about is the preferred way to go. In fact, at the clojure/conj conference I attended two weeks ago, Rich's keynote specifically addressed the notion of versioning in all levels of software, from the function level up to the application level. He even specifically mentions this notion of disallowing keys in the talk, which can be found on youtube. He says that it was intentionally designed so that only required keys can be spec'd. Disallowing keys really serves no good purpose, and it should be done with caution.
Regarding the :opt keys, I think the original answer still stands up pretty well--it's documentation, and practically, it allows these optionally specified keys to be generated:
(s/def ::name #{"Bob" "Josh" "Mary" "Susan"})
(s/def ::height-inches (s/int-in 48 90))
(s/def ::person (s/keys :req-un [::name] :opt-un [::height-inches]))
(map first (s/exercise ::person))
; some generated data have :height-inches, some do not
({:name "Susan"}
{:name "Mary", :height-inches 48}
{:name "Bob", :height-inches 49}
{:name "Josh"}
The point about optional keys is that the value will be validated if they appear in the map
I used the last days to dig deeper into clojure.spec in Clojure and ClojureScript.
Until now I find it most useful, to use specs as guards in :pre and :post in public functions that rely on data in a certain format.
(defn person-name [person]
{:pre [(s/valid? ::person person)]
:post [(s/valid? string? %)]}
(str (::first-name person) " " (::last-name person)))
The issue with that approach is, that I get a java.lang.AssertionError: Assert failed: (s/valid? ::person person) without any information about what exactly did not met the specification.
Has anyone an idea how to get a better error message in :pre or :post guards?
I know about conform and explain*, but that does not help in those :pre or :post guards.
In newer alphas, there is now s/assert which can be used to assert that an input or return value matches a spec. If valid, the original value is returned. If invalid, an assertion error is thrown with the explain result. Assertions can be turned on or off and can even optionally be omitted from the compiled code entirely to have 0 production impact.
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::person (s/keys :req [::first-name ::last-name]))
(defn person-name [person]
(s/assert ::person person)
(s/assert string? (str (::first-name person) " " (::last-name person))))
(s/check-asserts true)
(person-name 10)
=> CompilerException clojure.lang.ExceptionInfo: Spec assertion failed
val: 10 fails predicate: map?
:clojure.spec/failure :assertion-failed
#:clojure.spec{:problems [{:path [], :pred map?, :val 10, :via [], :in []}], :failure :assertion-failed}
I think the idea is that you use spec/instrument to validate function input and output rather than pre and post conditions.
There's a good example toward the bottom of this blog post: http://gigasquidsoftware.com/blog/2016/05/29/one-fish-spec-fish/ . Quick summary: you can define a spec for a function, including both input and return values using the :args and :ret keys (thus replacing both pre and post conditions), with spec/fdef, instrument it, and you get output similar to using explain when it fails to meet spec.
Minimal example derived from that link:
(spec/fdef your-func
:args even?
:ret string?)
(spec/instrument #'your-func)
And that's equivalent to putting a precondition that the function has an integer argument and a postcondition that it returns a string. Except you get much more useful errors, just like you're looking for.
More details in the official guide: https://clojure.org/guides/spec ---see under the heading "Spec'ing functions".
Without taking into account if you should use pre and post conditions to validate function arguments, there is a way to print somewhat clearer messages from pre and post conditions by wrapping your predicate with clojure.test/is, as suggested in the answer below:
How can I get Clojure :pre & :post to report their failing value?
So then your code could look like this:
(ns pre-post-messages.core
(:require [clojure.spec :as s]
[clojure.test :as t]))
(defn person-name [person]
{:pre [(t/is (s/valid? ::person person))]
:post [(t/is (s/valid? string? %))]}
(str (::first-name person) " " (::last-name person)))
(def try-1
{:first-name "Anna Vissi"})
(def try-2
{::first-name "Anna"
::last-name "Vissi"
::email "Anna#Vissi.com"})
(s/def ::person (s/keys :req [::first-name ::last-name ::email]))
Evaluating
pre-post-messages.core> (person-name try-2)
would produce
"Anna Vissi"
and evaluating
pre-post-messages.core> (person-name try-1)
would produce
FAIL in () (core.clj:6)
expected: (s/valid? :pre-post-messages.core/person person)
actual: (not (s/valid? :pre-post-messages.core/person {:first-name "Anna Vissi"}))
AssertionError Assert failed: (t/is (s/valid? :pre-post-messages.core/person person)) pre-post-messages.core/person-name (core.clj:5)
This is useful when you don't want to use s/assert, or can not enable s/check-assserts. Improving on MicSokoli's answer:
:pre simply cares that the values returned are all truthy, so we can convert the return value "Success!\n" to true (for strictness) and throw an error with the explanation and the input data in case the output is not successful.
(defn validate [spec input]
(let [explanation (s/explain-str spec input)]
(if (= explanation "Success!\n")
true
(throw (ex-info explanation {:input input}))))
A variation of this could be this one, but it would run the spec twice:
(defn validate [spec input]
(if (s/valid? spec input)
true
(throw (ex-info (s/explain spec input) {:input input}))))
Usage:
(defn person-name [person]
{:pre [(validate ::person person)]}
(str (::first-name person) " " (::last-name person)))