I'm looking to generate a set of specs based off some data I'm pulling from a request. I'd like to dynamically define some specs based off the data I receive.
(def my-key :frame-data/pretty_name) ;;imagine this and the validator aren't hardcoded
(def validator string?)
(s/def my-key validator?) ;; defines my-ns/my-key, instead of `:frame-data/pretty_name.
(s/describe my-key) ;; string? ;;which sorta works, but its looking up `my-ns/my-key` instead of :frame-data/pretty_name
My goal is to have a spec that looks like I wrote:
(s/def :frame-data/pretty_name string?)
I'm new to clojure so I don't have a great idea of how it could be done, but I've tried a few things:
(s/def (eval my-key) validator) ;;Assert Failed: k must be a namespaced keyword or resolveable symbol
(definemacro def-spec [key validator]
'(s/def ~key ~validator))
(def-spec my-key validator) ;; my-ns/my-key ;; returns the same as earlier
and many variations on that, but I'm not sure how defining a spec dynamically can be done but it feels like it should be.
You could do a macroexpand on a clojure.spec.alpha/def form to see what it expands to:
(macroexpand `(s/def :frame-data/pretty_name string?))
;; => (clojure.spec.alpha/def-impl (quote :frame-data/pretty_name) (quote clojure.core/string?) string?)
Or have a look at the source code of clojure.spec.alpha/def.
Then write your own macro that doesn't quote the spec key in order for it to get evaluated:
(defmacro defspec [k spec-form]
`(s/def-impl ~k (quote ~spec-form) ~spec-form))
Example:
(defspec my-key validator)
(s/valid? my-key "abc")
;; => true
(s/valid? my-key 123)
;; => false
(s/describe my-key)
;; => validator
If you don't like that (s/describe my-key) returns validator, try replacing (quote ~spec-form) in the macro definition by just ~spec-form, maybe you will like that better.
I'd love to have a better answer, but it seems like this works, only s/describe returns validator instead of string?
(eval `(s/def ~my-key validator)) ;; :frame-data/pretty_name
(s/describe my-key) ;; validator
(s/valid? my-key "some string") ;; true
(s/valid? my-key 123) ;; false
Alternatively a weird looking macro works, but the double tilde looks strange, and it feels weird to use eval:
(defmacro define-spec [spec-key validator]
`(eval `(s/def ~~spec-key validator)))
(define-spec my-key validator) ;; :frame-data/pretty_name
Related
I want to write a spec for a map that either has the key :rule/children or has two keys - :condition/field and :condition/predicate. This is what I have tried:
(s/keys :req [(s/or :children :rule/children :condition (s/and :condition/field :condition/predicate))])
It results in the error message:
Caused by: java.lang.AssertionError: Assert failed: all keys must be namespace-qualified keywords
(every? (fn* [p1__1917#] (c/and (keyword? p1__1917#) (namespace p1__1917#))) (concat req-keys req-un-specs opt opt-un))
I know that for s/or each path must be named. Here there are two paths - this map can either have :children or be a :condition. It is a condition only if it has the two keys :condition/field and :condition/predicate.
In keys specs you can use plain or and and to do this:
(s/def ::map-spec
(s/keys :req [(or :rule/children (and :condition/field :condition/predicate))]))
(s/conform ::map-spec {:rule/children 1}) ;; valid
(s/conform ::map-spec {:condition/field 1}) ;; invalid
(s/conform ::map-spec {:condition/field 1 :condition/predicate 2}) ;; valid
I am super confused by Clojure Spec. When I run in the repl by entering:
(require '[clojure.spec.alpha :as s])
And then add:
(s/valid? even? 10)
I get //true. And when I run:
(s/valid? even? 11)
//False. Ok so that works. Then when I require spec in my core.clj as:
(ns spam-problem.core
(:require [clojure.spec.alpha :as s]
[clojure.spec.gen.alpha :as gen]))
And try a simple validation to get it to throw an error, nothing happens:
(defn -main
"I don't do a whole lot ... yet."
[& args]
(s/valid? even? 11))
I have no idea what I'm doing wrong here and am very confused about how spec is supposed to work. I am running this by using command lein run. Is there another way you're supposed to run it?
I understand what you are feeling because once I got into Spec it caused me the same thoughts. What really helped me to solve the problem in my mind is to considering Spec being not a final library but rather a framework. In my projects, usually I've got a special module with high-level wrappers above basic spec capabilities. I believe, you might do the same: define a function that takes data, spec and raises those error message you desire to have in terms of your business-logic. Here is a small example of my code:
(ns project.spec
(:require [clojure.spec.alpha :as s]))
;; it's better to define that value is a constant
(def invalid :clojure.spec.alpha/invalid)
(defn validate
"Either returns coerced data or nil in case of error."
[spec value]
(let [result (s/conform spec value)]
(if (= result invalid)
nil
result)))
(defn spec-error
"Returns an error map for data structure that does not fit spec."
[spec data]
(s/explain-data spec data))
Now, let's prepare some specs:
(defn x-integer? [x]
(if (integer? x)
x
(if (string? x)
(try
(Integer/parseInt x)
(catch Exception e
invalid))
invalid)))
(def ->int (s/conformer x-integer?))
(s/def :opt.visits/fromDate ->int)
(s/def :opt.visits/toDate ->int)
(s/def :opt.visits/country string?)
(s/def :opt.visits/toDistance ->int)
(s/def :opt.visits/params
(s/keys :opt-un [:opt.visits/fromDate
:opt.visits/toDate
:opt.visits/country
:opt.visits/toDistance]))
And here are some usage examples:
(let [spec :opt.visits/params
data {:some :map :goes :here}]
(if-let [cleaned-data (validate spec data)]
;; cleaned-data has values coerced from strings to integers,
;; quite useful for POST parameters
(positive-logic cleaned-data)
;; error values holds a map that describes an error
(let [error (spec-error spec data)]
(error-logic-goes-here error))))
What might be improved here is to have a combo-function with both validate and error functionality. Such a function could return a vector of two values: success flag and either result or error data structure as follows:
[true {:foo 42}] ;; good result
[false {:error :map}] ;; bad result
The Spec library does not dictate a single way of processing data; that's why it's really good and flexible.
valid? is a predicate that returns true or false. Your program isn’t doing anything with the return value. Try printing it to console or using s/assert if you want to throw an exception:
If (check-asserts?) is false at runtime, always returns x. Defaults to
value of 'clojure.spec.check-asserts' system property, or false if not
set. You can toggle check-asserts? with (check-asserts bool).
So you may need to set (s/check-asserts true) to have s/assert throw exceptions:
(clojure.spec.alpha/assert even? 3)
=> 3
(clojure.spec.alpha/check-asserts?)
=> false
(clojure.spec.alpha/check-asserts true)
=> true
(clojure.spec.alpha/assert even? 3)
ExceptionInfo Spec assertion failed
val: 3 fails predicate: :clojure.spec.alpha/unknown
clojure.core/ex-info (core.clj:4739)
I'm learning Clojure and enjoying it but find an inconsistency in Records that puzzles me: why doesn't the default map constructor (map->Whatever) check for data integrity when creating a new Record? For instance:
user=> (defrecord Person [first-name last-name])
#<Class#46ffda99 user.Person>
user=> (map->Person {:first-name "Rich" :last-name "Hickey"})
#user.Person {:first-name "Rich" :last-name "Hickey"}
user=> (map->Person {:first-game "Rich" :last-name "Hickey"})
#user.Person {:first-game "Rich" :first-name nil :last-name "Hickey"}
I believe the Map is not required to define all the fields in the Record definition and it is also allowed to contain extra fields that aren't part of the Record definition. Also I understand that I can define my own constructor which wraps the default constructor and I think a :post condition can then be used to check for correct (and comprehensive) Record creation (have not been successful in getting that to work).
My question is: Is there an idiomatic Clojure way to verify data during Record construction from a Map? And, is there something that I'm missing here about Records?
Thank you.
I think your comprehensiveness requirement is already quite specific, so nothing built-in I know of covers this.
One thing you can do nowadays is use clojure.spec to provide an s/fdef for your constructor function (and then instrument it).
(require '[clojure.spec.alpha :as s]
'[clojure.spec.test.alpha :as stest])
(defrecord Person [first-name last-name])
(s/fdef map->Person
:args (s/cat :map (s/keys :req-un [::first-name ::last-name])))
(stest/instrument `map->Person)
(map->Person {:first-name "Rich", :last-name "Hickey"})
(map->Person {:first-game "Rich", :last-name "Hickey"}) ; now fails
(If specs are defined for ::first-name and ::last-name those will be checked as well.)
Another option is to use Plumatic Schema to create a wrapper "constructor" function specifying the allowed keys. For example:
(def FooBar {(s/required-key :foo) s/Str (s/required-key :bar) s/Keyword})
(s/validate FooBar {:foo "f" :bar :b})
;; {:foo "f" :bar :b}
(s/validate FooBar {:foo :f})
;; RuntimeException: Value does not match schema:
;; {:foo (not (instance? java.lang.String :f)),
;; :bar missing-required-key}
The first line defines a schema that accepts only maps like:
{ :foo "hello" :bar :some-kw }
You wrapper constructor would look something like:
(def NameMap {(s/required-key :first-name) s/Str (s/required-key :last-name) s/Str})
(s/defn safe->person
[name-map :- NameMap]
(map->Person name-map))
or
(s/defn safe->person-2
[name-map]
(assert (= #{:first-name :last-name} (set (keys name-map))))
(map->Person name-map))
I am using Clojure.spec to validate the structure of JSON (and later conform it into another format):
(s/def ::yes string?)
(s/def ::my-test (s/keys :req [::yes]))
(def my-json (json/read-json "{\"yes\": \"yes\"}")) ; => {:yes "yes"}
(s/valid? ::my-test my-json) ; => false
(s/valid? ::my-test {::yes "yes"}) ; => true
(s/explain ::my-test {:yes "yes"})
; => val: {:yes "yes"} fails spec: :spec.core/my-test predicate:
; (contains? % :spec.core/yes)
(Here s refers to the clojure.spec namespace and json to clojure.data.json.)
As can be seen above the s/valid? fails for the parsed JSON because the keywords are not namespaced.
How can I adjust the code so that the JSON is seen as valid?
You can do the following which will work:
(def my-json (json/read-str "{\"yes\": \"yes\"}" :key-fn #(keyword (str *ns*) %)))
I'm not sure if this is the right/idiomatic way to handle it - I guess it depends on the case.
I used the last days to dig deeper into clojure.spec in Clojure and ClojureScript.
Until now I find it most useful, to use specs as guards in :pre and :post in public functions that rely on data in a certain format.
(defn person-name [person]
{:pre [(s/valid? ::person person)]
:post [(s/valid? string? %)]}
(str (::first-name person) " " (::last-name person)))
The issue with that approach is, that I get a java.lang.AssertionError: Assert failed: (s/valid? ::person person) without any information about what exactly did not met the specification.
Has anyone an idea how to get a better error message in :pre or :post guards?
I know about conform and explain*, but that does not help in those :pre or :post guards.
In newer alphas, there is now s/assert which can be used to assert that an input or return value matches a spec. If valid, the original value is returned. If invalid, an assertion error is thrown with the explain result. Assertions can be turned on or off and can even optionally be omitted from the compiled code entirely to have 0 production impact.
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::person (s/keys :req [::first-name ::last-name]))
(defn person-name [person]
(s/assert ::person person)
(s/assert string? (str (::first-name person) " " (::last-name person))))
(s/check-asserts true)
(person-name 10)
=> CompilerException clojure.lang.ExceptionInfo: Spec assertion failed
val: 10 fails predicate: map?
:clojure.spec/failure :assertion-failed
#:clojure.spec{:problems [{:path [], :pred map?, :val 10, :via [], :in []}], :failure :assertion-failed}
I think the idea is that you use spec/instrument to validate function input and output rather than pre and post conditions.
There's a good example toward the bottom of this blog post: http://gigasquidsoftware.com/blog/2016/05/29/one-fish-spec-fish/ . Quick summary: you can define a spec for a function, including both input and return values using the :args and :ret keys (thus replacing both pre and post conditions), with spec/fdef, instrument it, and you get output similar to using explain when it fails to meet spec.
Minimal example derived from that link:
(spec/fdef your-func
:args even?
:ret string?)
(spec/instrument #'your-func)
And that's equivalent to putting a precondition that the function has an integer argument and a postcondition that it returns a string. Except you get much more useful errors, just like you're looking for.
More details in the official guide: https://clojure.org/guides/spec ---see under the heading "Spec'ing functions".
Without taking into account if you should use pre and post conditions to validate function arguments, there is a way to print somewhat clearer messages from pre and post conditions by wrapping your predicate with clojure.test/is, as suggested in the answer below:
How can I get Clojure :pre & :post to report their failing value?
So then your code could look like this:
(ns pre-post-messages.core
(:require [clojure.spec :as s]
[clojure.test :as t]))
(defn person-name [person]
{:pre [(t/is (s/valid? ::person person))]
:post [(t/is (s/valid? string? %))]}
(str (::first-name person) " " (::last-name person)))
(def try-1
{:first-name "Anna Vissi"})
(def try-2
{::first-name "Anna"
::last-name "Vissi"
::email "Anna#Vissi.com"})
(s/def ::person (s/keys :req [::first-name ::last-name ::email]))
Evaluating
pre-post-messages.core> (person-name try-2)
would produce
"Anna Vissi"
and evaluating
pre-post-messages.core> (person-name try-1)
would produce
FAIL in () (core.clj:6)
expected: (s/valid? :pre-post-messages.core/person person)
actual: (not (s/valid? :pre-post-messages.core/person {:first-name "Anna Vissi"}))
AssertionError Assert failed: (t/is (s/valid? :pre-post-messages.core/person person)) pre-post-messages.core/person-name (core.clj:5)
This is useful when you don't want to use s/assert, or can not enable s/check-assserts. Improving on MicSokoli's answer:
:pre simply cares that the values returned are all truthy, so we can convert the return value "Success!\n" to true (for strictness) and throw an error with the explanation and the input data in case the output is not successful.
(defn validate [spec input]
(let [explanation (s/explain-str spec input)]
(if (= explanation "Success!\n")
true
(throw (ex-info explanation {:input input}))))
A variation of this could be this one, but it would run the spec twice:
(defn validate [spec input]
(if (s/valid? spec input)
true
(throw (ex-info (s/explain spec input) {:input input}))))
Usage:
(defn person-name [person]
{:pre [(validate ::person person)]}
(str (::first-name person) " " (::last-name person)))