I want to write a spec with keys/keys* but being able to inline the value specs, which is not supported by design, and I get the reasoning behind it. However sometimes you do want (or simply have, by legacy or 3rd-party) coupling between keys and values when there is specific context to the map.
I'm still new to spec and this is just my first time integrating it with an existing project and it constantly gives me issues because it is assuming way too much, especially because of the reason mentioned above. E.g. imagine a map that describes a time period and has an until key for a date, and in the same ns there's a map for list processing and there's an until that takes a predicate function. I now need to mess with manually writing fully namespaced keys for namespaces that don't even exist (aliasing is cute but it would have to be duplicated constantly across multiple namespaces/files). Aside from being irritating I feel like it's also error-prone.
And another place where keys/keys* assumes too much is if I even want keywords as my keys. I'm writing a DSL for non-programmers but technical users right now, and bottom line is that I want to spec a map with symbols as keys. That doesn't seem to be supported whatsoever.
Is there something I'm not getting? or is spec really missing essential functionality?
You can spec a map with symbols as keys either with map-of:
(s/def ::sm (s/map-of symbol? any?))
or by spec'ing the map as a collection of entries:
(s/def ::sm (s/every (s/tuple symbol? any?) :kind map? :into {}))
The latter is particularly interesting as instead of a single tuple you can s/or many different kinds of tuples to describe more interesting maps. You can even connect those symbols to other existing specs in this manner:
(s/def ::attr1 int?)
(s/def ::attr2 boolean?)
(s/def ::sm (s/every (s/or :foo (s/tuple #{'foo} ::attr1)
:bar (s/tuple #{'bar} ::attr2))
:kind map? :into {}))
(s/valid? ::sm {'foo 10 'bar true}) ;; => true
I now need to mess with manually writing fully namespaced keys for
namespaces that don't even exist
I've been using this approach as well, and I think I actually like it more than I like making sure that your keywords' namespaces always correspond to real Clojure NS forms. I use keywords like :business-domain-concept/a-name rather than :my-project.util.lists/a-name.
You can make keywords with arbitrary namespaces that don't map to any Clojure NS. For instance, in your until situation, you could define a :date/until spec that describes dates, and a (perhaps there's a better name for this one) :list/until spec that describes your list processing map's field.
It sounds like you're already aware of this arbitrary-keyword-namespace approach - in particular, I buy that it feels error-prone, since you're typing this stuff out by hand and spec doesn't appear to choke if you feed your s/keys a :fate/until by accident. FWIW, though, I think you're currently feeling the pain that namespaced keywords are specifically intended to solve: you're in one Clojure file, you've got two maps with keys named until, and they mean two completely different things.
I'm writing a DSL for non-programmers but technical users right now,
and bottom line is that I want to spec a map with symbols as keys.
I think that map-of is what you want here:
user=> (s/def ::valid-symbols #{'foo 'bar 'baz})
:user/valid-symbols
user=> (s/def ::symbol-map (s/map-of ::valid-symbols int?))
:user/symbol-map
user=> (s/valid? ::symbol-map {'foo 1 'bar 3})
true
user=> (s/valid? ::symbol-map {'foo 1 'quux 3})
false
Related
I'm quite new to functional programming and clojure. I like it.
I'd like to know what the community thinks about following fn-naming approach:
Is it a feasible way to go with such naming, or is it for some reason something to avoid?
Example:
(defn coll<spec>?
"Checks wether the given thing is a collection of elements of ::spec-type"
[spec-type thing]
(and (coll? thing)
(every? #(spec/valid? spec-type %)
thing)))
(defn coll<spec>?nil
"Checks if the given thing is a collection of elements of ::spec-type or nil"
[spec-type thing]
(or (nil? thing)
(coll<spec>? spec-type thing)))
now ... somewhere else I'm using partial applications in clojure defrecrod/specs ...
similar to this:
; somewhere inside a defrecord
^{:spec (partial p?/coll<spec>?nil ::widgets)} widgets component-widgets
now, I created this for colls, sets, hash-maps with a generic-like form for specs (internal application of spec/valid? ...) and with a direct appication of predicates and respectively a <p> instead of the <spec>
currently I'm discussing with my colleagues wether this is a decent and meaningful and useful approach - since it is at least valid to name function like this in clojure - or wether the community thinks this is rather a no-go.
I'd very much like your educated opinions on that.
It's a weird naming convention, but a lot of projects use weird naming conventions because their authors believe they help. I imagine if I read your codebase, variable naming conventions would probably not be the thing that surprises me most. This is not an insult: every project has some stuff that surprises me.
But I don't see why you need these specific functions at all. As Sean Corfield says in a comment, there are good spec combinator functions to build fancier specs out of simpler ones. Instead of writing coll<spec>?, you could just combine s/valid? with s/coll-of:
(s/valid? (s/coll-of spec-type) thing)
Likewise you can use s/or and nil? to come up with coll<spec>?nil.
(s/valid? (s/or nil? (s/coll-of spec-type) thing))
This is obviously more to write at each call site than a mere coll<spec>?nil, so you may dismiss my advice. But the point is you can extract functions for defining these specs, and use those specs.
(defn nil-or-coll-of [spec]
(s/or nil? (s/coll-of spec)))
(s/valid? (nil-or-coll-of spec-type) thing)
Importantly, this means you could pass the result of nil-or-coll-of to some other function that expects a spec, perhaps to build a larger spec out of it, or to try to conform it. You can't do that if all your functions have s/valid? baked into them.
Think in terms of composing general functions, not defining a new function from scratch for each thing you want to do.
I'm trying Interactive Development with clojure.spec and have a problem with specs for function arguments that cannot change a lot. For instance, if a function receives a file name as argument, I can write the following spec:
(s/def ::file-name string?)
(s/fdef test-fn :args (s/cat :x ::file-name))
If I exercise it:
(s/exercise-fn `test-fn)
The function will be tested with lots of random file names that will fail to read any file. It is possible to limit the file names to a set of valid and invalid file names. That is fine for testing, but it will make the spec specific to the chosen set.
That is a problem not only with file names, but with any complex data structure where even small random changes may render it useless.
What should I do? Any relevant technique or good practice?
That is fine for testing, but it will make the spec specific to the chosen set.
This is where custom generators are useful:
(s/def ::file-name
(s/with-gen string? #(gen/elements #{"good.txt" "bad.txt"})))
(s/fdef test-fn :args (s/cat :x ::file-name))
(Where gen is clojure.test.check.generators or clojure.spec.gen.alpha.)
Now your spec's predicate is still string? but the values generated from this spec will always be from #{"good.txt" "bad.txt"}. You can compose generators in several ways, for example you could make a generator that took from a string set ~50% of the time and generated a purely "random" string for the other ~50%.
FYI clojure.spec.test.alpha/check also takes an opts map that allows you to override/specify generators.
With clojure.spec, is there a way to define a more “human-readable” spec for nested maps? The following doesn't read very well:
(s/def ::my-domain-entity (s/keys :req-un [:a :b])
(s/def :a (s/keys :req-un [:c :d]))
(s/def :b boolean?)
(s/def :c number?)
(s/def :d string?)
Given that the shape of a conforming entity is something like
{:a {:c 1 :d "hello"} :b false}
My complaint is that it becomes hard(er) to read a spec if it has any sort of nested maps or any deep structure… because you are chasing keys up and down a file and they aren’t “in place” declarations.
To compare, something like schema allows a more readable nested syntax that closely mirrors the actual data shape:
(m/defschema my-domain-entity {:a {:c sc/number :d sc/string} :b sc/bool})
Can this be done in clojure.spec?
One of spec's value propositions is that it does not attempt to define an actual schema. It does not bind the definition of an entity to the definition of its components. To quote from the spec rationale:
Most systems for specifying structures conflate the specification of the key set (e.g. of keys in a map, fields in an object) with the specification of the values designated by those keys. I.e. in such approaches the schema for a map might say :a-key’s type is x-type and :b-key’s type is y-type. This is a major source of rigidity and redundancy.
In Clojure we gain power by dynamically composing, merging and building up maps. We routinely deal with optional and partial data, data produced by unreliable external sources, dynamic queries etc. These maps represent various sets, subsets, intersections and unions of the same keys, and in general ought to have the same semantic for the same key wherever it is used. Defining specifications of every subset/union/intersection, and then redundantly stating the semantic of each key is both an antipattern and unworkable in the most dynamic cases.
So to answer the question directly, no, spec does not provide this type of specification, as it was designed specifically that way. You trade off some level of human readability that you would have in a schema-like definition, for a more dynamic, composable, flexible specification.
Though it was not in your question, consider the benefit of using a system that decouples the definition of an entity from the definition of its components. It's contrived, but consider defining a car (keeping it simple to save space here, just using tires and chassis):
(s/def ::car (s/keys :req [::tires ::chassis]))
We define it once, and we can put any configuration of tires we want on it:
(s/def ::tires (s/coll-of ::tire :count 4))
(s/def ::tire (s/or :goodyear ::goodyear}
:michelin ::michelin))
(s/def ::goodyear #{"all-season" "sport" "value"})
(s/def ::michelin #{"smooth ride" "sport performance"})
(s/def ::chassis #{"family sedan" "sports"})
The following are different configurations, but all are valid cars:
(s/valid? ::car {::tires ["sport" "sport" "sport" "sport"]
::chassis "sports"})
(s/valid? ::car {::tires ["smooth ride" "smooth ride"
"smooth ride" "smooth ride"]
::chassis "family sedan"})
It's contrived, but clear to see that there is flexibility in defining the components as separate from what the components come together to form. Tires have their own specifications, and their specification is not what defines a car, even though they are components of the car. It's more verbose, but much more flexible.
I've been trying out clojure.spec, and one idea I have for how to use it is to generate the UI for editing an instance of the map I'm specifying. For example, it might generate a web form with a datepicker field for a key that's specified to be a date, that kind of thing.
There is a get-spec method in the library, but it seems like there are no functions that operate on specifications-as-values in the way I need. Is there some way to do things like take a map spec and get back the required keys for that map as a vector? Is this kind of metaprogramming with specifications outside of the intended use case of clojure.spec?
Metaprogramming with specs is definitely within the intended use case of clojure.spec.
We have not yet released (but have written and intend to) specs for spec forms themselves. With these, it is possible to conform a spec form itself and get back a data structure representing the spec which can be used to (for example), grab the required keys from a map spec.
Conforming with a ::spec spec might look something like this:
user=> (s/def ::name string?)
:user/name
user=> (s/def ::m (s/keys :req [::name]))
:user/m
user=> (s/conform ::spec (s/form ::m))
[:form {:s clojure.spec/keys, :args {:req [[:key :user/name]]}}]
You could then pluck the set of keys out of that structure.
I have a defrecord called a bag. It behaves like a list of item to count. This is sometimes called a frequency or a census. I want to be able to do the following
(def b (bag/create [:k 1 :k2 3])
(keys bag)
=> (:k :k1)
I tried the following:
(defrecord MapBag [state]
Bag
(put-n [self item n]
(let [new-n (+ n (count self item))]
(MapBag. (assoc state item new-n))))
;... some stuff
java.util.Map
(getKeys [self] (keys state)) ;TODO TEST
Object
(toString [self]
(str ("Bag: " (:state self)))))
When I try to require it in a repl I get:
java.lang.ClassFormatError: Duplicate interface name in class file compile__stub/techne/bag/MapBag (bag.clj:12)
What is going on? How do I get a keys function on my bag? Also am I going about this the correct way by assuming clojure's keys function eventually calls getKeys on the map that is its argument?
Defrecord automatically makes sure that any record it defines participates in the ipersistentmap interface. So you can call keys on it without doing anything.
So you can define a record, and instantiate and call keys like this:
user> (defrecord rec [k1 k2])
user.rec
user> (def a-rec (rec. 1 2))
#'user/a-rec
user> (keys a-rec)
(:k1 :k2)
Your error message indicates that one of your declarations is duplicating an interface that defrecord gives you for free. I think it might actually be both.
Is there some reason why you cant just use a plain vanilla map for your purposes? With clojure, you often want to use plain vanilla data structures when you can.
Edit: if for whatever reason you don't want the ipersistentmap included, look into deftype.
Rob's answer is of course correct; I'm posting this one in response to the OP's comment on it -- perhaps it might be helpful in implementing the required functionality with deftype.
I have once written an implementation of a "default map" for Clojure, which acts just like a regular map except it returns a fixed default value when asked about a key not present inside it. The code is in this Gist.
I'm not sure if it will suit your use case directly, although you can use it to do things like
user> (:earth (assoc (DefaultMap. 0 {}) :earth 8000000000))
8000000000
user> (:mars (assoc (DefaultMap. 0 {}) :earth 8000000000))
0
More importantly, it should give you an idea of what's involved in writing this sort of thing with deftype.
Then again, it's based on clojure.core/emit-defrecord, so you might look at that part of Clojure's sources instead... It's doing a lot of things which you won't have to (because it's a function for preparing macro expansions -- there's lots of syntax-quoting and the like inside it which you have to strip away from it to use the code directly), but it is certainly the highest quality source of information possible. Here's a direct link to that point in the source for the 1.2.0 release of Clojure.
Update:
One more thing I realised might be important. If you rely on a special map-like type for implementing this sort of thing, the client might merge it into a regular map and lose the "defaulting" functionality (and indeed any other special functionality) in the process. As long as the "map-likeness" illusion maintained by your type is complete enough for it to be used as a regular map, passed to Clojure's standard function etc., I think there might not be a way around that.
So, at some level the client will probably have to know that there's some "magic" involved; if they get correct answers to queries like (:mars {...}) (with no :mars in the {...}), they'll have to remember not to merge this into a regular map (merge-ing the other way around would work fine).