Assuming I have already defined a spec from which I'd like to generate test data:
(s/def :customer/id uuid?)
(s/def :customer/given-name string?)
(s/def :customer/surname string?)
(s/def :customer/age pos?)
(s/def ::customer
(s/keys
:req-un [:customer/id
:customer/given-name
:customer/surname
:customer/age]))
In generating test data, I'd like to override how ids are generated in order to ensure they're from a smaller pool to encourage collisions:
(defn customer-generator
[id-count]
(gen/let [id-pool (gen/not-empty (gen/vector (s/gen :customer/id) id-count))]
(assoc (s/gen ::customer) :id (gen/element id-pool))))
Is there a way I can simplify this by overriding the :customer/id generator in my test code and then just using (s/gen ::customer)? So, something like the following:
(with-generators [:customer/id (gen/not-empty (gen/vector (s/gen :customer/id) id-count)))]
(s/gen ::customer))
Officially, you can override generators for specs by passing an overrides map to s/gen (See the docstring for more details):
(s/def :customer/id uuid?)
(s/def :customer/given-name string?)
(s/def :customer/surname string?)
(s/def :customer/age nat-int?)
(s/def ::customer
(s/keys
:req-un [:customer/id
:customer/given-name
:customer/surname
:customer/age]))
(def fixed-customer-id (java.util.UUID/randomUUID))
fixed-customer-id
;=> #uuid "c73ff5ea-8702-4066-a31d-bc4cc7015811"
(gen/generate (s/gen ::customer {:customer/id #(s/gen #{fixed-customer-id})}))
;=> {:id #uuid "c73ff5ea-8702-4066-a31d-bc4cc7015811",
; :given-name "1042IKQhd",
; :surname "Uw0AzJzj",
; :age 104}
Alternatively, there is a library for such stuff named genman, which I developed before :)
Using it, you can also write as:
(require '[genman.core :as genman :refer [defgenerator]])
(def fixed-customer-id (java.util.UUID/randomUUID))
(genman/with-gen-group :test
(defgenerator :customer/id
(s/gen #{fixed-customer-id})))
(genman/with-gen-group :test
(gen/generate (genman/gen ::customer)))
Clojure spec uses test.check internally to generate sample values. Here is how test.check can be overridden. Whenever trying to write unit tests with a "fake" function, with-redefs is your friend:
(ns tst.demo.core
(:use tupelo.core tupelo.test)
(:require
[clojure.test.check.generators :as gen]
))
(def id-gen gen/uuid)
(dotest
(newline)
(spyx-pretty (take 3 (gen/sample-seq id-gen)))
(newline)
(with-redefs [id-gen (gen/choose 1 5)]
(spyx-pretty (take 33 (gen/sample-seq id-gen))))
(newline)
)
with result:
-----------------------------------
Clojure 1.10.3 Java 15.0.2
-----------------------------------
Testing tst.demo.core
(take 3 (gen/sample-seq id-gen)) =>
[#uuid "cbfea340-1346-429f-ba68-181e657acba5"
#uuid "7c119cf7-0842-4dd0-a23d-f95b6a68f808"
#uuid "ca35cb86-1385-46ad-8fc2-e05cf7a1220a"]
(take 33 (gen/sample-seq id-gen)) =>
[5 4 3 3 2 2 3 1 2 1 4 1 2 2 4 3 5 2 3 5 3 2 3 2 3 5 5 5 5 1 3 2 2]
Example created
using my favorite template project.
Update
Unfortunately, the above technique does not work for Clojure Spec since (s/def ...) uses a global registery of Spec definitions, and is therefore immune to with-redefs. However, we can overcome this definition by simply redefining the desired spec in the unit test namespace like:
(ns tst.demo.core
(:use tupelo.core tupelo.test)
(:require
[clojure.spec.alpha :as s]
[clojure.spec.gen.alpha :as gen]
))
(s/def :app/id (s/int-in 9 99))
(s/def :app/name string?)
(s/def :app/cust (s/keys :req-un [:app/id :app/name]))
(dotest
(newline)
(spyx-pretty (gen/sample (s/gen :app/cust)))
(newline)
(s/def :app/id (s/int-in 2 5)) ; overwrite the definition of :app/id for testing
(spyx-pretty (gen/sample (s/gen :app/cust)))
(newline))
with result
-----------------------------------
Clojure 1.10.3 Java 15.0.2
-----------------------------------
Testing tst.demo.core
(gen/sample (s/gen :app/cust)) =>
[{:id 10, :name ""}
{:id 9, :name "n"}
{:id 10, :name "fh"}
{:id 9, :name "aI"}
{:id 11, :name "8v5F"}
{:id 10, :name ""}
{:id 10, :name "7"}
{:id 10, :name "3m6Wi"}
{:id 13, :name "OG2Qzfqe"}
{:id 10, :name ""}]
(gen/sample (s/gen :app/cust)) =>
[{:id 3, :name ""}
{:id 3, :name ""}
{:id 2, :name "5e"}
{:id 3, :name ""}
{:id 2, :name "y01C"}
{:id 3, :name "l2"}
{:id 3, :name "c"}
{:id 3, :name "pF"}
{:id 4, :name "0yrxyJ7l"}
{:id 4, :name "40"}]
So, it's a little ugly, but the redefinition of :app/id does the trick, and it only takes effect during unit test runs, leaving the main application unaffected.
user> (def ^:dynamic *idgen* (s/gen uuid?))
#'user/*idgen*
user> (s/def :customer/id (s/with-gen uuid? (fn [] ##'*idgen*)))
:customer/id
user> (s/def :customer/age pos-int?)
:customer/age
user> (s/def ::customer (s/keys :req-un [:customer/id :customer/age]))
:user/customer
user> (gen/sample (s/gen ::customer))
({:id #uuid "d18896f1-6199-42bf-9be3-3d0652583902", :age 1}
{:id #uuid "b6209798-4ffa-4e20-9a76-b3a799a31ec6", :age 2}
{:id #uuid "6f9c6400-8d79-417c-bc62-6b4557f7d162", :age 1}
{:id #uuid "47b71396-1b5f-4cf4-bd80-edf4792300c8", :age 2}
{:id #uuid "808692b9-0698-4fb8-a0c5-3918e42e8f37", :age 2}
{:id #uuid "ba663f0a-7c99-4967-a2df-3ec6cb04f514", :age 1}
{:id #uuid "8521b611-c38c-4ea9-ae84-35c8a2d2ff2f", :age 4}
{:id #uuid "c559d48d-4c50-438f-846c-780cdcdf39d5", :age 3}
{:id #uuid "03c2c114-03a0-4709-b9dc-6d326a17b69d", :age 40}
{:id #uuid "14715a50-81c5-48e4-bffe-e194631bb64b", :age 4})
user> (binding [*idgen* (let [idpool (gen/sample (s/gen :customer/id) 5)] (gen/elements idpool))] (gen/sample (s/gen ::customer)))
({:id #uuid "3e64131d-e7ad-4450-993d-fa651339df1c", :age 2}
{:id #uuid "575b2bef-956d-4c42-bdfa-982c7756a33c", :age 1}
{:id #uuid "575b2bef-956d-4c42-bdfa-982c7756a33c", :age 1}
{:id #uuid "3e64131d-e7ad-4450-993d-fa651339df1c", :age 1}
{:id #uuid "1a2eafed-8242-4229-b432-99edb361569d", :age 3}
{:id #uuid "1a2eafed-8242-4229-b432-99edb361569d", :age 1}
{:id #uuid "05bd521a-26f9-46e0-8b26-f798e0bf0452", :age 3}
{:id #uuid "575b2bef-956d-4c42-bdfa-982c7756a33c", :age 19}
{:id #uuid "31b80714-7ae0-40a0-b932-f7b5f078f2ad", :age 2}
{:id #uuid "05bd521a-26f9-46e0-8b26-f798e0bf0452", :age 5})
user>
A little clumsier than what you wanted, but maybe this is adequate.
You are probably better off using binding rather than with-redefs since binding modifies thread-local bindings, whereas with-redefs changes the root binding.
Since this is for generating bad test data, I'd consider avoiding the use of dynamic vars and binding altogether and just use a different spec that is only local to the test env.
I want to select paths of a deeply nested map to keep.
For example:
{:a 1
:b {:c [{:d 1 :e 1}
{:d 2 :e 2}]
:f 1}
:g {:h {:i 4 :j [1 2 3]}}}
I want to select by paths, like so:
(select-paths m [[:a]
[:b :c :e]
[:b :f]
[:g :h :i]])
This would return
{:a 1
:b {:c [{:e 1}
{:e 2}]
:f 1}
:g {:h {:i 4}}}
Essentially the same as Elasticsearch's fields parameter. The format of the paths argument can be something else, this is just the first idea.
I tried two different solutions
Go through the entire map and checking if the full path of the current element is in the given paths. I can't figure out how to handle lists of maps so that they are kept as lists of maps.
Creating select-keys statements from the given paths but again I run into problems with lists of maps - and especially trying to resolve paths of varying depths that have some common depth.
I looked at spectre but I didn't see anything that would do this. Any map or postwalk based solution I come up with turns into something incredibly convoluted at some point. I must be thinking about this the wrong way.
If there's a way to do this with raw json, that would be fine as well. Or even a Java solution.
There is no simple way to accomplish your goal. The automatic processing implied for the sequence under [:b :c] is also problematic.
You can get partway there using the Tupelo Forest library. See the Lightning Talk video from Clojure/Conj 2017.
I did some additional work in data destructuring that you may find useful building the tupelo.core/destruct macro (see examples here). You could follow a similar outline to build a recursive solution to your specific problem.
A related project is Meander. I have worked on my own version which is like a generalized version of tupelo.core/destruct. Given data like this
(def skynet-widgets [{:basic-info {:producer-code "Cyberdyne"}
:widgets [{:widget-code "Model-101"
:widget-type-code "t800"}
{:widget-code "Model-102"
:widget-type-code "t800"}
{:widget-code "Model-201"
:widget-type-code "t1000"}]
:widget-types [{:widget-type-code "t800"
:description "Resistance Infiltrator"}
{:widget-type-code "t1000"
:description "Mimetic polyalloy"}]}
{:basic-info {:producer-code "ACME"}
:widgets [{:widget-code "Dynamite"
:widget-type-code "c40"}]
:widget-types [{:widget-type-code "c40"
:description "Boom!"}]}])
You can search and extract data using a template like this:
(let [root-eid (td/add-entity-edn skynet-widgets)
results (td/match
[{:basic-info {:producer-code ?}
:widgets [{:widget-code ?
:widget-type-code wtc}]
:widget-types [{:widget-type-code wtc
:description ?}]}])]
(is= results
[{:description "Resistance Infiltrator" :widget-code "Model-101" :producer-code "Cyberdyne" :wtc "t800"}
{:description "Resistance Infiltrator" :widget-code "Model-102" :producer-code "Cyberdyne" :wtc "t800"}
{:description "Mimetic polyalloy" :widget-code "Model-201" :producer-code "Cyberdyne" :wtc "t1000"}
{:description "Boom!" :widget-code "Dynamite" :producer-code "ACME" :wtc "c40"}])))
This code is working (see here) but it needs more polish. You could use it as a guide to building a generalized select-paths function.
Can you add any details on how this problem arose or the specific context? That may point to ideas for an alternate solution.
One way of solving this problem would be to generate a set of all subpaths that you accept and then write a recursive function that traverses the data structure and keeps track of the path to the current node. The code that accomplishes that does not need to be very long:
(defn select-paths-from-set [current-path path-set data]
(cond
(map? data) (into {}
(remove nil?)
(for [[k v] data]
(let [p (conj current-path k)]
(if (contains? path-set p)
[k (select-paths-from-set p path-set v)]))))
(sequential? data) (mapv (partial select-paths-from-set current-path path-set) data)
:default data))
(defn select-paths [data paths]
(select-paths-from-set []
(into #{}
(mapcat #(take-while seq (iterate butlast %)))
paths)
data))
(select-paths {:a 1
:b {:c [{:d 1 :e 1}
{:d 2 :e 2}]
:f 1}
:g {:h {:i 4 :j [1 2 3]}}}
[[:a]
[:b :c :e]
[:b :f]
[:g :h :i]])
;; => {:a 1, :b {:c [{:e 1} {:e 2}], :f 1}, :g {:h {:i 4}}}
I am looking to transform a clojure tree structure into a map with its dependencies
For example, an input like:
[{:value "A"}
[{:value "B"}
[{:value "C"} {:value "D"}]
[{:value "E"} [{:value "F"}]]]]
equivalent to:
:A
:B
:C
:D
:E
:F
output:
{:A [:B :E] :B [:C :D] :C [] :D [] :E [:F] :F}
I have taken a look at tree-seq and zippers but can't figure it out!
Here's a way to build up the desired map while using a zipper to traverse the tree. First let's simplify the input tree to match your output format (maps of :value strings → keywords):
(def tree
[{:value "A"}
[{:value "B"} [{:value "C"} {:value "D"}]
{:value "E"} [{:value "F"}]]])
(def simpler-tree
(clojure.walk/postwalk
#(if (map? %) (keyword (:value %)) %)
tree))
;; [:A [:B [:C :D] :E [:F]]]
Then you can traverse the tree with loop/recur and clojure.zip/next, using two loop bindings: the current position in tree, and the map being built.
(loop [loc (z/vector-zip simpler-tree)
deps {}]
(if (z/end? loc)
deps ;; return map when end is reached
(recur
(z/next loc) ;; advance through tree
(if (z/branch? loc)
;; for (non-root) branches, add top-level key with direct descendants
(if-let [parent (some-> (z/prev loc) z/node)]
(assoc deps parent (filterv keyword? (z/children loc)))
deps)
;; otherwise add top-level key with no direct descendants
(assoc deps (z/node loc) [])))))
=> {:A [:B :E], :B [:C :D], :C [], :D [], :E [:F], :F []}
This is easy to do using the tupelo.forest library. I reformatted your source data to make it fit into the Hiccup syntax:
(dotest
(let [relationhip-data-hiccup [:A
[:B
[:C]
[:D]]
[:E
[:F]]]
expected-result {:A [:B :E]
:B [:C :D]
:C []
:D []
:E [:F]
:F []} ]
(with-debug-hid
(with-forest (new-forest)
(let [root-hid (tf/add-tree-hiccup relationhip-data-hiccup)
result (apply glue (sorted-map)
(forv [hid (all-hids)]
(let [parent-tag (grab :tag (hid->node hid))
kid-tags (forv [kid-hid (hid->kids hid)]
(let [kid-tag (grab :tag (hid->node kid-hid))]
kid-tag))]
{parent-tag kid-tags})))]
(is= (format-paths (find-paths root-hid [:A]))
[[{:tag :A}
[{:tag :B} [{:tag :C}] [{:tag :D}]]
[{:tag :E} [{:tag :F}]]]])
(is= result expected-result ))))))
API docs are here. The project README (in progress) is here. A video from the 2017 Clojure Conj is here.
You can see the above live code in the project repo.
I'm playing around with the graph examples from proto-repl-charts in Atom.
However, the vis.js-based graphs seems not to work on my setup.
The following code
(prc/graph "A Graph from a map" {:nodes [:a :b :c :d :e :f :g] :edges [[:d :b] [:b :c] [:d :c] [:e :f] [:f :g] [:d :a] [:f :c]]})
leads to a graph with only blocks instead of any text displayed. Similar behavior with the loom graphs.
Any clue on what could cause this?
I'm new to clojure and I've been staring at this for some time, I'm sure there's something basic I just don't see. I want to conj two sets, but they're nested, example:
(def foo {:b #{:test}})
(def bar {:a {:b #{:ab}} :c :d})
I tried:
=>(update-in bar [:a :b] conj (:b foo) )
{:a {:b #{#{:test} :ab}}, :c :d}
I guess that makes sense, but what I wanted was {:a {:b #{:test :ab}}, :c :d}
I just can't seem how to get either #{:test} out of the set to conj it, or to properly access :b as a set given the update-in syntax.
Any help is enormously appreciated.
You need to use into instead of conj:
(update-in bar [:a :b] into (:b foo))
;= {:a {:b #{:test :ab}}, :c :d}