How to successfully install ubergraph - clojure

First I have to say, I'm completely new to clojure, so forgive me if I'm missing something obvious.
I recently installed the clojure package on the atom text editor in order to create some graphs and tried to add ubergraph, an extension that makes weighted graphs possible, since these are not supported in the standard clojure package.
I followed the quickstart guide on ubergraphs github https://github.com/Engelberg/ubergraph and managed to complete the first step (adding ubergraph to leiningen dependencies). I downloaded the git repository and don't know how to carry on from here. Running the example code
(ns example.core
(:require [ubergraph.core :as uber]))
(def graph1
(uber/graph [:a :b] [:a :c] [:b :d]))
on the repl as described on github ends up with the following error:
CompilerException java.lang.NullPointerException, compiling:(ubergraph/core.clj:11:1)
The line that seems to cause the error in core.clj is:
(import-vars
[...])
I skipped over the vars since I don't think they're causing the problem.
Clojure runs on the correct version (1.9.0) and java 8 is installed. Help is appreciated, thanks in advance.

Based on your comment "Also, do I have to put the lib somewhere specific?", this seems to caused by a misunderstanding of how to install a library. You shouldn't be manually dealing with stuff like that; leiningen handles library installation for you.
Here's a quick guide that assumes you haven't created a project yet. If you have, skip to step 2.
Run lein new app you-project-name-here. This will create a barebones project with a project.clj and basic file structure. If you use an IDE like IntelliJ+Cursive, creating a new project will do this step automatically.
Go into your project.clj, and add [ubergraph "0.5.2"] to the :dependencies entry. As a minimal, reduced example, it should look something like:
(defproject example "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.10.0"]
[ubergraph "0.5.2"]]
:main example.core) ; The path to your core
Have your core as something like:
(ns example.core
(:require [ubergraph.core :as uber])
(:gen-class))
(def graph1
(uber/graph [:a :b] [:a :c] [:b :d]))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "The graph:" graph1))
Now run lein run. You should see it download the dependencies, then print something like this mess:
The graph: {:node-map {:a #ubergraph.core.NodeInfo{:out-edges {:b #{#ubergraph.core.UndirectedEdge{:id #uuid "0768ef5b-1507-4bb0-b3da-fc14a84d013d", :src :a, :dest :b, :mirror? false}}, :c #{#ubergraph.core.UndirectedEdge{:id #uuid "acddd770-52cc-4b1f-aec1-762861e70ee2", :src :a, :dest :c, :mirror? false}}}, :in-edges {:b #{#ubergraph.core.UndirectedEdge{:id #uuid "0768ef5b-1507-4bb0-b3da-fc14a84d013d", :src :b, :dest :a, :mirror? true}}, :c #{#ubergraph.core.UndirectedEdge{:id #uuid "acddd770-52cc-4b1f-aec1-762861e70ee2", :src :c, :dest :a, :mirror? true}}}, :out-degree 2, :in-degree 2}, :b #ubergraph.core.NodeInfo{:out-edges {:a #{#ubergraph.core.UndirectedEdge{:id #uuid "0768ef5b-1507-4bb0-b3da-fc14a84d013d", :src :b, :dest :a, :mirror? true}}, :d #{#ubergraph.core.UndirectedEdge{:id #uuid "ef931d4e-8143-4cd1-8a10-c3692c47072f", :src :b, :dest :d, :mirror? false}}}, :in-edges {:a #{#ubergraph.core.UndirectedEdge{:id #uuid "0768ef5b-1507-4bb0-b3da-fc14a84d013d", :src :a, :dest :b, :mirror? false}}, :d #{#ubergraph.core.UndirectedEdge{:id #uuid "ef931d4e-8143-4cd1-8a10-c3692c47072f", :src :d, :dest :b, :mirror? true}}}, :out-degree 2, :in-degree 2}, :c #ubergraph.core.NodeInfo{:out-edges {:a #{#ubergraph.core.UndirectedEdge{:id #uuid "acddd770-52cc-4b1f-aec1-762861e70ee2", :src :c, :dest :a, :mirror? true}}}, :in-edges {:a #{#ubergraph.core.UndirectedEdge{:id #uuid "acddd770-52cc-4b1f-aec1-762861e70ee2", :src :a, :dest :c, :mirror? false}}}, :out-degree 1, :in-degree 1}, :d #ubergraph.core.NodeInfo{:out-edges {:b #{#ubergraph.core.UndirectedEdge{:id #uuid "ef931d4e-8143-4cd1-8a10-c3692c47072f", :src :d, :dest :b, :mirror? true}}}, :in-edges {:b #{#ubergraph.core.UndirectedEdge{:id #uuid "ef931d4e-8143-4cd1-8a10-c3692c47072f", :src :b, :dest :d, :mirror? false}}}, :out-degree 1, :in-degree 1}}, :allow-parallel? false, :undirected? true, :attrs {}, :cached-hash #object[clojure.lang.Atom 0x16da1abc {:status :ready, :val -1}]}
I suspect the NPE was because you had installed ubergraph somehow, but didn't allow it to automatically resolve its dependencies. When it tried to run import-vals, one of the libraries it depends on wasn't found, and it threw a fit.

Related

Spec: partially overriding generators in a map spec

Assuming I have already defined a spec from which I'd like to generate test data:
(s/def :customer/id uuid?)
(s/def :customer/given-name string?)
(s/def :customer/surname string?)
(s/def :customer/age pos?)
(s/def ::customer
(s/keys
:req-un [:customer/id
:customer/given-name
:customer/surname
:customer/age]))
In generating test data, I'd like to override how ids are generated in order to ensure they're from a smaller pool to encourage collisions:
(defn customer-generator
[id-count]
(gen/let [id-pool (gen/not-empty (gen/vector (s/gen :customer/id) id-count))]
(assoc (s/gen ::customer) :id (gen/element id-pool))))
Is there a way I can simplify this by overriding the :customer/id generator in my test code and then just using (s/gen ::customer)? So, something like the following:
(with-generators [:customer/id (gen/not-empty (gen/vector (s/gen :customer/id) id-count)))]
(s/gen ::customer))
Officially, you can override generators for specs by passing an overrides map to s/gen (See the docstring for more details):
(s/def :customer/id uuid?)
(s/def :customer/given-name string?)
(s/def :customer/surname string?)
(s/def :customer/age nat-int?)
(s/def ::customer
(s/keys
:req-un [:customer/id
:customer/given-name
:customer/surname
:customer/age]))
(def fixed-customer-id (java.util.UUID/randomUUID))
fixed-customer-id
;=> #uuid "c73ff5ea-8702-4066-a31d-bc4cc7015811"
(gen/generate (s/gen ::customer {:customer/id #(s/gen #{fixed-customer-id})}))
;=> {:id #uuid "c73ff5ea-8702-4066-a31d-bc4cc7015811",
; :given-name "1042IKQhd",
; :surname "Uw0AzJzj",
; :age 104}
Alternatively, there is a library for such stuff named genman, which I developed before :)
Using it, you can also write as:
(require '[genman.core :as genman :refer [defgenerator]])
(def fixed-customer-id (java.util.UUID/randomUUID))
(genman/with-gen-group :test
(defgenerator :customer/id
(s/gen #{fixed-customer-id})))
(genman/with-gen-group :test
(gen/generate (genman/gen ::customer)))
Clojure spec uses test.check internally to generate sample values. Here is how test.check can be overridden. Whenever trying to write unit tests with a "fake" function, with-redefs is your friend:
(ns tst.demo.core
(:use tupelo.core tupelo.test)
(:require
[clojure.test.check.generators :as gen]
))
(def id-gen gen/uuid)
(dotest
(newline)
(spyx-pretty (take 3 (gen/sample-seq id-gen)))
(newline)
(with-redefs [id-gen (gen/choose 1 5)]
(spyx-pretty (take 33 (gen/sample-seq id-gen))))
(newline)
)
with result:
-----------------------------------
Clojure 1.10.3 Java 15.0.2
-----------------------------------
Testing tst.demo.core
(take 3 (gen/sample-seq id-gen)) =>
[#uuid "cbfea340-1346-429f-ba68-181e657acba5"
#uuid "7c119cf7-0842-4dd0-a23d-f95b6a68f808"
#uuid "ca35cb86-1385-46ad-8fc2-e05cf7a1220a"]
(take 33 (gen/sample-seq id-gen)) =>
[5 4 3 3 2 2 3 1 2 1 4 1 2 2 4 3 5 2 3 5 3 2 3 2 3 5 5 5 5 1 3 2 2]
Example created
using my favorite template project.
Update
Unfortunately, the above technique does not work for Clojure Spec since (s/def ...) uses a global registery of Spec definitions, and is therefore immune to with-redefs. However, we can overcome this definition by simply redefining the desired spec in the unit test namespace like:
(ns tst.demo.core
(:use tupelo.core tupelo.test)
(:require
[clojure.spec.alpha :as s]
[clojure.spec.gen.alpha :as gen]
))
(s/def :app/id (s/int-in 9 99))
(s/def :app/name string?)
(s/def :app/cust (s/keys :req-un [:app/id :app/name]))
(dotest
(newline)
(spyx-pretty (gen/sample (s/gen :app/cust)))
(newline)
(s/def :app/id (s/int-in 2 5)) ; overwrite the definition of :app/id for testing
(spyx-pretty (gen/sample (s/gen :app/cust)))
(newline))
with result
-----------------------------------
Clojure 1.10.3 Java 15.0.2
-----------------------------------
Testing tst.demo.core
(gen/sample (s/gen :app/cust)) =>
[{:id 10, :name ""}
{:id 9, :name "n"}
{:id 10, :name "fh"}
{:id 9, :name "aI"}
{:id 11, :name "8v5F"}
{:id 10, :name ""}
{:id 10, :name "7"}
{:id 10, :name "3m6Wi"}
{:id 13, :name "OG2Qzfqe"}
{:id 10, :name ""}]
(gen/sample (s/gen :app/cust)) =>
[{:id 3, :name ""}
{:id 3, :name ""}
{:id 2, :name "5e"}
{:id 3, :name ""}
{:id 2, :name "y01C"}
{:id 3, :name "l2"}
{:id 3, :name "c"}
{:id 3, :name "pF"}
{:id 4, :name "0yrxyJ7l"}
{:id 4, :name "40"}]
So, it's a little ugly, but the redefinition of :app/id does the trick, and it only takes effect during unit test runs, leaving the main application unaffected.
user> (def ^:dynamic *idgen* (s/gen uuid?))
#'user/*idgen*
user> (s/def :customer/id (s/with-gen uuid? (fn [] ##'*idgen*)))
:customer/id
user> (s/def :customer/age pos-int?)
:customer/age
user> (s/def ::customer (s/keys :req-un [:customer/id :customer/age]))
:user/customer
user> (gen/sample (s/gen ::customer))
({:id #uuid "d18896f1-6199-42bf-9be3-3d0652583902", :age 1}
{:id #uuid "b6209798-4ffa-4e20-9a76-b3a799a31ec6", :age 2}
{:id #uuid "6f9c6400-8d79-417c-bc62-6b4557f7d162", :age 1}
{:id #uuid "47b71396-1b5f-4cf4-bd80-edf4792300c8", :age 2}
{:id #uuid "808692b9-0698-4fb8-a0c5-3918e42e8f37", :age 2}
{:id #uuid "ba663f0a-7c99-4967-a2df-3ec6cb04f514", :age 1}
{:id #uuid "8521b611-c38c-4ea9-ae84-35c8a2d2ff2f", :age 4}
{:id #uuid "c559d48d-4c50-438f-846c-780cdcdf39d5", :age 3}
{:id #uuid "03c2c114-03a0-4709-b9dc-6d326a17b69d", :age 40}
{:id #uuid "14715a50-81c5-48e4-bffe-e194631bb64b", :age 4})
user> (binding [*idgen* (let [idpool (gen/sample (s/gen :customer/id) 5)] (gen/elements idpool))] (gen/sample (s/gen ::customer)))
({:id #uuid "3e64131d-e7ad-4450-993d-fa651339df1c", :age 2}
{:id #uuid "575b2bef-956d-4c42-bdfa-982c7756a33c", :age 1}
{:id #uuid "575b2bef-956d-4c42-bdfa-982c7756a33c", :age 1}
{:id #uuid "3e64131d-e7ad-4450-993d-fa651339df1c", :age 1}
{:id #uuid "1a2eafed-8242-4229-b432-99edb361569d", :age 3}
{:id #uuid "1a2eafed-8242-4229-b432-99edb361569d", :age 1}
{:id #uuid "05bd521a-26f9-46e0-8b26-f798e0bf0452", :age 3}
{:id #uuid "575b2bef-956d-4c42-bdfa-982c7756a33c", :age 19}
{:id #uuid "31b80714-7ae0-40a0-b932-f7b5f078f2ad", :age 2}
{:id #uuid "05bd521a-26f9-46e0-8b26-f798e0bf0452", :age 5})
user>
A little clumsier than what you wanted, but maybe this is adequate.
You are probably better off using binding rather than with-redefs since binding modifies thread-local bindings, whereas with-redefs changes the root binding.
Since this is for generating bad test data, I'd consider avoiding the use of dynamic vars and binding altogether and just use a different spec that is only local to the test env.

How to filter map content by path

I want to select paths of a deeply nested map to keep.
For example:
{:a 1
:b {:c [{:d 1 :e 1}
{:d 2 :e 2}]
:f 1}
:g {:h {:i 4 :j [1 2 3]}}}
I want to select by paths, like so:
(select-paths m [[:a]
[:b :c :e]
[:b :f]
[:g :h :i]])
This would return
{:a 1
:b {:c [{:e 1}
{:e 2}]
:f 1}
:g {:h {:i 4}}}
Essentially the same as Elasticsearch's fields parameter. The format of the paths argument can be something else, this is just the first idea.
I tried two different solutions
Go through the entire map and checking if the full path of the current element is in the given paths. I can't figure out how to handle lists of maps so that they are kept as lists of maps.
Creating select-keys statements from the given paths but again I run into problems with lists of maps - and especially trying to resolve paths of varying depths that have some common depth.
I looked at spectre but I didn't see anything that would do this. Any map or postwalk based solution I come up with turns into something incredibly convoluted at some point. I must be thinking about this the wrong way.
If there's a way to do this with raw json, that would be fine as well. Or even a Java solution.
There is no simple way to accomplish your goal. The automatic processing implied for the sequence under [:b :c] is also problematic.
You can get partway there using the Tupelo Forest library. See the Lightning Talk video from Clojure/Conj 2017.
I did some additional work in data destructuring that you may find useful building the tupelo.core/destruct macro (see examples here). You could follow a similar outline to build a recursive solution to your specific problem.
A related project is Meander. I have worked on my own version which is like a generalized version of tupelo.core/destruct. Given data like this
(def skynet-widgets [{:basic-info {:producer-code "Cyberdyne"}
:widgets [{:widget-code "Model-101"
:widget-type-code "t800"}
{:widget-code "Model-102"
:widget-type-code "t800"}
{:widget-code "Model-201"
:widget-type-code "t1000"}]
:widget-types [{:widget-type-code "t800"
:description "Resistance Infiltrator"}
{:widget-type-code "t1000"
:description "Mimetic polyalloy"}]}
{:basic-info {:producer-code "ACME"}
:widgets [{:widget-code "Dynamite"
:widget-type-code "c40"}]
:widget-types [{:widget-type-code "c40"
:description "Boom!"}]}])
You can search and extract data using a template like this:
(let [root-eid (td/add-entity-edn skynet-widgets)
results (td/match
[{:basic-info {:producer-code ?}
:widgets [{:widget-code ?
:widget-type-code wtc}]
:widget-types [{:widget-type-code wtc
:description ?}]}])]
(is= results
[{:description "Resistance Infiltrator" :widget-code "Model-101" :producer-code "Cyberdyne" :wtc "t800"}
{:description "Resistance Infiltrator" :widget-code "Model-102" :producer-code "Cyberdyne" :wtc "t800"}
{:description "Mimetic polyalloy" :widget-code "Model-201" :producer-code "Cyberdyne" :wtc "t1000"}
{:description "Boom!" :widget-code "Dynamite" :producer-code "ACME" :wtc "c40"}])))
This code is working (see here) but it needs more polish. You could use it as a guide to building a generalized select-paths function.
Can you add any details on how this problem arose or the specific context? That may point to ideas for an alternate solution.
One way of solving this problem would be to generate a set of all subpaths that you accept and then write a recursive function that traverses the data structure and keeps track of the path to the current node. The code that accomplishes that does not need to be very long:
(defn select-paths-from-set [current-path path-set data]
(cond
(map? data) (into {}
(remove nil?)
(for [[k v] data]
(let [p (conj current-path k)]
(if (contains? path-set p)
[k (select-paths-from-set p path-set v)]))))
(sequential? data) (mapv (partial select-paths-from-set current-path path-set) data)
:default data))
(defn select-paths [data paths]
(select-paths-from-set []
(into #{}
(mapcat #(take-while seq (iterate butlast %)))
paths)
data))
(select-paths {:a 1
:b {:c [{:d 1 :e 1}
{:d 2 :e 2}]
:f 1}
:g {:h {:i 4 :j [1 2 3]}}}
[[:a]
[:b :c :e]
[:b :f]
[:g :h :i]])
;; => {:a 1, :b {:c [{:e 1} {:e 2}], :f 1}, :g {:h {:i 4}}}

Clojure parse nested vectors

I am looking to transform a clojure tree structure into a map with its dependencies
For example, an input like:
[{:value "A"}
[{:value "B"}
[{:value "C"} {:value "D"}]
[{:value "E"} [{:value "F"}]]]]
equivalent to:
:A
:B
:C
:D
:E
:F
output:
{:A [:B :E] :B [:C :D] :C [] :D [] :E [:F] :F}
I have taken a look at tree-seq and zippers but can't figure it out!
Here's a way to build up the desired map while using a zipper to traverse the tree. First let's simplify the input tree to match your output format (maps of :value strings → keywords):
(def tree
[{:value "A"}
[{:value "B"} [{:value "C"} {:value "D"}]
{:value "E"} [{:value "F"}]]])
(def simpler-tree
(clojure.walk/postwalk
#(if (map? %) (keyword (:value %)) %)
tree))
;; [:A [:B [:C :D] :E [:F]]]
Then you can traverse the tree with loop/recur and clojure.zip/next, using two loop bindings: the current position in tree, and the map being built.
(loop [loc (z/vector-zip simpler-tree)
deps {}]
(if (z/end? loc)
deps ;; return map when end is reached
(recur
(z/next loc) ;; advance through tree
(if (z/branch? loc)
;; for (non-root) branches, add top-level key with direct descendants
(if-let [parent (some-> (z/prev loc) z/node)]
(assoc deps parent (filterv keyword? (z/children loc)))
deps)
;; otherwise add top-level key with no direct descendants
(assoc deps (z/node loc) [])))))
=> {:A [:B :E], :B [:C :D], :C [], :D [], :E [:F], :F []}
This is easy to do using the tupelo.forest library. I reformatted your source data to make it fit into the Hiccup syntax:
(dotest
(let [relationhip-data-hiccup [:A
[:B
[:C]
[:D]]
[:E
[:F]]]
expected-result {:A [:B :E]
:B [:C :D]
:C []
:D []
:E [:F]
:F []} ]
(with-debug-hid
(with-forest (new-forest)
(let [root-hid (tf/add-tree-hiccup relationhip-data-hiccup)
result (apply glue (sorted-map)
(forv [hid (all-hids)]
(let [parent-tag (grab :tag (hid->node hid))
kid-tags (forv [kid-hid (hid->kids hid)]
(let [kid-tag (grab :tag (hid->node kid-hid))]
kid-tag))]
{parent-tag kid-tags})))]
(is= (format-paths (find-paths root-hid [:A]))
[[{:tag :A}
[{:tag :B} [{:tag :C}] [{:tag :D}]]
[{:tag :E} [{:tag :F}]]]])
(is= result expected-result ))))))
API docs are here. The project README (in progress) is here. A video from the 2017 Clojure Conj is here.
You can see the above live code in the project repo.

How can I get vis.js charts to be rendered in Atom Editor correctly?

I'm playing around with the graph examples from proto-repl-charts in Atom.
However, the vis.js-based graphs seems not to work on my setup.
The following code
(prc/graph "A Graph from a map" {:nodes [:a :b :c :d :e :f :g] :edges [[:d :b] [:b :c] [:d :c] [:e :f] [:f :g] [:d :a] [:f :c]]})
leads to a graph with only blocks instead of any text displayed. Similar behavior with the loom graphs.
Any clue on what could cause this?

Update-in nested map

I'm new to clojure and I've been staring at this for some time, I'm sure there's something basic I just don't see. I want to conj two sets, but they're nested, example:
(def foo {:b #{:test}})
(def bar {:a {:b #{:ab}} :c :d})
I tried:
=>(update-in bar [:a :b] conj (:b foo) )
{:a {:b #{#{:test} :ab}}, :c :d}
I guess that makes sense, but what I wanted was {:a {:b #{:test :ab}}, :c :d}
I just can't seem how to get either #{:test} out of the set to conj it, or to properly access :b as a set given the update-in syntax.
Any help is enormously appreciated.
You need to use into instead of conj:
(update-in bar [:a :b] into (:b foo))
;= {:a {:b #{:test :ab}}, :c :d}