This question already has answers here:
Why does Datomic yield the same temporary ID twice in a row when iterating?
(2 answers)
Closed 6 years ago.
I am building Datomic transaction with this function, which I am then mapping over a list of input keywords:
(defn build-enum-transaction [inp]
(cond
(.contains (namespace (first inp)) "region")
[:db/add #db/id[:db.part/region] :db/ident (first inp)]
(.contains (namespace (first inp)) "sector")
[:db/add #db/id[:db.part/sector] :db/ident (first inp)]
(.contains (namespace (first inp)) "specialism")
[:db/add #db/id[:db.part/specialism] :db/ident (first inp)]))
(defn build-all-enum-transactions [inp]
(vec (map build-enum-transaction inp)))
The input data to the build-all-enum-transactions is:
([:region/EU]
[:region/UK]
[:region/NAFTA]
[:sector/NON-CYCLICALS]
[:sector/FINANCIALS]
[:specialism/INSURANCE]
[:specialism/VAT])
I get the following result:
[[:db/add #db/id[:db.part/region -1000289] :db/ident :region/EU]
[:db/add #db/id[:db.part/region -1000289] :db/ident :region/UK]
[:db/add #db/id[:db.part/region -1000289] :db/ident :region/NAFTA]
[:db/add #db/id[:db.part/sector -1000290] :db/ident :sector/NON-CYCLICALS]
[:db/add #db/id[:db.part/sector -1000290] :db/ident :sector/FINANCIALS]
[:db/add #db/id[:db.part/specialism -1000291] :db/ident :specialism/INSURANCE]
[:db/add #db/id[:db.part/specialism -1000291] :db/ident :specialism/VAT]]
As you can see, the :db.part/ should yield an incremental number for each however it only does so for each of the 'cond' clauses. Why is this? It appears as though the 'cond' is closing over the value and re-using it. Thanks.
You should use d/tempid to create a tempid at runtime. #db/id is a reader macro that will expand to a tempid when the program is read, i. e. compile time.
Related
Setup Consider the following DataScript database of films and cast, with data stolen from learndatalogtoday.org: the following code can be executed in a JVM/Clojure REPL or a ClojureScript REPL, as long as project.clj contains [datascript "0.15.0"] as a dependency.
(ns user
(:require [datascript.core :as d]))
(def data
[["First Blood" ["Sylvester Stallone" "Brian Dennehy" "Richard Crenna"]]
["Terminator 2: Judgment Day" ["Linda Hamilton" "Arnold Schwarzenegger" "Edward Furlong" "Robert Patrick"]]
["The Terminator" ["Arnold Schwarzenegger" "Linda Hamilton" "Michael Biehn"]]
["Rambo III" ["Richard Crenna" "Sylvester Stallone" "Marc de Jonge"]]
["Predator 2" ["Gary Busey" "Danny Glover" "Ruben Blades"]]
["Lethal Weapon" ["Gary Busey" "Mel Gibson" "Danny Glover"]]
["Lethal Weapon 2" ["Mel Gibson" "Joe Pesci" "Danny Glover"]]
["Lethal Weapon 3" ["Joe Pesci" "Danny Glover" "Mel Gibson"]]
["Alien" ["Tom Skerritt" "Veronica Cartwright" "Sigourney Weaver"]]
["Aliens" ["Carrie Henn" "Sigourney Weaver" "Michael Biehn"]]
["Die Hard" ["Alan Rickman" "Bruce Willis" "Alexander Godunov"]]
["Rambo: First Blood Part II" ["Richard Crenna" "Sylvester Stallone" "Charles Napier"]]
["Commando" ["Arnold Schwarzenegger" "Alyssa Milano" "Rae Dawn Chong"]]
["Mad Max 2" ["Bruce Spence" "Mel Gibson" "Michael Preston"]]
["Mad Max" ["Joanne Samuel" "Steve Bisley" "Mel Gibson"]]
["RoboCop" ["Nancy Allen" "Peter Weller" "Ronny Cox"]]
["Braveheart" ["Sophie Marceau" "Mel Gibson"]]
["Mad Max Beyond Thunderdome" ["Mel Gibson" "Tina Turner"]]
["Predator" ["Carl Weathers" "Elpidia Carrillo" "Arnold Schwarzenegger"]]
["Terminator 3: Rise of the Machines" ["Nick Stahl" "Arnold Schwarzenegger" "Claire Danes"]]])
(def conn (d/create-conn {:film/cast {:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many}
:film/name {:db/unique :db.unique/identity
:db/cardinality :db.cardinality/one}
:actor/name {:db/unique :db.unique/identity
:db/cardinality :db.cardinality/one}}))
(def all-datoms (mapcat (fn [[film actors]]
(into [{:film/name film}]
(map #(hash-map :actor/name %) actors)))
data))
(def all-relations (mapv (fn [[film actors]]
{:db/id [:film/name film]
:film/cast (mapv #(vector :actor/name %) actors)}) data))
(d/transact! conn all-datoms)
(d/transact! conn all-relations)
Description In a nutshell, there are two kinds of entities in this database—films and actors (word intended to be ungendered)—and three kinds of datoms:
film entity: :film/name (a unique string)
film entity: :film/cast (multiple refs)
actor entity: :actor/name (unique string)
Question I would like to construct a query which asks: which films have these N actors, and these N actors alone, appeared as the sole stars, for N>=2?
E.g., RoboCop starred Nancy Allen, Peter Weller, Ronny Cox, but no film starred solely the first two of these, Allen and Weller. Therefore, I would expect the following query to produce the empty set:
(d/q '[:find ?film-name
:where
[?film :film/name ?film-name]
[?film :film/cast ?actor-1]
[?film :film/cast ?actor-2]
[?actor-1 :actor/name "Nancy Allen"]
[?actor-2 :actor/name "Peter Weller"]]
#conn)
; => #{["RoboCop"]}
However, the query is flawed because I don't know how to express that any matches should exclude any actors who are not Allen or Weller—again, I want to find the movies where only Allen and Weller have collaborated without any other actors, so I want to adapt the above query to produce the empty set. How can I adjust this query to enforce this requirement?
Because DataScript doesn't have negation (as of May 2016), I don't believe that's possible with one static query in 'pure' Datalog.
My way to go would be:
build the query programmatically to add the N clauses that state that the cast must contain the N actors
Add a predicate function which, given a movie, the database, and the set of actors ids, uses the EAVT index to find if each movie has an actor that is not in the set.
Here's a basic implementation
(defn only-those-actors? [db movie actors]
(->> (datoms db :eavt movie :film/cast) seq
(every? (fn [[_ _ actor]]
(contains? actors actor)))
))
(defn find-movies-with-exact-cast [db actors-names]
(let [actors (set (d/q '[:find [?actor ...] :in $ [?name ...] ?only-those-actors :where
[?actor :actor/name ?name]]
db actors-names))
query {:find '[[?movie ...]]
:in '[$ ?actors ?db]
:where
(concat
(for [actor actors]
['?movie :film/cast actor])
[['(only-those-actors? ?db ?movie ?actors)]])}]
(d/q query db actors db only-those-actors?)))
You can use predicate fun and d/entity together for filtering datoms by :film/cast field of an entity. This approach looks much more straightforward until Datascript doesn't support negation (not operator and so on).
Look at the row (= a (:age (d/entity db e)) in the test case of the Datascript here
[{:db/id 1 :name "Ivan" :age 10}
{:db/id 2 :name "Ivan" :age 20}
{:db/id 3 :name "Oleg" :age 10}
{:db/id 4 :name "Oleg" :age 20}]
...
(let [pred (fn [db e a]
(= a (:age (d/entity db e))))]
(is (= (q/q '[:find ?e
:in $ ?pred
:where [?e :age ?a]
[(?pred $ ?e 10)]]
db pred)
#{[1] [3]})))))
In your case, the predicate body could look something like this
(clojure.set/subset? actors (:film/cast (d/entity db e))
In regards to performance, the d/entity call is fast because it is a lookup by index.
In the mbrainz sample data, the :artist/type is an enum. Is it possible to pull the value of the enum out of :db/ident and associate it as the value of the :artist/type key using pull syntax?
This is as close as I could get:
[:find (pull ?e [:artist/name {:artist/type [:db/ident]}])
:where
[?e :artist/name "Ray Charles"]
]
;;=> [[{:artist/name "Ray Charles", :artist/type {:db/ident :artist.type/person}}]]
Is it possible to use pull syntax to reshape the result into something like this?
;;=> [[{:artist/name "Ray Charles", :artist/type :artist.type/person}]]
I don't think you can do it using the Pull API the way you are seeking. You may find that it is easier to use the Tupelo Datomic library:
(require '[tupelo.datomic :as td]
'[tupelo.core :refer [spyx]] )
(let [x1 (td/query-scalar :let [$ db-val]
:find [ ?e ]
:where [ [?e :artist/name "Ray Charles"] ] )
x2 (td/entity-map db-val x1)
]
(spyx x1)
(spyx x2)
)
which gives the result:
x1 => 17592186049074
x2 => {:artist/sortName "Charles, Ray", :artist/name "Ray Charles",
:artist/type :artist.type/person, :artist/country :country/US,
:artist/gid #uuid "2ce02909-598b-44ef-a456-151ba0a3bd70",
:artist/startDay 23, :artist/endDay 10, :artist/startYear 1930,
:artist/endMonth 6, :artist/endYear 2004, :artist/startMonth 9,
:artist/gender :artist.gender/male}
So :artist/type is already converted into the :db/ident value and you can just pull it out of the map.
You can use specter on the result that the pull expression returns:
(->> pull-result
(sp/transform (sp/walker :db/ident) :db/ident))
The value of key :db/ident is extracted for every map that has that key.
Was quite easy to do with postwalk
for any pulled :db/ident you can transform with this function
(defn flatten-ident [coll]
(clojure.walk/postwalk
(fn [item] (get item :db/ident item)) coll))
When trying to persist a list of node entities with a :threshold attribute defined thus in the schema:
{:db/id #db/id[:db.part/db]
:db/ident :node/threshold
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one
:db/fulltext false
:db/doc "Threshold"
:db.install/_attribute :db.part/db}
i get the following error:
CompilerException java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException:
:db.error/wrong-type-for-attribute Value 90 is not a valid :int
for attribute :node/threshold
I use the following code:
(defn store-tree [tree]
#(d/transact dbconn/conn (into [] (vals tree))))
(store-tree parsed-tree-with-refs)
where tree is a map of node names to nodes.
Curiously enough, i took the EDN for the specific entity with :node threshold 90 from the REPL and manually transact'ed it, and it worked without any problems. I used this code:
#(d/transact dbconn/conn [{:db/id (d/tempid :db.part/user),
:node/threshold 90, :node/location "US"}])
Can someone please help?
Thanks,
Vitaliy.
I'm getting into datomic and still don't grok it. How do I build a transaction that has references to a variable number of entities?
For example this creates a transaction with a child entity and a family entity with a child attribute that references the new child entity:
(defn insert-child [id child]
{:db/id #db/id id
:child/first-name (:first-name child)
:child/middle-name (:middle-name child)
:child/last-name (:last-name child)
:child/date-of-birth {:date-of-birth child}})
(defn insert-family [id]
(let [child-id #db/id[:db.part/user]]
(vector
(insert-child child-id
{:first-name "Richard"
:middle-name "M"
:last-name "Stallman"})
{:db/id id
:family/child child-id})))
(insert-family #db/id[:db.part/user])
=> [{:db/id #db/id[:db.part/user -1000012],
:child/first-name "Richard",
:child/middle-name "M",
:child/last-name "Stallman",
:child/date-of-birth nil}
{:db/id #db/id[:db.part/user -1000013],
:family/child #db/id[:db.part/user -1000012]}]
Notice I used let for child-id. I'm not sure how to write this such that I can map over insert-child while having a family entity that references each one.
I thought about using iterate over #db/id[:db.part/user] and the number of children then mapping over both the result of iterate and a vector of children. Seems kind of convoluted and #db/id[:db.part/user] isn't a function to iterate over to begin with.
Instead of using the macro form #db/id[:db.part/user] which is meant for EDN files and data literals, you should use d/tempid.
You could do something like this (using simplified child entities):
(ns family-tx
(:require [datomic.api :refer [q db] :as d]))
(def uri "datomic:mem://testfamily")
(d/delete-database uri)
(d/create-database uri)
(def conn (d/connect uri))
(def schema [
{:db/id (d/tempid :db.part/db)
:db/ident :first-name
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:db.install/_attribute :db.part/db}
{:db/id (d/tempid :db.part/db)
:db/ident :last-name
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:db.install/_attribute :db.part/db}
{:db/id (d/tempid :db.part/db)
:db/ident :family/child
:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many
:db.install/_attribute :db.part/db}
])
#(d/transact conn schema)
(defn make-family-tx [kids]
(let [kids-tx (map #(into {:db/id (d/tempid :db.part/user)} %) kids)
kids-id (map :db/id kids-tx)]
(conj kids-tx {:db/id (d/tempid :db.part/user)
:family/child kids-id})))
(def kids [{:first-name "Billy" :last-name "Bob"}
{:first-name "Jim" :last-name "Beau"}
{:first-name "Junior" :last-name "Bacon"}])
#(d/transact conn (make-family-tx kids))
There are a few strategies for this discussed in the Transactions docs as well (see the "Identifying Entities" section).
I am getting a FileNotFoundException when using a database function that requires a namespace. I only get the error when using the persistent datomic free database but not when I'm using the memory database.
(ns test.core
(:use [datomic.api :only [q db] :as d]))
(def uris ["datomic:mem://test"
"datomic:free://localhost:4334/test"])
(map
d/delete-database uris)
(map
d/create-database uris)
(def conns (map d/connect uris))
(defn test-entity []
[{:db/id #db/id[:db.part/db]
:test/test "hello"}])
(def db-function
#db/fn {:lang :clojure
:params [database]
:requires [[test.core :as c]]
:code (c/test-entity)})
(map
#(d/transact % [{:db/id #db/id[:db.part/user]
:db/ident :db-function
:db/fn db-function}])
conns)
(map
#(d/transact % [{:db/id #db/id[:db.part/db]
:db/ident :test/test
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:db.install/_attribute :db.part/db}])
conns)
(comment
(db-function nil)
(d/transact (first conns) [[:db-function]])
(d/transact (second conns) [[:db-function]]))
When you evaluate the first and second line in the comment, it's fine but when you evaluate the third line you get an exception.
Do I need to configure something in datomic so that it can "see" my project?
When you use an in memory database, the transactor runs in the same JVM instance as the peer, hence with the same classpath. But with the free database, the transactor is running in its own JVM instance, and is not aware about namespaces in the peers.
You can add jars to the transactor classpath by putting them in the lib/ folder.