I have two Datomic clients open in two REPLs connected to the same database in Datomic Cloud. After transacting data in REPL 1, the connection in REPL 2 is not aware that time has advanced until a query is executed.
REPL 1
I transact some data bringing the {:db-after {:t 768}}
(d/transact conn {:tx-data [{:person/first-name "Alice"}]})
=>
{:db-before {:database-id "<some-squuid>",
:db-name "datomic-test",
:t 767,
:next-t 768,
:history false,
:type :datomic.client/db},
:db-after {:database-id "<some-squuid>",
:db-name "datomic-test",
:t 768,
:next-t 769,
:history false,
:type :datomic.client/db},
:tx-data [#datom[13194139534080 50 #inst"2018-05-15T09:18:22.565-00:00" 13194139534080 true]
#datom[43976067064531301 73 "Alice" 13194139534080 true]],
:tempids {}}
(d/db conn)
=>
{:t 768,
:next-t 769,
:db-name "datomic-test",
:database-id "<some-squuid>",
:type :datomic.client/db}
REPL 2
I get the latest known database value, but it's still at {:t 767}
(d/db conn)
=>
{:t 767,
:next-t 768,
:db-name "datomic-test",
:database-id "<some-squuid>",
:type :datomic.client/db}
Repeatedly calling d/db will forever return the database at :t 767, one transaction behind REPL 1. It's locked there no matter how long I wait.
... Until I run a query:
(d/q '[:find (pull ?person [*])
:in $ ?person-name
:where
[?person :person/first-name ?person-name]]
(d/db conn)
"Alice")
=> []
Querying for the recently transacted "Alice" returns an empty collection because the database value returned from d/db in REPL 2 is old. However, the action of querying advances the database returned from d/db. Now it's "caught up":
(d/db conn)
=>
{:t 768,
:next-t 769,
:db-name "datomic-test",
:database-id "<some-squuid>",
:type :datomic.client/db}
And re-running the same query returns the data transacted from REPL 1:
(d/q '[:find (pull ?person [*])
:in $ ?person-name
:where
[?person :person/first-name ?person-name]]
(d/db conn)
"Alice")
=> [[{:db/id 43976067064531301, :person/first-name "Alice"}]]
Datomic's peer API has a sync function which seems to solve this problem for on-prem, however the Datomic Client API doesn't appear to have anything similar.
Given that REPL 1 and REPL 2 represent two independent systems, how can I tell them to always use the latest version of the database without executing a bogus query?
Related
Tested on datascript 1.3.0
datoms:
[{:db/id -1 :name "Oliver Smith" :hobbies ["reading" "sports" "music"]}]
tried to run the query below to find who like sports, but the empty set returned.
'[:find ?name
:where
[?p :name ?name]
[?p :hobbies ?hobbies]
[(some #{"sports"} ?hobbies)]]
How to formulate the query correctly to get the expected result below?
#{[Oliver Smith]}
We have to explicitly define the schema with cardinality/many against the attribute of multiple values to solve the problem since schemaless doesn't work here.
(require '[datascript.core :as d])
(def schema {:hobbies {:db/cardinality db.cardinality/many}})
(def conn (d/create-conn schema))
(def datoms [{:db/id -1 :name "Oliver Smith" :hobbies ["reading" "sports" "music"]}])
(d/transact! conn datoms)
(def query '[:find ?name :where [?p :name ?name] [?p :hobbies "sports"]])
(-> (d/q query #conn) println)
I've got this simple schema and data:
(def product-offer-schema
[{:db/ident :product-offer/product
:db/valueType :db.type/ref
:db/cardinality :db.cardinality/one}
{:db/ident :product-offer/vendor
:db/valueType :db.type/ref
:db/cardinality :db.cardinality/one}
{:db/ident :product-offer/price
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one}
{:db/ident :product-offer/stock-quantity
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one}
])
(d/transact conn product-offer-schema)
(d/transact conn
[{:db/ident :vendor/Alice}
{:db/ident :vendor/Bob}
{:db/ident :product/BunnyBoots}
{:db/ident :product/Gum}
])
(d/transact conn
[{:product-offer/vendor :vendor/Alice
:product-offer/product :product/BunnyBoots
:product-offer/price 9981 ;; $99.81
:product-offer/stock-quantity 78
}
{:product-offer/vendor :vendor/Alice
:product-offer/product :product/Gum
:product-offer/price 200 ;; $2.00
:product-offer/stock-quantity 500
}
{:product-offer/vendor :vendor/Bob
:product-offer/product :product/BunnyBoots
:product-offer/price 9000 ;; $90.00
:product-offer/stock-quantity 15
}
])
When I retrieve the cheapest bunny boots, only retrieving the price, I get the expected result (9000):
(def cheapest-boots-q '[:find (min ?p) .
:where
[?e :product-offer/product :product/BunnyBoots]
[?e :product-offer/price ?p]
])
(d/q cheapest-boots-q db)
;; => 9000
However, when I want to get the entity ID along with the price, it gives me the higher-priced boots:
(def db (d/db conn))
(def cheapest-boots-q '[:find [?e (min ?p)]
:where
[?e :product-offer/product :product/BunnyBoots]
[?e :product-offer/price ?p]
])
(d/q cheapest-boots-q db)
;; => [17592186045423 9981]
I tried adding :with but that gives me an error:
(def cheapest-boots-q '[:find [?e (min ?p)]
:with ?e
:where
[?e :product-offer/product :product/BunnyBoots]
[?e :product-offer/price ?p]
])
(d/q cheapest-boots-q db)
;; => => Execution error (ArrayIndexOutOfBoundsException) at datomic.datalog/fn$project (datalog.clj:503).
What am I doing wrong?
As a commenter kind of pointed out, ?e isn't bound in any way to the (min ?p) expression, so it's not defined what you'll get there, beyond a product entity id of some sort.
What you actually want to do is unify those values somehow as part of the query, and not perform aggregation on the results, for example:
(d/q '[:find [?e ?p]
:where
[?e :product-offer/product :product/BunnyBoots]
[?e :product-offer/price ?p]
[(min ?p)]]
db)
You can see that the min clause is part of the query, and as such will take part in the unification on the result, giving you what you want.
Setup Consider the following DataScript database of films and cast, with data stolen from learndatalogtoday.org: the following code can be executed in a JVM/Clojure REPL or a ClojureScript REPL, as long as project.clj contains [datascript "0.15.0"] as a dependency.
(ns user
(:require [datascript.core :as d]))
(def data
[["First Blood" ["Sylvester Stallone" "Brian Dennehy" "Richard Crenna"]]
["Terminator 2: Judgment Day" ["Linda Hamilton" "Arnold Schwarzenegger" "Edward Furlong" "Robert Patrick"]]
["The Terminator" ["Arnold Schwarzenegger" "Linda Hamilton" "Michael Biehn"]]
["Rambo III" ["Richard Crenna" "Sylvester Stallone" "Marc de Jonge"]]
["Predator 2" ["Gary Busey" "Danny Glover" "Ruben Blades"]]
["Lethal Weapon" ["Gary Busey" "Mel Gibson" "Danny Glover"]]
["Lethal Weapon 2" ["Mel Gibson" "Joe Pesci" "Danny Glover"]]
["Lethal Weapon 3" ["Joe Pesci" "Danny Glover" "Mel Gibson"]]
["Alien" ["Tom Skerritt" "Veronica Cartwright" "Sigourney Weaver"]]
["Aliens" ["Carrie Henn" "Sigourney Weaver" "Michael Biehn"]]
["Die Hard" ["Alan Rickman" "Bruce Willis" "Alexander Godunov"]]
["Rambo: First Blood Part II" ["Richard Crenna" "Sylvester Stallone" "Charles Napier"]]
["Commando" ["Arnold Schwarzenegger" "Alyssa Milano" "Rae Dawn Chong"]]
["Mad Max 2" ["Bruce Spence" "Mel Gibson" "Michael Preston"]]
["Mad Max" ["Joanne Samuel" "Steve Bisley" "Mel Gibson"]]
["RoboCop" ["Nancy Allen" "Peter Weller" "Ronny Cox"]]
["Braveheart" ["Sophie Marceau" "Mel Gibson"]]
["Mad Max Beyond Thunderdome" ["Mel Gibson" "Tina Turner"]]
["Predator" ["Carl Weathers" "Elpidia Carrillo" "Arnold Schwarzenegger"]]
["Terminator 3: Rise of the Machines" ["Nick Stahl" "Arnold Schwarzenegger" "Claire Danes"]]])
(def conn (d/create-conn {:film/cast {:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many}
:film/name {:db/unique :db.unique/identity
:db/cardinality :db.cardinality/one}
:actor/name {:db/unique :db.unique/identity
:db/cardinality :db.cardinality/one}}))
(def all-datoms (mapcat (fn [[film actors]]
(into [{:film/name film}]
(map #(hash-map :actor/name %) actors)))
data))
(def all-relations (mapv (fn [[film actors]]
{:db/id [:film/name film]
:film/cast (mapv #(vector :actor/name %) actors)}) data))
(d/transact! conn all-datoms)
(d/transact! conn all-relations)
Description In a nutshell, there are two kinds of entities in this database—films and actors (word intended to be ungendered)—and three kinds of datoms:
film entity: :film/name (a unique string)
film entity: :film/cast (multiple refs)
actor entity: :actor/name (unique string)
Question I would like to construct a query which asks: which films have these N actors, and these N actors alone, appeared as the sole stars, for N>=2?
E.g., RoboCop starred Nancy Allen, Peter Weller, Ronny Cox, but no film starred solely the first two of these, Allen and Weller. Therefore, I would expect the following query to produce the empty set:
(d/q '[:find ?film-name
:where
[?film :film/name ?film-name]
[?film :film/cast ?actor-1]
[?film :film/cast ?actor-2]
[?actor-1 :actor/name "Nancy Allen"]
[?actor-2 :actor/name "Peter Weller"]]
#conn)
; => #{["RoboCop"]}
However, the query is flawed because I don't know how to express that any matches should exclude any actors who are not Allen or Weller—again, I want to find the movies where only Allen and Weller have collaborated without any other actors, so I want to adapt the above query to produce the empty set. How can I adjust this query to enforce this requirement?
Because DataScript doesn't have negation (as of May 2016), I don't believe that's possible with one static query in 'pure' Datalog.
My way to go would be:
build the query programmatically to add the N clauses that state that the cast must contain the N actors
Add a predicate function which, given a movie, the database, and the set of actors ids, uses the EAVT index to find if each movie has an actor that is not in the set.
Here's a basic implementation
(defn only-those-actors? [db movie actors]
(->> (datoms db :eavt movie :film/cast) seq
(every? (fn [[_ _ actor]]
(contains? actors actor)))
))
(defn find-movies-with-exact-cast [db actors-names]
(let [actors (set (d/q '[:find [?actor ...] :in $ [?name ...] ?only-those-actors :where
[?actor :actor/name ?name]]
db actors-names))
query {:find '[[?movie ...]]
:in '[$ ?actors ?db]
:where
(concat
(for [actor actors]
['?movie :film/cast actor])
[['(only-those-actors? ?db ?movie ?actors)]])}]
(d/q query db actors db only-those-actors?)))
You can use predicate fun and d/entity together for filtering datoms by :film/cast field of an entity. This approach looks much more straightforward until Datascript doesn't support negation (not operator and so on).
Look at the row (= a (:age (d/entity db e)) in the test case of the Datascript here
[{:db/id 1 :name "Ivan" :age 10}
{:db/id 2 :name "Ivan" :age 20}
{:db/id 3 :name "Oleg" :age 10}
{:db/id 4 :name "Oleg" :age 20}]
...
(let [pred (fn [db e a]
(= a (:age (d/entity db e))))]
(is (= (q/q '[:find ?e
:in $ ?pred
:where [?e :age ?a]
[(?pred $ ?e 10)]]
db pred)
#{[1] [3]})))))
In your case, the predicate body could look something like this
(clojure.set/subset? actors (:film/cast (d/entity db e))
In regards to performance, the d/entity call is fast because it is a lookup by index.
Looking at the tutorial documents, I've tried creating a partition by loading my schema from a file, which, among other things, contains the following:
{:db/id #db/id[:db.part/db],
:db/ident :account,
:db.install/_partition :db.part/db}
If I try inserting data with the following:
(d/transact conn
[{:db/id #db/id[:db.part/user -1]
:validation/email email
:validation/code code}])
Everthing works as expected. But if I replace "user" with my partitions name "account", like this:
(d/transact conn
[{:db/id #db/id[:db.part/account -1]
:validation/email email
:validation/code code}])
I get this error:
IllegalArgumentExceptionInfo :db.error/not-a-db-id Invalid db/id: #db/id[:db.part/account -1] datomic.error/arg (error.clj:57)
What am I doing wrong? How can I even be sure that I've created the partitions?
your partition name is :account, not :db.part/account.
this code below should work.
(d/transact conn
[{:db/id #db/id[:account -1]
:validation/email email
:validation/code code}])
You can query for list of installed partitions like this
(d/q '[:find [?ident ...]
:where
[?e :db/ident ?ident]
[_ :db.install/partition ?e]]
db)
=> [:account :db.part/tx :db.part/user]
On a html-post page a user can input various fields and hit submit,
My router.clj code looks like
(POST "/postGO" [ post-title post-input post-tags :as request ]
(def email (get-in request [:session :ze-auth-email]))
;; connect to datomic and write in the request
(dbm/add-ze-post post-title post-input post-tags email) ;; db insert
{:status 200,
:body "successfully added the post to the database",
:headers {"Content-Type" "text/plain"}}) ;;generic return page
It works well, but I want to redirect the user afterwards to a page that can show them their uploaded post. To do this, it would be very helpful to have the eid of the entity just transacted.
;; code from my dbm file for playing directly with the database
;; id est: the db transact code
(defn add-ze-blurb [title, content, tags, useremail]
(d/transact conn [{:db/id (d/tempid :db.part/user),
:post/title title,
:post/content content,
:post/tag tags,
:author/email useremail}]))
Is there any way to have datomic return the eid as soon as something is added to the DB successfully, or should I use another query right afterward to make sure it's there?
For simple cases, simply deref the future returned by d/transact and use :tx-data. Full example:
(require '[datomic.api :refer [db q] :as d])
(def uri "datomic:mem://test")
(d/create-database uri)
(def conn (d/connect uri))
(def schema [{:db/id (d/tempid :db.part/db)
:db/ident :foo
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one
:db.install/_attribute :db.part/db}])
#(d/transact conn schema)
(def tempid (d/tempid :db.part/user))
(def tx [{:db/id tempid :foo "bar"}])
(def result #(d/transact conn tx)) ;;
(def eid (:e (second (:tx-data result))))
(assert (= "bar" (:foo (d/entity (db conn) eid))))
You can alternatively use d/resolve-tempid:
(def eid (d/resolve-tempid (db conn) (:tempids result) tempid))
(assert (= "bar" (:foo (d/entity (db conn) eid))))
As described in the Datomic documentation, after the deref the transaction will have been applied by the transactor and the data returned reflects the new value of the database.