After I run a transaction in datomic to insert a value, how I can use the return value of the transaction to get the ids of any entities that were created?
Here is a sample of the return value I get after an insert:
#<promise$settable_future$reify__4841#7c92b2e9: {:db-before datomic.db.Db#62d0401f, :db-after datomic.db.Db#bba61dfc,
:tx-data [#Datum{:e 13194139534331 :a 50
:v #inst "2013-06-19T11:38:08.025-00:00"
:tx 13194139534331 :added true} #Datum{:e 17592186045436 .....
I can see the underlying datums...how can I extract their values?
Use d/resolve-tempid. If you were to transact a single entity, looking at :tx-data would work but if your transaction contained more than one entity, then you wouldn't know the order in which they appear in :tx-data.
What you should do is give temporary ids to your entities (before transacting them) using either (d/tempid) or its literal representation #db/id[:db.part/user _negativeId_] and then use d/resolve-tempid to go from your temporary id to the real id given by the database. The code would look something like:
(d/resolve-tempid (d/db conn) (:tempids tx) (d/tempid :db.part/user _negativeId_))
For a full code sample, see this gist.
Ah, figured it out.
I had to deref the Clojure promise, and then I was able to yank out the values I wanted:
(:e (second (:tx-data #(transact! conn query))))
Wrote a quick function based on a2ndrade's answer. The naming isn't ideal and I may be committing idiomatic faux pas; suggestions are very much welcome.
(ns my.datomic.util
(:require [datomic.api :as d]))
(defn transact-and-get-id
"Transact tx and return entity id."
[conn tx]
(let [tempid (:db/id tx)
post-tx #(d/transact conn [tx])
db (:db-after post-tx)
entid (d/resolve-tempid db (:tempids post-tx) tempid)]
entid))
Example usage:
(def my-conn
(d/connect (str "datomic:sql://datomic?jdbc:postgresql://"
"127.0.1:5432/datomic?user=datomic&password=somepw")
(defn thing-tx
"Create transaction for new thing."
[name]
{:db/id (d/tempid :db.part/user)
:thing/name name})
(transact-and-get-id my-conn (thing-tx "Bob")) ;; => 17592186045502
The Tupelo Datomic library has a function (td/eids tx-result) to easily extract the EIDs created in a transaction. For example:
; Create Honey Rider and add her to the :people partition
(let [tx-result #(td/transact *conn*
(td/new-entity :people ; <- partition is first arg (optional) to td/new-entity
{ :person/name "Honey Rider" :location "Caribbean" :weapon/type #{:weapon/knife} } ))
[honey-eid] (td/eids tx-result) ; retrieve Honey Rider's EID from the seq (destructuring)
]
Related
I'm hitting an API that returns some json that I am trying to parse. I look a the keys of the returned parsed values and it says that it has a key "id", but when I try to get the id from the map, I get nothing back.
E.g.:
(require '[clj-http.client :as client]
'[cheshire.core :refer :all])
(def app-url "http://myapp.com/api/endpoint")
(defn get-application-ids [] (parse-string (:body (client/get app-url))))
(defn -main [] (println (map :id (get-application-ids))))
This returns (nil nil nil). Which, AFAIKT, it shouldn't - instead it should return the value of the :id field, which is not null.
Helpful facts:
When I run (map keys (get-application-ids)) to find the keys of the returned structure, I get ((id name attempts) (id name attempts) (id name attempts)).
(type (get-application-ids)) returns clojure.lang.LazySeq
(map type (get-application-ids)) returns (clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap)
(println (get-application-ids)) returns (one example of the three returned):
({id application_1595586907236_1211, name myname, attempts [{appSparkVersion 2.4.0-cdh6.1.0, lastUpdated 2020-07-26T20:18:47.088GMT, completed true, lastUpdatedEpoch 1595794727088, sparkUser super, endTimeEpoch 1595794726804, startTime 2020-07-26T20:04:05.998GMT, attemptId 1, duration 880806, endTime 2020-07-26T20:18:46.804GMT, startTimeEpoch 1595793845998}]})
Everything about this tells me that (map :id (get-application-ids)) should return the value of the id field, but it doesn't. What am I missing?
It seems you are using cheshire.core/parse-string. This will return string keys, not keywords. See this example.
So, it appears that your key is the string "id", not the keyword :id. To verify this theory, try putting in the debugging statement:
(prn (:body (client/get app-url)))
To ask Cheshire to convert map keys from strings to keywords, use the form
(parse-string <json-src> true) ; `true` => output keyword keys
See also this list of documentation sources. Especially study the Clojure CheatSheet daily.
I'd like to know how to I can find the id of the entity that was modified/created/deleted by the latest transaction in Datomic. How can I do this?
For this kind of read pattern (temporal-based), you'll want to use the Log API. Note that:
there may be more than one entity that was affected by the last transaction.
actually, the transaction itself is represented by an entity which is created for that transaction, which you may want to filter out of your result.
Here's a sample implementation:
(defn affected-entities
"Given a Datomic connection, returns the set of entity ids that were affected
by the last transaction (in e position), excluding the entity representing the
transaction itself."
[conn]
(let [db (d/db conn)]
(->>
(d/q '[:find [?e ...] :in ?log ?t1 ?t2 :where
[(tx-ids ?log ?t1 ?t2) [?tx ...]] ;; binds the last tx
[(tx-data ?log ?tx) [[?e]]]]
(d/log conn) (d/basis-t db) (d/next-t db))
;; filtering out the transaction entity
(remove (fn [eid]
(->> eid d/part (d/ident db) (= :db.part/tx))))
set)))
Setup Consider the following DataScript database of films and cast, with data stolen from learndatalogtoday.org: the following code can be executed in a JVM/Clojure REPL or a ClojureScript REPL, as long as project.clj contains [datascript "0.15.0"] as a dependency.
(ns user
(:require [datascript.core :as d]))
(def data
[["First Blood" ["Sylvester Stallone" "Brian Dennehy" "Richard Crenna"]]
["Terminator 2: Judgment Day" ["Linda Hamilton" "Arnold Schwarzenegger" "Edward Furlong" "Robert Patrick"]]
["The Terminator" ["Arnold Schwarzenegger" "Linda Hamilton" "Michael Biehn"]]
["Rambo III" ["Richard Crenna" "Sylvester Stallone" "Marc de Jonge"]]
["Predator 2" ["Gary Busey" "Danny Glover" "Ruben Blades"]]
["Lethal Weapon" ["Gary Busey" "Mel Gibson" "Danny Glover"]]
["Lethal Weapon 2" ["Mel Gibson" "Joe Pesci" "Danny Glover"]]
["Lethal Weapon 3" ["Joe Pesci" "Danny Glover" "Mel Gibson"]]
["Alien" ["Tom Skerritt" "Veronica Cartwright" "Sigourney Weaver"]]
["Aliens" ["Carrie Henn" "Sigourney Weaver" "Michael Biehn"]]
["Die Hard" ["Alan Rickman" "Bruce Willis" "Alexander Godunov"]]
["Rambo: First Blood Part II" ["Richard Crenna" "Sylvester Stallone" "Charles Napier"]]
["Commando" ["Arnold Schwarzenegger" "Alyssa Milano" "Rae Dawn Chong"]]
["Mad Max 2" ["Bruce Spence" "Mel Gibson" "Michael Preston"]]
["Mad Max" ["Joanne Samuel" "Steve Bisley" "Mel Gibson"]]
["RoboCop" ["Nancy Allen" "Peter Weller" "Ronny Cox"]]
["Braveheart" ["Sophie Marceau" "Mel Gibson"]]
["Mad Max Beyond Thunderdome" ["Mel Gibson" "Tina Turner"]]
["Predator" ["Carl Weathers" "Elpidia Carrillo" "Arnold Schwarzenegger"]]
["Terminator 3: Rise of the Machines" ["Nick Stahl" "Arnold Schwarzenegger" "Claire Danes"]]])
(def conn (d/create-conn {:film/cast {:db/valueType :db.type/ref
:db/cardinality :db.cardinality/many}
:film/name {:db/unique :db.unique/identity
:db/cardinality :db.cardinality/one}
:actor/name {:db/unique :db.unique/identity
:db/cardinality :db.cardinality/one}}))
(def all-datoms (mapcat (fn [[film actors]]
(into [{:film/name film}]
(map #(hash-map :actor/name %) actors)))
data))
(def all-relations (mapv (fn [[film actors]]
{:db/id [:film/name film]
:film/cast (mapv #(vector :actor/name %) actors)}) data))
(d/transact! conn all-datoms)
(d/transact! conn all-relations)
Description In a nutshell, there are two kinds of entities in this database—films and actors (word intended to be ungendered)—and three kinds of datoms:
film entity: :film/name (a unique string)
film entity: :film/cast (multiple refs)
actor entity: :actor/name (unique string)
Question I would like to construct a query which asks: which films have these N actors, and these N actors alone, appeared as the sole stars, for N>=2?
E.g., RoboCop starred Nancy Allen, Peter Weller, Ronny Cox, but no film starred solely the first two of these, Allen and Weller. Therefore, I would expect the following query to produce the empty set:
(d/q '[:find ?film-name
:where
[?film :film/name ?film-name]
[?film :film/cast ?actor-1]
[?film :film/cast ?actor-2]
[?actor-1 :actor/name "Nancy Allen"]
[?actor-2 :actor/name "Peter Weller"]]
#conn)
; => #{["RoboCop"]}
However, the query is flawed because I don't know how to express that any matches should exclude any actors who are not Allen or Weller—again, I want to find the movies where only Allen and Weller have collaborated without any other actors, so I want to adapt the above query to produce the empty set. How can I adjust this query to enforce this requirement?
Because DataScript doesn't have negation (as of May 2016), I don't believe that's possible with one static query in 'pure' Datalog.
My way to go would be:
build the query programmatically to add the N clauses that state that the cast must contain the N actors
Add a predicate function which, given a movie, the database, and the set of actors ids, uses the EAVT index to find if each movie has an actor that is not in the set.
Here's a basic implementation
(defn only-those-actors? [db movie actors]
(->> (datoms db :eavt movie :film/cast) seq
(every? (fn [[_ _ actor]]
(contains? actors actor)))
))
(defn find-movies-with-exact-cast [db actors-names]
(let [actors (set (d/q '[:find [?actor ...] :in $ [?name ...] ?only-those-actors :where
[?actor :actor/name ?name]]
db actors-names))
query {:find '[[?movie ...]]
:in '[$ ?actors ?db]
:where
(concat
(for [actor actors]
['?movie :film/cast actor])
[['(only-those-actors? ?db ?movie ?actors)]])}]
(d/q query db actors db only-those-actors?)))
You can use predicate fun and d/entity together for filtering datoms by :film/cast field of an entity. This approach looks much more straightforward until Datascript doesn't support negation (not operator and so on).
Look at the row (= a (:age (d/entity db e)) in the test case of the Datascript here
[{:db/id 1 :name "Ivan" :age 10}
{:db/id 2 :name "Ivan" :age 20}
{:db/id 3 :name "Oleg" :age 10}
{:db/id 4 :name "Oleg" :age 20}]
...
(let [pred (fn [db e a]
(= a (:age (d/entity db e))))]
(is (= (q/q '[:find ?e
:in $ ?pred
:where [?e :age ?a]
[(?pred $ ?e 10)]]
db pred)
#{[1] [3]})))))
In your case, the predicate body could look something like this
(clojure.set/subset? actors (:film/cast (d/entity db e))
In regards to performance, the d/entity call is fast because it is a lookup by index.
When I query for a list of Datomic entities, e.g like in the example below:
'[:find ?e
:where
[?e :category/name]]
Usually, I'd like to create a list of maps that represent the full entities, i.e
#{[1234] [2223]} => [{:category/name "x" :db/id 1234}, {:category/name "y" :db/id 2223}]
Here is my approach at the moment, in the form of a helper function.
(defn- db-ids->entity-maps
"Takes a list of datomic entity ids retrieves and returns
a list of hydrated entities in the form of a list of maps."
[db-conn db-ids]
(->>
db-ids
seq
flatten
(map #(->>
%
;; id -> lazy entity map
(d/entity (d/db db-conn))
;; realize all values, except for db/id
d/touch
(into {:db/id %})))))
Is there a better way?
With the pull api, this is pretty easy now.
'[:find [(pull ?e [*]) ...]
:in $ [[?e] ...]
:where [?e]]
I used to take this approach to save queries to the DB, the code is probably less reusable but it depends on what is more critical in your current scenario. I haven't a Datomic instance configured as I am not working with it right now so it may contain syntax error but I hope you get the idea.
(def query-result '[:find ?cat-name ?id
:where
[?cat-name :category/name
[?id :db/id]])
=>
#{["x" 1234] ["x" 2223]}
(defn- describe-values
"Adds proper keys to the given values."
[keys-vec query-result]
(vec (map #(zipmap keys-vec %) query-result))
(describe-values [:category/name :db/id] query-result)
=>
[{:db/id 2223, :category/name "x"} {:db/id 1234, :category/name "x"}]
I have a function that begins like this:
(defn data-one [suser]
(def suser-first-name
(select db/firstNames
(fields :firstname)
(where {:username suser})))
(def suser-middle-name
(select db/middleNames
(fields :middlename)
(where {:username suser})))
(def suser-last-name
(select db/middleNames
(fields :lastname)
(where {:username suser})))
;; And it just continues on and on...
)
Of course, I don't like this at all. I have this pattern repeating in many areas in my code-base and I'd like to generalize this.
So, I came up with the following to start:
(def data-input {:one '[suser-first-name db/firstNames :firstname]
'[suser-middle-name db/middleNames :middlename]
'[suser-last-name db/lastNames :lastname]})
(defpartial data-build [data-item suser]
;; data-item takes the arg :one in this case
`(def (data-input data-item)
(select (data-input data-item)
(fields (data-input data-item))
(where {:username suser}))))
There's really a few questions here:
-- How can I deconstruct the data-input so that it creates x functions when x is unknown, ie. that the values of :one is unknown, and that the quantities of keys in data-input is unknown.
-- I'm thinking that this is a time to create a macro, but I've never built one before, so I am hesitant on the idea.
And to give a little context, the functions must return values to be deconstructed, but I think once I get this piece solved, generalizing all of this will be doable:
(defpage "/page-one" []
(let [suser (sesh/get :username)]
(data-one suser)
[:p "Firat Name: "
[:i (let [[{fname :firstname}] suser-first-name]
(format "%s" fname))]
[:p "Middle Name: "
[:i (let [[{mname :emptype}] suser-middle-name]
(format "%s" mname))]
[:p "Last Name: "
[:i (let [[{lname :months}] suser-last-name]
(format "%s" lname))]]))
Some suggestions:
def inside a function is really nasty - you are altering the global environment, and it can cause all kinds of issues with concurrency. I would suggest storing the results in a map instead.
You don't need a macro here - all of the data fetches can be done relatively easily within a function
I would therefore suggest something like:
(def data-input [[:suser-first-name db/firstNames :firstname]
[:suser-middle-name db/middleNames :middlename]
[:suser-last-name db/lastNames :lastname]])
(def data-build [data-input suser]
(loop [output {}
items (seq data-input)]
(if items
(recur
(let [[kw db fieldname] (first items)]
(assoc output kw (select db (fields fieldname) (where {:username suser}))))
(next items))
output)))
Not tested as I don't have your database setup - but hopefully that gives you an idea of how to do this without either macros or mutable globals!
Nice question. First of all here's the macro that you asked for:
(defmacro defquery [fname table fields ]
(let [arg-name (symbol 'user-name)
fname (symbol fname)]
`(defn ~fname [~arg-name]
(print ~arg-name (str ~# fields)))))
You can call it like that:
(defquery suser-first-name db/firstNames [:firstname])
or if you prefer to keep all your configurations in a map, then it will accept string as the first argument instead of a symbol:
(defquery "suser-first-name" db/firstNames [:firstname])
Now, if you don't mind me recommending another solution, I would probably chose to use a single function closed around configuration. Something like that:
(defn make-reader [query-configurations]
(fn [query-type user-name]
(let [{table :table field-names :fields}
(get query-configurations query-type)]
(select table
(apply fields field-names)
(where {:username suser})))))
(def data-input {:firstname {:table db/firstNames :fields :firstname}
:middlename {:table db/middleNames :fields :middlename}
:lastname {:table db/lastNames :fields :lastname}})
(def query-function (make-reader data-input))
;; Example of executing a query
(query-function :firstname "tom")
By the way there's another way to use Korma:
;; This creates a template select from the table
(def table-select (select* db/firstNames))
;; This creates new select query for a specific field
(def first-name-select (fields table-select :firstname))
;; Creating yet another query that filters results by :username
(defn mkselect-for-user [suser query]
(where query {:username suser}))
;; Running the query for username "tom"
;; I fully specified exec function name only to show where it comes from.
(korma.core/exec (mkselect-for-user "tom" first-name-select))
For more information I highly recommend looking at Korma sources.