I'm having problems with the datomic tutorial at the "Revisiting the past" section http://datomic.com/company/resources/tutorial.html
For the two queries below:
query = "[:find ?c :where [?c :community/name]]";
db_asOf_schema = conn.db().asOf(schema_tx_date);
System.out.println(Peer.q(query, db_asOf_schema).size()); // 0
db_since_data = conn.db().since(data_tx_date);
System.out.println(Peer.q(query, db_since_data).size()); // 0
I have tried these commands in clojure, but cannot get them working as described in the tutorial:
(since (db conn) (java.util.Date.) )
;; It should return 0 but returns the whole database instead
(def ts (q '[:find ?when :where [?tx :db/txInstant ?when]] (db conn)))
(count (since (db conn) (ffirst (reverse (sort ts))))))
;; returns 13, but should return 0
(count (as-of (db conn) (ffirst (sort ts)))))
;; returns 13, but should return 0
I'm not too sure is this is the right behaviour, is there anything I'm doing wrong?
If you are working through the Seattle tutorial in Clojure, probably the most important single thing to know is that working Clojure code is included in the Datomic distribution. The filename is samples/seattle/getting-started.clj, and you can simply follow along at the REPL.
Two observations on the Clojure code in your question:
The since function is documented to return a database value, not a number, so the behavior you are seeing is as expected. In order to see what is in the database, you need to issue a query.
Databases do not have any documented semantics for the Clojure count function, so you should not call count on them. Again, if you want to see what is in the database, you need to issue a query, e.g.
;; Find all transaction times, sort them in reverse order
(def tx-instants (reverse (sort (q '[:find ?when :where [_ :db/txInstant ?when]]
(db conn)))))
;; pull out two most recent transactions, most recent loaded
;; seed data, second most recent loaded schema
(def data-tx-date (ffirst tx-instants))
(def schema-tx-date (first (second tx-instants)))
;; make query to find all communities
(def communities-query '[:find ?c :where [?c :community/name]])
;; find all communities as of schema transaction
(let [db-asof-schema (-> conn db (d/as-of schema-tx-date))]
(println (count (seq (q communities-query db-asof-schema)))))
;; find all communities as of seed data transaction
(let [db-asof-data (-> conn db (d/as-of data-tx-date))]
(println (count (seq (q communities-query db-asof-data)))))
;; find all communities since seed data transaction
(let [db-since-data (-> conn db (d/since data-tx-date))]
(println (count (seq (q communities-query db-since-data)))))
Hope this helps. There is also a Datomic google group if you have more questions.
Related
I am using taoensso.carmine redis client and want to achieve the following: given sequence s, get all its elements that aren't exist in redis. (I mean for which redis's EXISTS command return false)
At first I thought to do the following:
(wcar conn
(remove #(car/exists %) s))
but it returns sequence of car/exists responses rather than filtering my sequence by them
(remove #(wcar conn (car exists %)) s)
Does the job but takes a lot of time because no-pipeling and using new connection each time.
So I end up with some tangled map manipulation below, but I believe there should be simplier way to achieve it. How?
(let [s (range 1 100)
existance (wcar conn
(doall
(for [i s]
(car/exists i))))
existance-map (zipmap s existance)]
(mapv first (remove (fn [[k v]] (= v 1)) existance-map)))
Your remove function is lazy, so it won't do anything. You also can't do data manipulation inside the wcar macro so I'd so something like this:
(let [keys ["exists" "not-existing"]]
(zipmap keys
(mapv pos?
(car/wcar redis-db
(mapv (fn [key]
(car/exists key))
keys)))))
Could you reexamine you're first solution? I don't know what wcar does, but this example shows that you're on the right track:
> (remove #(odd? %) (range 9))
(0 2 4 6 8)
The anonymous function #(odd? %) returns either true or false results which are used to determine which numbers to keep. However, it is the original numbers that are returned by (remove...), not true/false.
I'm trying to filter through a database, based on the keyword patter match.
To do this, I wrote ->
(defn find-users
[db keyword]
(if (>= (count keyword) 3)
(let [login-pattern (login-pattern keyword)]
(->> (d/datoms db :aevt :user/name)
(filter #(re-matches login-pattern (:v %)))
(map #(d/entity db (:e %)))))
[]))
But, I'm getting a StackOverflow error.
I think it's because of (map #(d/entity db (:e %)))
When I plainly do (map :e), the function works.
I'm a bit confused as to why the Stackoverflow would happen though, the queries I'm performing with map :v are returning only a few entities.
What's happening here?
I'm a little curious as to why not just use query? You can use predicate expression clauses for the same purpose that you've built the filter/map scenario above. See "Expression Clauses" at: http://docs.datomic.com/query.html
(defn find-users
[db keyword]
(if (>= (count keyword) 3)
(map #(d/entity db (first %))
(d/q '[:find ?e
:in $ ?login-pattern
:where
[?e :user/name ?name]
[(re-matches ?login-pattern ?name)]]
db
(login-pattern keyword)))
[]))
The query engine is likely to handle the size of intermediate results better than raw sequence manipulations.
If I'm using a Clojure atom to keep count of something (votes for example), I can do this:
(def votes (atom {}))
(defn vote! [candidate]
(swap! votes update-in [candidate] (fnil inc 0)))
(vote! "Abraham Lincoln")
(vote! "Abraham Lincoln")
(vote! "Winston Churchill")
votes ;=> {"Abraham Lincoln" 2, "Winston Churchill" 2}
Here, update-in neatly transforms the value at the given key, without having to look it up first.
How can I accomplish the same in Datomic? I could do something like this...
(defn vote! [db candidate]
(let [[e v] (first (q '[:find ?e ?v
:in $ ?candidate
:where [[?e :name ?candidate]
[?e :votes ?v]] db candidate)
(transact! conn [{:db/id e :votes (inc v)}]))
But it seems a bit cumbersome, having to run the query, return the value and then transact with the new value. Is there a more idiomatic way of doing this (like a swap! or update-in)?
To me it seems most idiomatic to record the fact of the vote, then make a count query when you want to know the total. Depending on the needs of your application, this approach supports things like checking for double votes, vote change/retraction, counts in the last 24 hours, ...
I have a function that begins like this:
(defn data-one [suser]
(def suser-first-name
(select db/firstNames
(fields :firstname)
(where {:username suser})))
(def suser-middle-name
(select db/middleNames
(fields :middlename)
(where {:username suser})))
(def suser-last-name
(select db/middleNames
(fields :lastname)
(where {:username suser})))
;; And it just continues on and on...
)
Of course, I don't like this at all. I have this pattern repeating in many areas in my code-base and I'd like to generalize this.
So, I came up with the following to start:
(def data-input {:one '[suser-first-name db/firstNames :firstname]
'[suser-middle-name db/middleNames :middlename]
'[suser-last-name db/lastNames :lastname]})
(defpartial data-build [data-item suser]
;; data-item takes the arg :one in this case
`(def (data-input data-item)
(select (data-input data-item)
(fields (data-input data-item))
(where {:username suser}))))
There's really a few questions here:
-- How can I deconstruct the data-input so that it creates x functions when x is unknown, ie. that the values of :one is unknown, and that the quantities of keys in data-input is unknown.
-- I'm thinking that this is a time to create a macro, but I've never built one before, so I am hesitant on the idea.
And to give a little context, the functions must return values to be deconstructed, but I think once I get this piece solved, generalizing all of this will be doable:
(defpage "/page-one" []
(let [suser (sesh/get :username)]
(data-one suser)
[:p "Firat Name: "
[:i (let [[{fname :firstname}] suser-first-name]
(format "%s" fname))]
[:p "Middle Name: "
[:i (let [[{mname :emptype}] suser-middle-name]
(format "%s" mname))]
[:p "Last Name: "
[:i (let [[{lname :months}] suser-last-name]
(format "%s" lname))]]))
Some suggestions:
def inside a function is really nasty - you are altering the global environment, and it can cause all kinds of issues with concurrency. I would suggest storing the results in a map instead.
You don't need a macro here - all of the data fetches can be done relatively easily within a function
I would therefore suggest something like:
(def data-input [[:suser-first-name db/firstNames :firstname]
[:suser-middle-name db/middleNames :middlename]
[:suser-last-name db/lastNames :lastname]])
(def data-build [data-input suser]
(loop [output {}
items (seq data-input)]
(if items
(recur
(let [[kw db fieldname] (first items)]
(assoc output kw (select db (fields fieldname) (where {:username suser}))))
(next items))
output)))
Not tested as I don't have your database setup - but hopefully that gives you an idea of how to do this without either macros or mutable globals!
Nice question. First of all here's the macro that you asked for:
(defmacro defquery [fname table fields ]
(let [arg-name (symbol 'user-name)
fname (symbol fname)]
`(defn ~fname [~arg-name]
(print ~arg-name (str ~# fields)))))
You can call it like that:
(defquery suser-first-name db/firstNames [:firstname])
or if you prefer to keep all your configurations in a map, then it will accept string as the first argument instead of a symbol:
(defquery "suser-first-name" db/firstNames [:firstname])
Now, if you don't mind me recommending another solution, I would probably chose to use a single function closed around configuration. Something like that:
(defn make-reader [query-configurations]
(fn [query-type user-name]
(let [{table :table field-names :fields}
(get query-configurations query-type)]
(select table
(apply fields field-names)
(where {:username suser})))))
(def data-input {:firstname {:table db/firstNames :fields :firstname}
:middlename {:table db/middleNames :fields :middlename}
:lastname {:table db/lastNames :fields :lastname}})
(def query-function (make-reader data-input))
;; Example of executing a query
(query-function :firstname "tom")
By the way there's another way to use Korma:
;; This creates a template select from the table
(def table-select (select* db/firstNames))
;; This creates new select query for a specific field
(def first-name-select (fields table-select :firstname))
;; Creating yet another query that filters results by :username
(defn mkselect-for-user [suser query]
(where query {:username suser}))
;; Running the query for username "tom"
;; I fully specified exec function name only to show where it comes from.
(korma.core/exec (mkselect-for-user "tom" first-name-select))
For more information I highly recommend looking at Korma sources.
I am trying to create a very simple API with korma
Users can query a database like so:
localhost:8080/my_postgres_db/users.json?where[age]=50&limit=1
Currently I am getting an error when trying to apply a where clause to an existing, composable, query.
clojure.lang.ArityException: Wrong number of args (2) passed to: core$where
The code in question:
(defn- comp-query [q [func arg]]
(let [sql-fn (ns-resolve 'korma.core (-> func name symbol))]
(sql-fn q arg)))
(defn compose-query [table col]
(reduce comp-query (select* table) col))
Usage:
(def clauses {:where {:column1 10} :fields "a,b" :limit 10 :offset 500})
(-> (compose-query table clauses) select)
Everything behaves as expected, except for where clauses. I can combine limit, offset and fields in any way I choose and I get the expected results. Only when I have a :where key in my map do I run into the error.
Am I attempting something I shouldn't? Is this bad clojure? Any help would be appreciated.
Note: I have read this SO question
Edit: from lein repl I can manually compose a query in the same fashion and it works
(where (select* "my_table") {:a 5})
Edit:
If I modify my compose-query function to this:
(defn compose-query [table col]
; remove where clause to process seperately
(let [base (reduce comp-query (select* table) (dissoc col :where))]
(if-let [where-clause (:where col)]
(-> base (where where-clause))
base)))
Everything works as expected.
The problem here is that korma.core/where is not a function and needs to be handled specially. Where can't be implemented as a function and still correctly handle things like (where query (or (= :hits 1) (> :hits 5)))
You can use where* function as you are using select*.
Just make your clause map like:
(def clauses {:where* {:column1 10} :fields "a,b" :limit 10 :offset 500})
Just a hunch; expanding some of the threading macros makes it a little hard to see if they are correct:
core> (macroexpand-1 '(-> (compose-query table clauses) select))
(select (compose-query table clauses))
core> (macroexpand-1 '(-> func name symbol))
(clojure.core/-> (clojure.core/-> func name) symbol)
core> (macroexpand-1 '(clojure.core/-> func name))
(name func)
Passing func to name looks suspicious.