I play around with clojure and its korma library using an sqlite3 database on windows. I follow an example of the 7web book. It introduces the select* function and its friends.
But using the fields function adds fields instead of restricting.
;; models.clj
(defentity issue
(entity-fields :id :project_id :title :description :status)
(has-many comment))
;; repl
test=> (-> (select* issue)
#_=> (fields :title)
#_=> (as-sql))
"SELECT \"issue\".\"id\", \"issue\".\"project_id\", \"issue\".\"title\", \"issue\".\"description\", \"issue\".\"status\", \"issue\".\"title\" FROM \"issue\""
Did i miss anything?
Like mentioned in issue #251 the reason is the entity-fields expression. It defines the default fields for queries. The fields function adds more fields to the defaults. Works as designed.
Therefore I removed entity-fields within defentity.
;; models.clj
(defentity issue
(has-many comment))
;; repl
test=> (-> (select* issue)
#_=> (fields :title)
#_=> (as-sql))
"SELECT \"issue\".\"title\" FROM \"issue\""
test=> (-> (select* issue)
#_=> (as-sql))
"SELECT \"issue\".* FROM \"issue\""
Related
I tried to use directly Clojure's hashmap with MapDB and ran into weird behaviour. I checked Clojure and MapDB sources and couldn't understand the problem.
First everything looks fine:
lein try org.mapdb/mapdb "1.0.6"
; defining a db for the first time
(import [org.mapdb DB DBMaker])
(defonce db (-> (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
.closeOnJvmShutdown
.compressionEnable
.make))
(defonce fruits (.getTreeMap db "fruits-store"))
(do (.put fruits :banana {:qty 2}) (.commit db))
(get fruits :banana)
=> {:qty 2}
(:qty (get fruits :banana))
=> 2
(first (keys (get fruits :banana)))
=> :qty
(= :qty (first (keys (get fruits :banana))))
=> true
CTRL-D
=> Bye for now!
Then I try to access the data again:
lein try org.mapdb/mapdb "1.0.6"
; loading previsously created db
(import [org.mapdb DB DBMaker])
(defonce db (-> (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
.closeOnJvmShutdown
.compressionEnable
.make))
(defonce fruits (.getTreeMap db "fruits-store"))
(get fruits :banana)
=> {:qty 2}
(:qty (get fruits :banana))
=> nil
(first (keys (get fruits :banana)))
=> :qty
(= :qty (first (keys (get fruits :banana))))
=> false
(class (first (keys (get fruits :banana))))
=> clojure.lang.Keyword
How come the same keyword be different with respect to = ?
Is there some weird reference problem happening ?
The problem is caused by the way equality of keywords works. Looking at the
implementation of the = function we see that since keywords are not
clojure.lang.Number or clojure.lang.IPersistentCollection their equality is
determined in terms of the Object.equals method. Skimming the source of
clojure.lang.Keyword we learn that keywords don't not override
Object.equals and therefore two keywords are equal iff they are the same
object.
The default serializer of MapDB is org.mapdb.SerializerPojo, a subclass of
org.mapdb.SerializerBase. In its documentation we can read that
it's a
Serializer which uses ‘header byte’ to serialize/deserialize most of classes
from ‘java.lang’ and ‘java.util’ packages.
Unfortunately, it doesn't work that well with clojure.lang classes; It doesn't
preserve identity of keywords, thus breaking equality.
In order to fix it let's attempt to write our own serializer using the
EDN format—alternatively, you could consider, say, Nippy—and use
it in our MapDB.
(require '[clojure.edn :as edn])
(deftype EDNSeralizer []
;; See docs of org.mapdb.Serializer for semantics.
org.mapdb.Serializer
(fixedSize [_]
-1)
(serialize [_ out obj]
(.writeUTF out (pr-str obj)))
(deserialize [_ in available]
(edn/read-string (.readUTF in)))
;; MapDB expects serializers to be serializable.
java.io.Serializable)
(def edn-serializer (EDNSeralizer.))
(import [org.mapdb DB DBMaker])
(def db (.. (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
closeOnJvmShutdown
compressionEnable
make))
(def more-fruits (.. db
(createTreeMap "more-fruits")
(valueSerializer (EDNSeralizer.))
(makeOrGet)))
(.put more-fruits :banana {:qty 2})
(.commit db)
Once the more-fruits tree map is reopened in a JVM with EDNSeralizer defined
the :qty object stored inside will be the same object as any other :qty
instance. As a result equality checks will work properly.
I have a MySQL field called thing_id, but I want to reference it as :thing-id in my code. I can define an entity like this:
(defentity thing
(entity-fields :id [:thing_id :thing-id]))
so that when I fetch things:
(select thing)
The MySQL field that contains an underscore is transformed:
[{:id 1 :thing-id 2}]
But I can't select with the aliased:
(select thing (where (= :thing-id 2)))
gives
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException
Unknown column 'thing.thing-id' in 'where clause'
I can fix it in each where call:
(select thing (where (= :thing_id 2)))
But I was hoping that the alias works both way. It doesn't appear to. Is there a way to set an alias that can be used in a select?
Bit too late, but here it goes...
Use a naming strategy, more here. So your spec should look like:
...
(:require [camel-snake-kebab.core :refer [->kebab-case ->snake_case]])
...
(def db-spec
{:classname "com.mysql.jdbc.Driver"
:subprotocol "mysql"
:delimiters "`"
:unsafe true
:subname db-subname
:user db-user
:password db-password
:naming {:keys ->kebab-case
:fields ->snake_case}})
I have a function that begins like this:
(defn data-one [suser]
(def suser-first-name
(select db/firstNames
(fields :firstname)
(where {:username suser})))
(def suser-middle-name
(select db/middleNames
(fields :middlename)
(where {:username suser})))
(def suser-last-name
(select db/middleNames
(fields :lastname)
(where {:username suser})))
;; And it just continues on and on...
)
Of course, I don't like this at all. I have this pattern repeating in many areas in my code-base and I'd like to generalize this.
So, I came up with the following to start:
(def data-input {:one '[suser-first-name db/firstNames :firstname]
'[suser-middle-name db/middleNames :middlename]
'[suser-last-name db/lastNames :lastname]})
(defpartial data-build [data-item suser]
;; data-item takes the arg :one in this case
`(def (data-input data-item)
(select (data-input data-item)
(fields (data-input data-item))
(where {:username suser}))))
There's really a few questions here:
-- How can I deconstruct the data-input so that it creates x functions when x is unknown, ie. that the values of :one is unknown, and that the quantities of keys in data-input is unknown.
-- I'm thinking that this is a time to create a macro, but I've never built one before, so I am hesitant on the idea.
And to give a little context, the functions must return values to be deconstructed, but I think once I get this piece solved, generalizing all of this will be doable:
(defpage "/page-one" []
(let [suser (sesh/get :username)]
(data-one suser)
[:p "Firat Name: "
[:i (let [[{fname :firstname}] suser-first-name]
(format "%s" fname))]
[:p "Middle Name: "
[:i (let [[{mname :emptype}] suser-middle-name]
(format "%s" mname))]
[:p "Last Name: "
[:i (let [[{lname :months}] suser-last-name]
(format "%s" lname))]]))
Some suggestions:
def inside a function is really nasty - you are altering the global environment, and it can cause all kinds of issues with concurrency. I would suggest storing the results in a map instead.
You don't need a macro here - all of the data fetches can be done relatively easily within a function
I would therefore suggest something like:
(def data-input [[:suser-first-name db/firstNames :firstname]
[:suser-middle-name db/middleNames :middlename]
[:suser-last-name db/lastNames :lastname]])
(def data-build [data-input suser]
(loop [output {}
items (seq data-input)]
(if items
(recur
(let [[kw db fieldname] (first items)]
(assoc output kw (select db (fields fieldname) (where {:username suser}))))
(next items))
output)))
Not tested as I don't have your database setup - but hopefully that gives you an idea of how to do this without either macros or mutable globals!
Nice question. First of all here's the macro that you asked for:
(defmacro defquery [fname table fields ]
(let [arg-name (symbol 'user-name)
fname (symbol fname)]
`(defn ~fname [~arg-name]
(print ~arg-name (str ~# fields)))))
You can call it like that:
(defquery suser-first-name db/firstNames [:firstname])
or if you prefer to keep all your configurations in a map, then it will accept string as the first argument instead of a symbol:
(defquery "suser-first-name" db/firstNames [:firstname])
Now, if you don't mind me recommending another solution, I would probably chose to use a single function closed around configuration. Something like that:
(defn make-reader [query-configurations]
(fn [query-type user-name]
(let [{table :table field-names :fields}
(get query-configurations query-type)]
(select table
(apply fields field-names)
(where {:username suser})))))
(def data-input {:firstname {:table db/firstNames :fields :firstname}
:middlename {:table db/middleNames :fields :middlename}
:lastname {:table db/lastNames :fields :lastname}})
(def query-function (make-reader data-input))
;; Example of executing a query
(query-function :firstname "tom")
By the way there's another way to use Korma:
;; This creates a template select from the table
(def table-select (select* db/firstNames))
;; This creates new select query for a specific field
(def first-name-select (fields table-select :firstname))
;; Creating yet another query that filters results by :username
(defn mkselect-for-user [suser query]
(where query {:username suser}))
;; Running the query for username "tom"
;; I fully specified exec function name only to show where it comes from.
(korma.core/exec (mkselect-for-user "tom" first-name-select))
For more information I highly recommend looking at Korma sources.
I am trying to create a very simple API with korma
Users can query a database like so:
localhost:8080/my_postgres_db/users.json?where[age]=50&limit=1
Currently I am getting an error when trying to apply a where clause to an existing, composable, query.
clojure.lang.ArityException: Wrong number of args (2) passed to: core$where
The code in question:
(defn- comp-query [q [func arg]]
(let [sql-fn (ns-resolve 'korma.core (-> func name symbol))]
(sql-fn q arg)))
(defn compose-query [table col]
(reduce comp-query (select* table) col))
Usage:
(def clauses {:where {:column1 10} :fields "a,b" :limit 10 :offset 500})
(-> (compose-query table clauses) select)
Everything behaves as expected, except for where clauses. I can combine limit, offset and fields in any way I choose and I get the expected results. Only when I have a :where key in my map do I run into the error.
Am I attempting something I shouldn't? Is this bad clojure? Any help would be appreciated.
Note: I have read this SO question
Edit: from lein repl I can manually compose a query in the same fashion and it works
(where (select* "my_table") {:a 5})
Edit:
If I modify my compose-query function to this:
(defn compose-query [table col]
; remove where clause to process seperately
(let [base (reduce comp-query (select* table) (dissoc col :where))]
(if-let [where-clause (:where col)]
(-> base (where where-clause))
base)))
Everything works as expected.
The problem here is that korma.core/where is not a function and needs to be handled specially. Where can't be implemented as a function and still correctly handle things like (where query (or (= :hits 1) (> :hits 5)))
You can use where* function as you are using select*.
Just make your clause map like:
(def clauses {:where* {:column1 10} :fields "a,b" :limit 10 :offset 500})
Just a hunch; expanding some of the threading macros makes it a little hard to see if they are correct:
core> (macroexpand-1 '(-> (compose-query table clauses) select))
(select (compose-query table clauses))
core> (macroexpand-1 '(-> func name symbol))
(clojure.core/-> (clojure.core/-> func name) symbol)
core> (macroexpand-1 '(clojure.core/-> func name))
(name func)
Passing func to name looks suspicious.
I am implementing a simple drop-down using hiccup:
;DATASET/CREATE
(defn get-cols-nms [table]
"This function gets the list of columns of a specific table".
(do (db/cols-list table)))
(defpartial form-dataset [cols-list]
(text-field "dataset_nm" "Input here dataset name")[:br]
(drop-down "table" tables-n)
(submit-button "Refresh")[:br]
(mapcat #(vector (check-box %) % [:br]) cols-list)
)
(defpage "/dataset/create" []
(common/layout
(form-to [:post "/dataset/create"]
(form-dataset (get-cols-nms (first tables-n))))))
(defpage [:post "/dataset/create"] {:as ks}
(common/layout
(let [table (ks :table)]
(form-to [:post "/dataset/create"]
(form-dataset (get-cols-nms table))))))
What I need is to issue a post request (as I think this the only way to do it, but I am open to suggestions) when the drop-down is selected on a specific table (so that "get-cols-nms" gets called with the selected table). In this way, when a table of the database is selected in the drop-down, the table columns will be automatically showed.
So, ultimately, the main point is for me to understand better this function:
(drop-down "table" tables-n)
I think that to do what I want I need the tag to have an "onchange" attribute that calls a javascript function. But I don't know: 1) if I can do this using the hiccup form-helper drop-down; 2) how can I issue (if this is the only solution, maybe there is an hiccup way?) a post request with javascript.
==EDIT==
Following the answer to this question, I rewrote the code above.It should be pretty straightforward. As I think there are not so many examples of hiccup out there, I will post my code here for reference.
Please, bear in mind that there is still a problem with this code: the drop-down won't stay on the selected item, but it will return at the default. This is because it submits "onchange". I still could not find a solution for that, maybe somebody could help...
;DATASET/CREATE
(defn get-cols-nms [table]
(do (db/cols-list table)))
(defpartial form-dataset [cols-list]
(text-field "dataset_nm" "Input here dataset name")[:br]
(assoc-in (drop-down "table" tables-n) [1 :onclick] "this.form.submit()")[:br]
[:input {:type "submit" :value "Submit" :name "name"}][:br]
(mapcat #(vector (check-box %) % [:br]) cols-list)
)
(defpage "/dataset/create" []
(common/layout
(form-to [:post "/dataset/create"]
(form-dataset(get-cols-nms (first tables-n))))))
(defpage [:post "/dataset/create"] {:as ks}
(common/layout
(prn ks)
(let [table (ks :table)]
(form-to [:post "/dataset/create"]
(if (= (:name ks) nil)
(form-dataset (get-cols-nms table))
[:p "It works!"])))))
hiccup.form-helpers/drop-down doesn't directly support adding attributes to its select element, but it does guarantee there is a standard hiccup attribute map in its return value - meaning the attributes are a map at index 1 (the second element) of the returned vector.
That means you can do something like
(assoc-in (drop-down ....) [1 :onchange] "this.form.submit()")
to generate a select tag with an onchange property.