I'm hitting an API that returns some json that I am trying to parse. I look a the keys of the returned parsed values and it says that it has a key "id", but when I try to get the id from the map, I get nothing back.
E.g.:
(require '[clj-http.client :as client]
'[cheshire.core :refer :all])
(def app-url "http://myapp.com/api/endpoint")
(defn get-application-ids [] (parse-string (:body (client/get app-url))))
(defn -main [] (println (map :id (get-application-ids))))
This returns (nil nil nil). Which, AFAIKT, it shouldn't - instead it should return the value of the :id field, which is not null.
Helpful facts:
When I run (map keys (get-application-ids)) to find the keys of the returned structure, I get ((id name attempts) (id name attempts) (id name attempts)).
(type (get-application-ids)) returns clojure.lang.LazySeq
(map type (get-application-ids)) returns (clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap)
(println (get-application-ids)) returns (one example of the three returned):
({id application_1595586907236_1211, name myname, attempts [{appSparkVersion 2.4.0-cdh6.1.0, lastUpdated 2020-07-26T20:18:47.088GMT, completed true, lastUpdatedEpoch 1595794727088, sparkUser super, endTimeEpoch 1595794726804, startTime 2020-07-26T20:04:05.998GMT, attemptId 1, duration 880806, endTime 2020-07-26T20:18:46.804GMT, startTimeEpoch 1595793845998}]})
Everything about this tells me that (map :id (get-application-ids)) should return the value of the id field, but it doesn't. What am I missing?
It seems you are using cheshire.core/parse-string. This will return string keys, not keywords. See this example.
So, it appears that your key is the string "id", not the keyword :id. To verify this theory, try putting in the debugging statement:
(prn (:body (client/get app-url)))
To ask Cheshire to convert map keys from strings to keywords, use the form
(parse-string <json-src> true) ; `true` => output keyword keys
See also this list of documentation sources. Especially study the Clojure CheatSheet daily.
Related
I made service endpoint api for getting single object by id and it works as expected. I tested it with Postman and in handler function. I use cljs-ajax library for asynchronous client. I cant change the state of Reagent atom when I get response. Here is the code:
(ns businesspartners.core
(:require [reagent.core :as r]
[ajax.core :refer [GET POST]]
[clojure.string :as string]))
(def business-partner (r/atom nil))
(defn get-partner-by-id [id]
(GET "/api/get-partner-by-id"
{:headers {"Accept" "application/transit+json"}
:params {:id id}
:handler #(reset! business-partner (:business-partner %))}))
When I tried to access business-partner atom I got nil value for that atom. I can't figure out why because another method is almost the same except it get's list of business partners and works fine.
When I change the get-partner-by-id function:
(defn get-partner-by-id [id]
(GET "/api/get-partner-by-id"
{:headers {"Accept" "application/transit+json"}
:params {:id id}
:handler (fn [arg]
(println :handler-arg arg)
(reset! business-partner (:business-partner arg))
(println "Business partner from handler: " #business-partner))}))
Output in the browser console:
:handler-arg {:_id 5e7ad2c84b5c2d44583e8ecd,
:address Main Street,
:email nenmit#gmail.com,
:phone 555888,
:name Nen Mit}
Business partner from handler: nil
So, as you can see, I have my object in handler as desired, but when I try to reset my atom nothing happens. That's the core of the problem I think. Thank you Alan.
When in doubt, use debug print statements. Make your handler look like this:
:handler (fn [arg]
(println :handler-arg arg)
(reset! business-partner (:business-partner arg)))
You may also want to use clojure.pprint/pprint to pretty-print the output, or also add (type arg) to the output.
You may also want to initialize the atom to a specific value like
:bp-default so you can see if the nil you observe is the original one or if it is being reset to nil.
Update
So it is clear the key :business-partner does not exist in the map you are receiveing. This is what you must debug.
Trying to pull a non-existent key out of a map always returns nil. You could also use the 3-arg version of get to make this explicit. Convert
(:business-partner arg) => (get arg :business-partner ::not-found)
and you'll see the keyword ::not-found appear in your atom, verifying what is occurring.
In order to catch these problems early, I nearly always use a simple function grab from the Tupelo library like so:
(:business-partner arg) => (grab :business-partner arg)
The grab function will throw an exception if the expected key is not found. This provides early-warning of problems so you can track them down faster.
Another hint: next time use prn instead of println and it will retain double-quotes on string output like:
"Main Street"
I'm new to Clojure and as a learning exercise I'm trying to write a function that validates the present of keys in a map.
when I try to run the code below I get an error saying
java.lang.UnsupportedOperationException: nth not supported on this type: Keyword
(def record {:name "Foobar"})
(def validations [:name (complement nil?)])
(defn validate
[candidate [key val]]
(val (get candidate key))
(def actual (every? (partial validate record) validations))
(= true actual)
As I understand I'm partial applying the validate function and asserting every validations function on the map - but it doesn't seem to work - so I must be misunderstanding something?
The error is coming from the destructuring that you are using in validate: [key val]. Under the hood, destructuring uses the function nth, and that's what's failing.
Your issue is that you are passing to every? a list of [keyword validation-function]. And every? is iterating over each element of that list and calling the partially applied validate function with it. That means that your validate is called first with the keyword :name and that throws an exception, because you can not extract a [key val] pair out of the keyword :name, causing the exception.
To fix it, you need to make your validations list a list of lists as so:
(def record {:name "Foobar"})
(def validations [[:name (complement nil?)]])
(defn validate
[candidate [key val]]
(val (get candidate key)))
(def actual (every? (partial validate record) validations))
(= true actual)
;; => true
That way, every? is now iterating over each pair of [keyword validation-function], one at a time, and calling validate with that. Since this is a pair, it can be destructured into a [key val] and everything works.
And just so you know, in newer Clojure versions (1.6 and above), there is now a function called some? which is equivalent to (complement nil?).
every? takes a collection as the second argument, and so does your validate function. Since you're passing a vector to every?, validate is being called on the contents of the vector (that is, :name and (complement nil?)). You're also missing a closing paren in the definition of validate. Try the following:
(def record {:name "Foobar"})
(def validations [:name (complement nil?)])
(defn validate
[candidate [key val]]
(val (get candidate key)))
(def actual (every? (partial validate record) [validations]))
(= true actual)
BTW, you could use some? instead of (complement nil?)
I am completely new to Clojure. I still struggle with reading functions sometimes.
I am trying to change this function to use checkshire.
Here is my attempt :
defn- json->messages [json]
(let [records (:amazon.aws.sqs/records (cheshire/decode
json
:key-fn key-reader
:value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
The function key-reader function:
(def ^:private
key-reader
(memoize (fn [key]
(let [kebab-key (if (= "md5OfBody" key)
"md5-of-body"
(csk/->kebab-case key))]
(keyword "amazon.aws.sqs" kebab-key)))))
The function :
(def ^:private
value-reader
(memoize (fn [key value]
(if (= key :amazon.aws.sqs/receipt-handle)
value-reader
value))))
I than call the function like so :
(json->messages msg)
msg is a json string.
However I am getting the error below with that attempt :
Execution error (ArityException) at tech.matterindustries.titan.ion.lambda.sqs-receive/json->messages (sqs_receive.clj:36).
Wrong number of args (5) passed to: cheshire.core/parse-smile
You are sending the wrong number of args to cheshire.core/parse-smile. Do you have a piece of sample data?
Please also keep your code clean & formatted, like this:
(defn- json->messages
[json]
(let [records (:amazon.aws.sqs/records (cheshire/decode json :key-fn key-reader :value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
I could not find decode in the Cheshire docs, but in the source it has this:
(def decode "Alias to parse-string for clojure-json users" parse-string)
I am disappointed in their incomplete docs.
A quick google shows the docs:
(parse-string string & [key-fn array-coerce-fn])
Returns the Clojure object corresponding to the given JSON-encoded string.
An optional key-fn argument can be either true (to coerce keys to keywords),
false to leave them as strings, or a function to provide custom coercion.
The array-coerce-fn is an optional function taking the name of an array field,
and returning the collection to be used for array values.
This may not be clear. What it means is there are 3 legal ways to call parse-string:
(parse-string <json-str>)
(parse-string <json-str> <key-fn>)
(parse-string <json-str> <key-fn> <array-coerce-fn>)
So you can call it with 1, 2, or 3 args. You cannot add in :key-fn or :value-fn map keys, as in your example.
Please also note that your key-reader and value-reader look like they do not match what cheshire/read-string is expecting.
Below is my attempt to iterate over a result set and get its values
(sql/with-connection db
(sql/with-query-results rs ["select * from user where UserID=?" 10000]
(doseq [rec rs
s rec]
(println (val s))
)))
But how do you extract one particular value from it; i need only the user name field.
Can anyone please demonstarte how to do this?
The result set is a sequence of maps, so if you wanted to obtain one field (e.g. one called name) then:
(sql/with-connection db
(sql/with-query-results rs ["select * from user where UserID=?" 10000]
(doseq [rec rs]
(let [name (:name rec)]
(println "User name:" name)
(println "Full record (including name):" rec)))))
But as mentioned in the comments, if you only want name, then select name from would be the more efficient option. The code above is useful when you need the full row for something else.
The with-connection / with-query-results syntax is deprecated as of clojure.java.jdbc 3.0. Filtering results can be done much easier with the new query syntax and additional :row-fn and :result-set-fn parameters.
(query db ["select * from user"]
:row-fn :name
:result-set-fn #(doall (take 1000 (drop 10000 %))))
Be sure to make the result-set-fn realize all values, it shouldn't return a lazy sequence (hence the doall in this example).
I have a function that begins like this:
(defn data-one [suser]
(def suser-first-name
(select db/firstNames
(fields :firstname)
(where {:username suser})))
(def suser-middle-name
(select db/middleNames
(fields :middlename)
(where {:username suser})))
(def suser-last-name
(select db/middleNames
(fields :lastname)
(where {:username suser})))
;; And it just continues on and on...
)
Of course, I don't like this at all. I have this pattern repeating in many areas in my code-base and I'd like to generalize this.
So, I came up with the following to start:
(def data-input {:one '[suser-first-name db/firstNames :firstname]
'[suser-middle-name db/middleNames :middlename]
'[suser-last-name db/lastNames :lastname]})
(defpartial data-build [data-item suser]
;; data-item takes the arg :one in this case
`(def (data-input data-item)
(select (data-input data-item)
(fields (data-input data-item))
(where {:username suser}))))
There's really a few questions here:
-- How can I deconstruct the data-input so that it creates x functions when x is unknown, ie. that the values of :one is unknown, and that the quantities of keys in data-input is unknown.
-- I'm thinking that this is a time to create a macro, but I've never built one before, so I am hesitant on the idea.
And to give a little context, the functions must return values to be deconstructed, but I think once I get this piece solved, generalizing all of this will be doable:
(defpage "/page-one" []
(let [suser (sesh/get :username)]
(data-one suser)
[:p "Firat Name: "
[:i (let [[{fname :firstname}] suser-first-name]
(format "%s" fname))]
[:p "Middle Name: "
[:i (let [[{mname :emptype}] suser-middle-name]
(format "%s" mname))]
[:p "Last Name: "
[:i (let [[{lname :months}] suser-last-name]
(format "%s" lname))]]))
Some suggestions:
def inside a function is really nasty - you are altering the global environment, and it can cause all kinds of issues with concurrency. I would suggest storing the results in a map instead.
You don't need a macro here - all of the data fetches can be done relatively easily within a function
I would therefore suggest something like:
(def data-input [[:suser-first-name db/firstNames :firstname]
[:suser-middle-name db/middleNames :middlename]
[:suser-last-name db/lastNames :lastname]])
(def data-build [data-input suser]
(loop [output {}
items (seq data-input)]
(if items
(recur
(let [[kw db fieldname] (first items)]
(assoc output kw (select db (fields fieldname) (where {:username suser}))))
(next items))
output)))
Not tested as I don't have your database setup - but hopefully that gives you an idea of how to do this without either macros or mutable globals!
Nice question. First of all here's the macro that you asked for:
(defmacro defquery [fname table fields ]
(let [arg-name (symbol 'user-name)
fname (symbol fname)]
`(defn ~fname [~arg-name]
(print ~arg-name (str ~# fields)))))
You can call it like that:
(defquery suser-first-name db/firstNames [:firstname])
or if you prefer to keep all your configurations in a map, then it will accept string as the first argument instead of a symbol:
(defquery "suser-first-name" db/firstNames [:firstname])
Now, if you don't mind me recommending another solution, I would probably chose to use a single function closed around configuration. Something like that:
(defn make-reader [query-configurations]
(fn [query-type user-name]
(let [{table :table field-names :fields}
(get query-configurations query-type)]
(select table
(apply fields field-names)
(where {:username suser})))))
(def data-input {:firstname {:table db/firstNames :fields :firstname}
:middlename {:table db/middleNames :fields :middlename}
:lastname {:table db/lastNames :fields :lastname}})
(def query-function (make-reader data-input))
;; Example of executing a query
(query-function :firstname "tom")
By the way there's another way to use Korma:
;; This creates a template select from the table
(def table-select (select* db/firstNames))
;; This creates new select query for a specific field
(def first-name-select (fields table-select :firstname))
;; Creating yet another query that filters results by :username
(defn mkselect-for-user [suser query]
(where query {:username suser}))
;; Running the query for username "tom"
;; I fully specified exec function name only to show where it comes from.
(korma.core/exec (mkselect-for-user "tom" first-name-select))
For more information I highly recommend looking at Korma sources.