As a result of creating a Schema coercer and then trying to coerce a set of data I get as a result:
#schema.utils.ErrorContainer{:error #<ValidationError schema.utils.ValidationError#2abfe6ca>}
How do I get an explanation of what the actual validation error is?
You can find the definition of the ValidationError type here (since you seem to be using Clojure on the JVM I deleted the #+cljs expressions):
(deftype ValidationError [schema value expectation-delay fail-explanation])
And the definition for the ErrorContainer record here:
(defrecord ErrorContainer [error])
So to get more information about the error you could just access any of the fields of the inner ValidationError:
(defn validation-error-details [error]
(let [values (juxt #(.schema %)
#(.value %)
#(.expectation-delay %)
#(.fail-explanation %))]
(->> error :error values)))
;; Usage
(validation-error-details error) ; where error holds the value you posted
Related
I'm hitting an API that returns some json that I am trying to parse. I look a the keys of the returned parsed values and it says that it has a key "id", but when I try to get the id from the map, I get nothing back.
E.g.:
(require '[clj-http.client :as client]
'[cheshire.core :refer :all])
(def app-url "http://myapp.com/api/endpoint")
(defn get-application-ids [] (parse-string (:body (client/get app-url))))
(defn -main [] (println (map :id (get-application-ids))))
This returns (nil nil nil). Which, AFAIKT, it shouldn't - instead it should return the value of the :id field, which is not null.
Helpful facts:
When I run (map keys (get-application-ids)) to find the keys of the returned structure, I get ((id name attempts) (id name attempts) (id name attempts)).
(type (get-application-ids)) returns clojure.lang.LazySeq
(map type (get-application-ids)) returns (clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap)
(println (get-application-ids)) returns (one example of the three returned):
({id application_1595586907236_1211, name myname, attempts [{appSparkVersion 2.4.0-cdh6.1.0, lastUpdated 2020-07-26T20:18:47.088GMT, completed true, lastUpdatedEpoch 1595794727088, sparkUser super, endTimeEpoch 1595794726804, startTime 2020-07-26T20:04:05.998GMT, attemptId 1, duration 880806, endTime 2020-07-26T20:18:46.804GMT, startTimeEpoch 1595793845998}]})
Everything about this tells me that (map :id (get-application-ids)) should return the value of the id field, but it doesn't. What am I missing?
It seems you are using cheshire.core/parse-string. This will return string keys, not keywords. See this example.
So, it appears that your key is the string "id", not the keyword :id. To verify this theory, try putting in the debugging statement:
(prn (:body (client/get app-url)))
To ask Cheshire to convert map keys from strings to keywords, use the form
(parse-string <json-src> true) ; `true` => output keyword keys
See also this list of documentation sources. Especially study the Clojure CheatSheet daily.
I'm new to Clojure and as a learning exercise I'm trying to write a function that validates the present of keys in a map.
when I try to run the code below I get an error saying
java.lang.UnsupportedOperationException: nth not supported on this type: Keyword
(def record {:name "Foobar"})
(def validations [:name (complement nil?)])
(defn validate
[candidate [key val]]
(val (get candidate key))
(def actual (every? (partial validate record) validations))
(= true actual)
As I understand I'm partial applying the validate function and asserting every validations function on the map - but it doesn't seem to work - so I must be misunderstanding something?
The error is coming from the destructuring that you are using in validate: [key val]. Under the hood, destructuring uses the function nth, and that's what's failing.
Your issue is that you are passing to every? a list of [keyword validation-function]. And every? is iterating over each element of that list and calling the partially applied validate function with it. That means that your validate is called first with the keyword :name and that throws an exception, because you can not extract a [key val] pair out of the keyword :name, causing the exception.
To fix it, you need to make your validations list a list of lists as so:
(def record {:name "Foobar"})
(def validations [[:name (complement nil?)]])
(defn validate
[candidate [key val]]
(val (get candidate key)))
(def actual (every? (partial validate record) validations))
(= true actual)
;; => true
That way, every? is now iterating over each pair of [keyword validation-function], one at a time, and calling validate with that. Since this is a pair, it can be destructured into a [key val] and everything works.
And just so you know, in newer Clojure versions (1.6 and above), there is now a function called some? which is equivalent to (complement nil?).
every? takes a collection as the second argument, and so does your validate function. Since you're passing a vector to every?, validate is being called on the contents of the vector (that is, :name and (complement nil?)). You're also missing a closing paren in the definition of validate. Try the following:
(def record {:name "Foobar"})
(def validations [:name (complement nil?)])
(defn validate
[candidate [key val]]
(val (get candidate key)))
(def actual (every? (partial validate record) [validations]))
(= true actual)
BTW, you could use some? instead of (complement nil?)
I am completely new to Clojure. I still struggle with reading functions sometimes.
I am trying to change this function to use checkshire.
Here is my attempt :
defn- json->messages [json]
(let [records (:amazon.aws.sqs/records (cheshire/decode
json
:key-fn key-reader
:value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
The function key-reader function:
(def ^:private
key-reader
(memoize (fn [key]
(let [kebab-key (if (= "md5OfBody" key)
"md5-of-body"
(csk/->kebab-case key))]
(keyword "amazon.aws.sqs" kebab-key)))))
The function :
(def ^:private
value-reader
(memoize (fn [key value]
(if (= key :amazon.aws.sqs/receipt-handle)
value-reader
value))))
I than call the function like so :
(json->messages msg)
msg is a json string.
However I am getting the error below with that attempt :
Execution error (ArityException) at tech.matterindustries.titan.ion.lambda.sqs-receive/json->messages (sqs_receive.clj:36).
Wrong number of args (5) passed to: cheshire.core/parse-smile
You are sending the wrong number of args to cheshire.core/parse-smile. Do you have a piece of sample data?
Please also keep your code clean & formatted, like this:
(defn- json->messages
[json]
(let [records (:amazon.aws.sqs/records (cheshire/decode json :key-fn key-reader :value-fn value-reader))
add-origin-queue (fn [record]
(let [event-source-arn (:amazon.aws.sqs/event-source-arn record)
queue-name (arn->queue-name event-source-arn)]
(assoc record :amazon.aws.sqs/queue-name queue-name)))]
(map add-origin-queue records)))
I could not find decode in the Cheshire docs, but in the source it has this:
(def decode "Alias to parse-string for clojure-json users" parse-string)
I am disappointed in their incomplete docs.
A quick google shows the docs:
(parse-string string & [key-fn array-coerce-fn])
Returns the Clojure object corresponding to the given JSON-encoded string.
An optional key-fn argument can be either true (to coerce keys to keywords),
false to leave them as strings, or a function to provide custom coercion.
The array-coerce-fn is an optional function taking the name of an array field,
and returning the collection to be used for array values.
This may not be clear. What it means is there are 3 legal ways to call parse-string:
(parse-string <json-str>)
(parse-string <json-str> <key-fn>)
(parse-string <json-str> <key-fn> <array-coerce-fn>)
So you can call it with 1, 2, or 3 args. You cannot add in :key-fn or :value-fn map keys, as in your example.
Please also note that your key-reader and value-reader look like they do not match what cheshire/read-string is expecting.
Using prismatic/schema coerce is it possible to have default values when the coercion fails instead of the error message.
I have a value in a csv file which could be blank (nil) or s/Int. At the moment with the below code I get this for blanks:
#schema.utils.ErrorContainer{:error (not (integer? nil))}
code:
(def answers (slurp "excel/answers.csv"))
(def answers-field-schemas [s/Int s/Int s/Str s/Str s/Str s/Int s/Str s/Int s/Str s/Int s/Str s/Int s/Str s/Int s/Str])
(def answers-field-coercers
(mapv coerce/coercer
answers-field-schemas
(repeat coerce/string-coercion-matcher)))
(defn answers-coerce-fields [fields]
(mapv #(%1 %2) answers-field-coercers fields))
(def answers->data (map answers-coerce-fields (csv/parse-csv answers :end-of-line "\r")))
1 . Error you get is not coercion error, but validation error. Values must conform initial schema.
2 . To fix it, you need to loose your schema for fields that may be nil. Let's say it's second field:
(def answers-field-schemas [s/Int (s/maybe s/Int) ...])
At this point you will get nils instead of errors for nil fields:
user> (answers-coerce-fields ["1" nil])
[1 nil]
3 . If you really want default values instead of nils after coercion, you will need custom coercion matcher. Something like this:
(import 'schema.core.Maybe)
(defn field-matcher-with-default [default]
(fn [s]
(let [string-coercion-fn (or (coerce/string-coercion-matcher s)
identity)
maybe-coercion-fn (if (instance? schema.core.Maybe s)
(fnil identity default)
identity)]
(comp
string-coercion-fn
maybe-coercion-fn))))
Also modify previous coercers as follows:
(def answers-field-coercers
(mapv coerce/coercer
answers-field-schemas
;; your default here
(repeat (field-matcher-with-default -1))))
and then:
user> (answers-coerce-fields ["1" nil])
[1 -1]
Note that default value must conform schema as well, so it's not possible to set default value of type String for schema (s/maybe s/Int).
Using 'load-data' below from the Clojure repl (using 'util.clj' from the tutorial https://github.com/swannodette/om/wiki/Intermediate-Tutorial with a modified schema and initial data set) to load data into a new Datomic database, the data does not show up in the Datomic console.
However, I get no error message when performing the 'load-data' action from the repl.
The schema shows up as expected in the Datomic console. Using code unmodified from the tutorial, I can see both the schema and the data.
I must have a problem in the code that sets the initial data. But I don't know where it is since there is no error message.
How can I get error messages and other detail from an init transaction on a Datomic database?
Code:
(defn transact-all [conn f]
(doseq [txd (read-all f)]
(d/transact conn txd))
:done)
(defn load-schema []
(transact-all (get-conn) (io/resource "data/schema.edn")))
(defn load-data []
(transact-all (get-conn) (io/resource "data/initial.edn")))
;; Logging provides some comparison with the known working scenario, but so far I only can log entity id's:
(defn read-log []
(d/q '[:find ?e
:in ?log ?t1 ?t2
:where [(tx-ids ?log ?t1 ?t2) [?tx ...]]
[(tx-data ?log ?tx) [[?e]]]]
(d/log (get-conn)) #inst "2014-07-14" #inst "2015-07-01")
)
In Clojure you can use # or deref to get a transaction's results, e.g.:
#(d/transact conn txd)
The map it returns is described in the docs for d/transact:
http://docs.datomic.com/clojure/#datomic.api/transact
See in particular:
If the transaction aborts, attempts to get the future's value throw a java.util.concurrent.ExecutionException, wrapping a java.lang.Error containing error information. If the transaction times out, the call to transact itself will throw a RuntimeException. The transaction timeout can be set via the system property datomic.txTimeoutMsec, and defaults to 10000 (10 seconds).
Invalid transactions will also throw an IllegalArgumentException (or some other exception).