Using Plumatic Schema to coerce to bigdec - clojure

I have incoming data of the type {:loan/amount 1200}.
Would it be possible to use plumatic Schema to coerce this to {:loan/amount 1200M}, ie coercing numbers (or even strings of digits) to bigdecimals?
I don't how to define a new data type (like s/Bigdec) and then make sure it uses clojure.core/bigdec to coerce a certain value to a java.math.BigDecimal.

There are two separate concepts in Schema: validation and coercion.
For the first one you need to define your schema. Classes are treated as schemas so you don't need to create a custom one for java.math.BigDecimal. Your schema might look like:
(require '[schema.core :as s])
(require '[schema.coerce :as c])
(s/defschema Loan {:loan/amount java.math.BigDecimal})
Now you can validate your data against the schema:
(s/validate Loan {:loan/amount 10M})
;; => {:loan/amount 10M}
Now if you have some data that you would like to coerce you need to define a coercion function which is a matcher from a desired target schema (java.math.BigDecimal in your case) to a function that will convert the actual value to the desired bigdec value.
(def safe-bigdec (c/safe bigdec)
schema.coerce/safe is a utility function which wraps the original function and if the original one throws an exception when called, safe will return original input value instead of throwing the exception.
Our matcher function will check if the current schema element is BigDecimal and return converting function or nil otherwise (meaning there is no coercion for other types):
(defn big-decimal-matcher [schema]
(when (= java.math.BigDecimal schema)
safe-bigdec))
And finally we need a coercer for performing actual coercion:
(def loan-coercer (c/coercer Loan big-decimal-matcher))
With all the setup we can now use our coercer:
(loan-coercer {:loan/amount "12.34"})
;; => {:loan/amount 12.34M}
(loan-coercer {:loan/amount 1234})
;; => {:loan/amount 1234M}
(loan-coercer {:loan/amount "abc"})
;; => #schema.utils.ErrorContainer{:error {:loan/amount (not (instance? java.math.BigDecimal "abc"))}}

Related

Can't extract key from Map

I'm hitting an API that returns some json that I am trying to parse. I look a the keys of the returned parsed values and it says that it has a key "id", but when I try to get the id from the map, I get nothing back.
E.g.:
(require '[clj-http.client :as client]
'[cheshire.core :refer :all])
(def app-url "http://myapp.com/api/endpoint")
(defn get-application-ids [] (parse-string (:body (client/get app-url))))
(defn -main [] (println (map :id (get-application-ids))))
This returns (nil nil nil). Which, AFAIKT, it shouldn't - instead it should return the value of the :id field, which is not null.
Helpful facts:
When I run (map keys (get-application-ids)) to find the keys of the returned structure, I get ((id name attempts) (id name attempts) (id name attempts)).
(type (get-application-ids)) returns clojure.lang.LazySeq
(map type (get-application-ids)) returns (clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap clojure.lang.PersistentArrayMap)
(println (get-application-ids)) returns (one example of the three returned):
({id application_1595586907236_1211, name myname, attempts [{appSparkVersion 2.4.0-cdh6.1.0, lastUpdated 2020-07-26T20:18:47.088GMT, completed true, lastUpdatedEpoch 1595794727088, sparkUser super, endTimeEpoch 1595794726804, startTime 2020-07-26T20:04:05.998GMT, attemptId 1, duration 880806, endTime 2020-07-26T20:18:46.804GMT, startTimeEpoch 1595793845998}]})
Everything about this tells me that (map :id (get-application-ids)) should return the value of the id field, but it doesn't. What am I missing?
It seems you are using cheshire.core/parse-string. This will return string keys, not keywords. See this example.
So, it appears that your key is the string "id", not the keyword :id. To verify this theory, try putting in the debugging statement:
(prn (:body (client/get app-url)))
To ask Cheshire to convert map keys from strings to keywords, use the form
(parse-string <json-src> true) ; `true` => output keyword keys
See also this list of documentation sources. Especially study the Clojure CheatSheet daily.

compojure-api spec coercion on response body

I'm trying to figure out how to do custom coercion with compojure-api and spec. By reading the docs and code I have been able to do coercion on the input (the body) but am unable to do coercion on the response body.
Specifically, I have a custom type, a timestamp, that is represented as a long within my app but for the web API I want to consume and return ISO timestamps (no, I don't want to use Joda internally).
The following is what I have that works for input coercion but I have been unable to properly coerce the response.
(ns foo
(:require [clj-time.core :as t]
[clj-time.coerce :as t.c]
[spec-tools.conform :as conform]
[spec-tools.core :as st]))
(def timestamp (st/create-spec
{:spec pos-int?
:form `pos-int?
:json-schema/default "2017-10-12T05:04:57.585Z"
:type :timestamp}))
(defn json-timestamp->long [_ val]
(t.c/to-long val))
(def custom-json-conforming
(st/type-conforming
(merge
conform/json-type-conforming
{:timestamp json-timestamp->long}
conform/strip-extra-keys-type-conforming)))
(def custom-coercion
(-> compojure.api.coercion.spec/default-options
(assoc-in [:body :formats "application/json"] custom-json-
conforming)
compojure.api.coercion.spec/create-coercion))
;; how do I use this for the response coercion?
(defn timestamp->json-string [_ val]
(t.c/to-string val))
;; I've tried the following but it doesn't work:
#_(def custom-coercion
(-> compojure.api.coercion.spec/default-options
(assoc-in [:body :formats "application/json"] custom-json-
conforming)
(assoc-in [:response :formats "application/json"]
(st/type-conforming
{:timestamp timestamp->json-string}))
compojure.api.coercion.spec/create-coercion))
Problem is that Spec Conforming is a one-way transformation pipeline:
s/conform (and because of that st/conform) does both transform and validate for the result. Your response coercion first converts the integer into date string and the validates it against the original spec predicate, which is pos-int? and it fails on that.
To support two-way transformations, you need to define the end result as either of the possible formats: e.g. change your predicate to something like #(or (pos-int? %) (string? %)) and it should work.
Or you can have two different Spec Records, one for input (timestamp-long with pos-int? predicate) and another for outputs (timestamp-string with string? predicate). But one needs to remember to use correct ones for request & responses.
CLJ-2251 could possible help if there was and extra :transform mode (not written in the issue yet), which would do conforming without validating the end results.
Normally, return transformations are done by the format encoder, but they usually dispatch on value types. For example Cheshire just sees an Long and has no clue that it should be written as date string.
Maybe people on the #clojure-spec slack could help. Would also like to know how to build this kind of two-way transformation with spec.

Use of prismatic schema =>

What is the intended use of the => macro?
https://github.com/plumatic/schema/blob/master/src/cljx/schema/core.cljx#L1077-L1091
It seems to pass as long as a function is passed, for instance:
(s/defschema MySchema s/Int)
(def a (s/=> MySchema s/Num))
(s/validate a :ok)

Om Next's query->ast and ast->query functions

According to Om Next's documentation:
query->ast
(om.next/query->ast '[(:foo {:bar 1})])
Given a query expression return the AST.
ast->query
(om.next/ast->query ast)
Given a query expression AST, unparse it into a query expression.
Question: Why would one need these functions? That is, why would one need to directly manipulate a query abstract syntax tree (which I'm assuming are clojure maps that represent a query tree, along with some meta data) in om next?
There are some scenarios where you need to manipulate the query ast directly. In remote parsing mode, the parser expects your read functions to return either {:remote-name true } or a (possibly modified) {:remote-name AST-node} (which comes in as :ast in env). Most often you'll have to modify the AST to restructure it or add some data.
Example 1:
You have a query: [{:widget {:list [:name :created]}}]
The :widget part is pure UI related, your server doesn't need to know it exists, it only cares/knows about the :list.
Basically you'll have to modify the AST in the parser:
(defmethod read :list
[{:keys [ast query state]} key _ ]
(let [st #state]
{:value (om/db->tree query (get st key) st)
:remote (assoc ast :query-root true)}))
If you use om/process-rootsin your send function, it'll pick up the :query-root out of the ast and rewrite the query from [{:widget {:list [:name :created]}}] to [{:list [:name :created]}].
Example 2:
Another example would be when you want to mutate something at a remote:
(defmethod mutate 'item/update
[{:keys [state ast]} key {:keys [id title]}]
{:remote (assoc ast :params {:data {:id id :title title })})
Here you need to explicitly tell Om to include the data you want to send in the AST. At your remote you then pick apart :data to update the title at the given id
Most of the time you won't use the functions you described in your questions directly. The env available in every method of the parser has the ast in it.
Something I stumbled on, while trying to use Compassus:
Let's say you have a complex union/join query that includes parametric sub-queries. Something like this:
`[({:foo/info
{:foo/header [:foo-id :name]
:foo/details [:id :description :title]}} {:foo-id ~'?foo-id
:foo-desc ~'?foo-desc})]
Now let's say you want to set parameters so on the server you can parse it with om/parser and see those params as 3rd argument of read dispatch. Of course it's possible to write a function that would find all necessary parameters in the query and set the values. That's not easy though, and as I said - imagine your queries can be quite complex.
So what you can do - is to modify ast, ast includes :children :params key. So let's say the actual values for :foo-id and :foo-desc are in the state atom under :route-params key:
(defn set-ast-params [children params]
"traverses given vector of `children' in an AST and sets `params`"
(mapv
(fn [c]
(let [ks (clojure.set/intersection (-> params keys set)
(-> c :params keys set))]
(update-in c [:params] #(merge % (select-keys params (vec ks))))))
children))
(defmethod readf :foo/info
[{:keys [state query ast] :as env} k params]
(let [{:keys [route-params] :as st} #state
ast' (-> ast
(update :children #(set-ast-params % route-params))
om/ast->query
om.next.impl.parser/expr->ast)]
{:value (get st k)
:remote ast'}))
So basically you are:
- grabbing ast
- modifying it with actual values
you think maybe you can send it to server right then. Alas, no! Not yet. Thing is - when you do {:remote ast}, Om takes :query part of the ast, composes ast out of it and then sends it to the server. So you actually need to: turn your modified ast into query and then convert it back to ast again.
Notes:
set-ast-params function in this example would only work for the first level (if you have nested parametrized queries - it won't work),
make it recursive - it's not difficult
there are two different ways to turn ast to query and vice-versa:
(om/ast->query) ;; retrieves query from ast and sets the params based
;; of `:params` key of the ast, BUT. it modifies the query,
;; if you have a join query it takes only the first item in it. e.g. :
[({:foo/foo [:id]
:bar/bar [:id]} {:id ~'?id})]
;; will lose its `:bar` part
(om.next.impl.parser/ast->expr) ;; retrieves query from an ast,
;; but doesn't set query params based on `:params` keys of the ast.
;; there are also
(om/query->ast) ;; and
(om.next.impl.parser/expr->ast)

Function validation using Prismatic/schema in Clojure

I have a very simple question about using Prismatic/schema to validate functions. I have a schema for a map that has a single key, the value of which is a function that takes a Bar schema as its single argument and returns anything (used for side effects):
(require '[schema.core :as s])
(def Bar {:baz s/Int})
(def Action :???)
(def Foo {:action Action})
The question is, how do I define Action? I've tried this:
(require '[schema.macros :as sm])
(def Action (sm/=> s/Any Bar))
This looks promising, but I can't get it to fail validation:
(s/explain Action)
;=> (=> Any {:baz Int})
;; This should fail
(s/validate Foo {:action :anything-goes})
;=> {:action :anything-goes}
What am I doing wrong here?
I read the docs and the tests in core_test, but I can't figure out how to do this.
I've found this: https://github.com/Prismatic/schema/blob/a21cc0113ed497f6410c55d92d9088bd710f0b47/src/cljx/schema/core.cljx#L888
So it would be something like:
(def Action (s/make-fn-schema s/Any [[Bar]]))
Although, the documentation does say this:
Currently function schemas are purely descriptive; they validate against any function, regardless of actual input and output types