I am processing streams in Riemann and all the fields are strings. However I'd like to perform some numeric comparisons on some fields. Therefore I tried to convert them in float.
Let's say my stream is like that :
#riemann.codec.Event{:host "myHost", :service nil, :state nil, :description "my description", :pred_score "0.156"}
I've tried to modify the riemann.conf file with something like this :
(streams
(with :new_field (read-string :pred_score))
prn)
However I got some error and I feel like this is not the correct way to do this. I have recently read some stuffs making me think that I should use smap or adjust but I'm not sure.
I am absolutely not familiar with clojure by the way. (In fact I discovered it with riemann).
Have you got any idea on how to tackle the problem ?
Thanks in advance,
Robin.
I don't know about the riemann part, but in plain clojure you could use either Java interop or the Tupelo library. Using Java interop, you'd do:
> (def x {:host "myHost", :service nil, :state nil, :description "my description", :pred_score "0.156"} )
> (Double/parseDouble (:pred_score x))
0.156
The 'tupelo.parse' namespace wraps this inside a normal clojure function, and adds the ability to specify a default value to return in the event of a malformed value (instead of throwing an exception):
> (require '[tupelo.parse :as parse])
> (parse/parse-double (:pred_score x))
0.156
> (parse/parse-double "123.4x6" :default 0)
0
Related
I'm wondering if there is a way to convert from a less granular to more granular time type using tick (or java.time upon which tick is based)?
This doesn't work:
(-> (t/date "2015") (t/date))
;; => No implementation of method: :date of protocol: #'tick.core/IExtraction found
;; => for class: java.time.Year
It makes sense to me that it might not be possible to convert in this direction given that you are basically adding data, and so you'd have to make assumptions, for example picking the month and day to move to pick a date. Given that, you can "cast" from more to less granularity:
(-> (t/date "2015-01-01") (t/year))
;; => #time/year "2015"
The only way I can think to do this is to convert via strings:
(->> (t/year "2015")
(t/format (t/formatter "yyyy"))
(#(str % "-01-01"))
(t/date))
;; => #time/date "2015-01-01"
But this feels a bit hack-ey. What is the right way to do this?
(require '[clojure.instant :as ci])
(ci/read-instant-timestamp "2015")
=> #inst"2015-01-01T00:00:00.000000000-00:00"
or
(t/zoned-date-time 2015)
I'm trying to figure out how to do custom coercion with compojure-api and spec. By reading the docs and code I have been able to do coercion on the input (the body) but am unable to do coercion on the response body.
Specifically, I have a custom type, a timestamp, that is represented as a long within my app but for the web API I want to consume and return ISO timestamps (no, I don't want to use Joda internally).
The following is what I have that works for input coercion but I have been unable to properly coerce the response.
(ns foo
(:require [clj-time.core :as t]
[clj-time.coerce :as t.c]
[spec-tools.conform :as conform]
[spec-tools.core :as st]))
(def timestamp (st/create-spec
{:spec pos-int?
:form `pos-int?
:json-schema/default "2017-10-12T05:04:57.585Z"
:type :timestamp}))
(defn json-timestamp->long [_ val]
(t.c/to-long val))
(def custom-json-conforming
(st/type-conforming
(merge
conform/json-type-conforming
{:timestamp json-timestamp->long}
conform/strip-extra-keys-type-conforming)))
(def custom-coercion
(-> compojure.api.coercion.spec/default-options
(assoc-in [:body :formats "application/json"] custom-json-
conforming)
compojure.api.coercion.spec/create-coercion))
;; how do I use this for the response coercion?
(defn timestamp->json-string [_ val]
(t.c/to-string val))
;; I've tried the following but it doesn't work:
#_(def custom-coercion
(-> compojure.api.coercion.spec/default-options
(assoc-in [:body :formats "application/json"] custom-json-
conforming)
(assoc-in [:response :formats "application/json"]
(st/type-conforming
{:timestamp timestamp->json-string}))
compojure.api.coercion.spec/create-coercion))
Problem is that Spec Conforming is a one-way transformation pipeline:
s/conform (and because of that st/conform) does both transform and validate for the result. Your response coercion first converts the integer into date string and the validates it against the original spec predicate, which is pos-int? and it fails on that.
To support two-way transformations, you need to define the end result as either of the possible formats: e.g. change your predicate to something like #(or (pos-int? %) (string? %)) and it should work.
Or you can have two different Spec Records, one for input (timestamp-long with pos-int? predicate) and another for outputs (timestamp-string with string? predicate). But one needs to remember to use correct ones for request & responses.
CLJ-2251 could possible help if there was and extra :transform mode (not written in the issue yet), which would do conforming without validating the end results.
Normally, return transformations are done by the format encoder, but they usually dispatch on value types. For example Cheshire just sees an Long and has no clue that it should be written as date string.
Maybe people on the #clojure-spec slack could help. Would also like to know how to build this kind of two-way transformation with spec.
Let's consider a Clojure Spec regexp for hiccup syntax
(require '[clojure.spec :as spec])
(spec/def ::hiccup
(spec/cat :tag keyword?
:attributes (spec/? map?)
:content (spec/* (spec/or :terminal string?
:element ::hiccup))))
which works splendidly
(spec/conform ::hiccup [:div#app [:h5 {:id "loading-message"} "Connecting..."]])
; => {:tag :div#app, :content [[:element {:tag :h5, :attributes {:id "loading-message"}, :content [[:terminal "Connecting..."]]}]]}
until you try to generate some example data for your functions from the spec
(require '[clojure.spec.gen :as gen])
(gen/generate (spec/gen ::hiccup))
; No return value but:
; 1. Unhandled java.lang.OutOfMemoryError
; GC overhead limit exceeded
Is there a way to rewrite the spec so that it produces a working generator? Or do we have to attach some simplified generator to the spec?
The intent of spec/*recursion-limit* (default 4) is to limit recursive generation such that this should work. So either that's not working properly in one of the spec impls (* or or), or you are seeing rapid growth in something else (like map? or the strings). Without doing some tinkering, it's hard to know which is the problem.
This does generate (a very large example) for me:
(binding [spec/*recursion-limit* 1] (gen/generate (spec/gen ::hiccup)))
I do see several areas where the cardinalities are large even in that one example - the * and the size of the generated attributes map?. Both of those could be further constrained. It would be easiest to break these parts up further into more fine-grained specs and supply override generators where necessary (the attribute map could just be handled with map-of and :gen-max).
I have a very simple question about using Prismatic/schema to validate functions. I have a schema for a map that has a single key, the value of which is a function that takes a Bar schema as its single argument and returns anything (used for side effects):
(require '[schema.core :as s])
(def Bar {:baz s/Int})
(def Action :???)
(def Foo {:action Action})
The question is, how do I define Action? I've tried this:
(require '[schema.macros :as sm])
(def Action (sm/=> s/Any Bar))
This looks promising, but I can't get it to fail validation:
(s/explain Action)
;=> (=> Any {:baz Int})
;; This should fail
(s/validate Foo {:action :anything-goes})
;=> {:action :anything-goes}
What am I doing wrong here?
I read the docs and the tests in core_test, but I can't figure out how to do this.
I've found this: https://github.com/Prismatic/schema/blob/a21cc0113ed497f6410c55d92d9088bd710f0b47/src/cljx/schema/core.cljx#L888
So it would be something like:
(def Action (s/make-fn-schema s/Any [[Bar]]))
Although, the documentation does say this:
Currently function schemas are purely descriptive; they validate against any function, regardless of actual input and output types
I am adding Swagger annotations to JaxRs annotated services.
I have the following:
(^{
GET true
Path "/{who}"
ApiOperation {:value "Get a hello" :notes "simple clojure GET"}
Produces ["text/plain; charset=UTF-8"]
ApiResponses {:value [(ApiResponse {:code 200 :message "yay!"})]}
}
If I decompile the produced class the annotations look like this:
#ApiResponses({#com.wordnik.swagger.annotations.ApiResponse(code=200L, message="yay!")})
#Produces({"text/plain; charset=UTF-8"})
#ApiOperation(value="Get a hello", notes="simple clojure GET")
#Path("/{who}")
#GET(true)
notes that in the first annotation code = 200L
During runtime, this value must be an int, and I cannot figure out how to make this happen
if I try
ApiResponses {:value [(ApiResponse {:code (int 200) :message "yay!"})]}
I get a compilation error (using the maven swagger plugin)
Exception in thread "main" java.lang.ClassCastException: clojure.lang.Var cannot be cast to java.lang.Class, compiling:(pocclj/resourceclj.clj:14)
I have tried
(def success (int 200))
...
ApiResponses {:value [(ApiResponse {:code success :message "yay!"})]}
Which produces this compilation error:
Exception in thread "main" java.lang.IllegalArgumentException: Unsupported annotation value: success of class class java.lang.Integer, compiling:(pocclj/resourceclj.clj:14)
I have tried a bunch of other stuff (deref etc) but cant find the secret sauce.
I am fairly new to clojure and desperate for some help on this.
Thanks in advance
Martin
You are setting the type of ':code' correctly. Which can be tested independently:
user> (def something ^{:ApiResponses {:code (int 200) :message "yay!"}} {:some :data :goes :here})
#'user/something
user> (meta something)
{:ApiResponses {:code 200, :message "yay!"}}
user> (-> (meta something) :ApiResponses :code type)
java.lang.Integer
And without the cast the metadata contains the wrong type:
user> (def something-wrong ^{:ApiResponses {:code 200 :message "yay!"}} {:some :data :goes :here})
#'user/something-wrong
user> (meta something)
{:ApiResponses {:code 200, :message "yay!"}}
user> (-> (meta something-wrong) :ApiResponses :code type)
java.lang.Long
From the exception it looks like perhaps the call to ApiResponse is crashing. If ApiResponse is a macro that expects a number and not an s-expression, then I could see it not handling this properly. If it is a function you would need to look into why it is crashing.
If I provide a stub implementation for ApiResponse then It works for me:
user> (definterface Fooi (Something []))
user.Fooi
user> (def ApiResponse identity)
#'user/ApiResponse
user> (deftype Foo []
Fooi
(Something
^{GET true
Path "/{who}"
ApiOperation {:value "Get a hello" :notes "simple clojure GET"}
Produces ["text/plain; charset=UTF-8"]
ApiResponses {:value [(ApiResponse {:code (int 200) :message "yay!"})]}}
[this] (identity this)))
user.Foo
I don't know about ApiResponse, or much about annotations really, but: it looks like some macro (deftype?) is producing annotations for you, and you need it to see 200 as an int. Clojure doesn't have int literals, so the only way to hand an Integer object directly to a macro is through some other macro that calls it. It's not really possible to do this in a nice way; as far as I know you have to either use eval, or be very narrow by aiming specifically at int literals. Here's a sketch of a solution:
user> (use 'clojure.walk)
user> (defmacro with-int-literals [named-ints & body]
(prewalk-replace (into {}
(for [[k v] (partition 2 named-ints)]
[k (int v)]))
`(do ~#body)))
user> (map class (second (macroexpand-1 '(with-int-literals [x 100, y 200] [x y]))))
(java.lang.Integer java.lang.Integer)
So if you wrap your entire deftype (or whatever macro is generating these annotations) with a with-int-literals form, you can produce integers for it instead of longs. I don't actually know that this will work; perhaps there's something in the annotation processor that's fundamentally unable to handle ints for some reason. But at least this way you can offer it ints and hope for the best.
Edit
Since you actually need int literals in metadata, rather than in "ordinary" code, prewalk won't actually look at the data you care about. You'll have to write a version of walk that treats metadata in a sensible way, and then use it instead of prewalk here.