I have the following function which gets a map with service name and threshold. It checks if the service crossed a defined threshold and then calls multiple downstream children on the event.
(defn tc
[s & children]
(where
(and (service (:service_name s)) (not (expired? event)))
(by [:host :service]
(where (> metric (:threshold s)
(with :state "critical"
(apply sdo children)))))))
I would like to build a stream dynamically using a vector of maps:
(def services [{:service "cpu/usage" :threshold 90}
{:service "memory/usage" :threshold 90}])
When trying to run it in a stream i'm getting the following warning:
(streams
(doseq [s services] (tc s prn)))
WARN [2015-01-05 14:27:07,187] Thread-15 - riemann.core - instrumentation service caught
java.lang.NullPointerException
at riemann.core$stream_BANG_$fn__11140.invoke(core.clj:19)
at riemann.core$stream_BANG_.invoke(core.clj:18)
at riemann.core$instrumentation_service$measure__11149.invoke(core.clj:57)
at riemann.service.ThreadService$thread_service_runner__8782$fn__8783.invoke(service.clj:66)
at riemann.service.ThreadService$thread_service_runner__8782.invoke(service.clj:65)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:701)
It works, if i run the streams function inside the doseq.
This one works and gives the following output:
(doseq [s services]
(streams (tc s prn)))
#riemann.codec.Event{:host "testhost", :service "memory/usage", :state "critical", :description nil, :metric 91.0, :tags nil, :time 1420460856, :ttl 60.0}
It seems to blow up if your events don't have all the required fields, here's a sample from a similar project where I build an event from a sequence of events (reducing) It's not exactly what you are doing though I'm generating events in the same way:
{:service (:service (first events))
:metric (->> events count)
:host "All-counts"
:state "OK"
:time (:time (last events))
:ttl default-interval}
I got NPE specifically when time was missing. If you can't inherit it form somewhere, just make it up (use now for instance) without a reasonable value here, event expiration will not work and you'll run out of RAM
Related
I love re-frame, but I realize that I'm having a bit of trouble finding a nice pattern for handling AJAX responses.
My situation is the following:
I have a "global" event handler that triggers some AJAX call and dispatches to some other global event handler, depending on whether that call was successful, e.g.:
(reg-event-db :store-value
(fn [db [_ value]]
(do-ajax-store value
:on-success #(dispatch [:store-value-success %])
:on-error #(dispatch [:store-value-error %])
db))
(reg-event-db :store-value-success
(fn [db [_ result]]
(assoc db :foobar result)))
(reg-event-db :store-value-error
(fn [db [_ error]]
(assoc db :foobar nil
:last-error error)))
(I am aware of reg-event-fx and stuff, I'm just avoiding it here for the sake of brevity and because I think it does not make any difference for my problem).
I also have (multiple, distinct) UI components that might trigger the :store-value event, like so:
(defn button []
(let [processing? (reagent/atom false)]
(fn button-render []
[:button {:class (when #processing? "processing")
:on-click (fn []
(reset! processing? true)
(dispatch [:store-value 123]))}])))
So in this case the component has local state (processing?) that is supposed to depend on whether the AJAX call is still in progress or not.
Now, what is the proper pattern here to have the button component react to the events :store-value-success and :store-value-error in order to reset the processing? flag back to false after the AJAX call has finished?
Currently, I'm working around that problem by passing down callbacks but that seems really ugly because it clutters the event handlers' code with stuff that does not really belong there.
The best solution that I've thought of would be to have the button component being able to hook into the :store-value-success and :store-value-error events and install its own handler for those events, like this:
(defn button []
(let [processing? (reagent/atom false)]
(reg-event-db :store-value-success
(fn [db _]
(reset! processing? false)))
(reg-event-db :store-value-error
(fn [db _]
(reset! processing? false)))
(fn button-render []
[:button {:class (when #processing? "processing")
:on-click (fn []
(reset! processing? true)
(dispatch [:store-value 123]))}])))
However, that does not work. As it seems, re-frame does not allow multiple event handlers per event. Instead, a subsequent invocation of reg-event-db on one single event id will replace the previous event handler.
So how do you guys handle situations like this?
I think reg-event-fx (src) might indeed help solve your problem.
You could add a subscription that watches app-state e.g.
(rf/reg-sub
:app-state
(fn [db]
(get db :app-state)))
and add this to your button, perhaps with a state function e.g.
(defn btn-state [state]
(if (= state :processing)
"processing"))
And then in the AJAX handler, you could add an fx to update the state-
(reg-event-fx ;; -fx registration, not -db registration
:ajax-success
(fn [{:keys [db]} [_ result]]
{:db (assoc db :app-state :default)
:dispatch [:store-value-success result]}))
(reg-event-fx ;; -fx registration, not -db registration
:ajax-error
(fn [{:keys [db]} [_ result]]
{:db (assoc db :app-state :default)
:dispatch [:store-value-error result]}))
and update the AJAX handler
(reg-event-db :store-value
(fn [db [_ value]]
(do-ajax-store value
:on-success #(dispatch [:ajax-success %])
:on-error #(dispatch [:ajax-error %])
db))
This would be one way to handle it via -fx. I think you have already started to see the need for tracking app state, and I think bumping it up into the subscriptions would help with complexity, at which point your button render is greatly simplified.
(defn button []
[:button {:class (btn-state #app-state)
:on-click (dispatch [:store-value 123]))}])))
As others have mentioned, I would recommend to use http-fx and make processing? part of your global state. The code would look like this:
Events:
(reg-event-fx
:request
(fn [{:keys [db]} [_ method url data]]
{:http-xhrio {:method method
:uri url
:params data
:format (ajax/json-request-format)
:response-format (ajax/json-response-format {:keywords? true})
:on-success [:success-response method url]
:on-failure [:error-response method url]}
:db (assoc db :processing? true)}))
(reg-event-db
:success-response
(fn [db [_ method url result]]
(assoc db :foobar response
:processing? false)}))
(reg-event-db
:error
(fn [db [_ method url result]]
(assoc db :foobar nil
:last-error result
:processing? false)}))
Subscriptions:
(reg-sub
:processing
(fn [db _]
(:processing? db)))
View:
(defn button []
(let [processing? #(rf/subscribe [:processing])]
[:button {:class (when processing? "processing")
:on-click #(dispatch [:store-value 123]))}])))
Hint: You could reuse this code with all your requests.
I have a clojure code(riemann) to send an email if certain condition was met. I am facing some issue while passing the event to riemann server.
Riemann code
(let [email (mailer {"......"})]
(streams
(where (service "system_log")
(by :RefNo
(smap
(fn [events]
(let [count-of-failures (count (filter #(= "Failed" (:Status %)) events))]
(assoc (first events)
:status "Failure"
:metric count-of-failures
:total-fail (>= count-of-failures 2))))
(where (and (= (:status event) "Failure")
(:total-fail event))
(email "XXXXX#gmail.com"))prn)))))
O/P in riemann server
WARN [2015-11-18 05:24:49,596] defaultEventExecutorGroup-2-2 - riemann.streams - riemann.streams$smap$stream__3695#7addde9e threw
java.lang.IllegalArgumentException: Key must be integer
at clojure.lang.APersistentVector.assoc(APersistentVector.java:335)
at clojure.lang.APersistentVector.assoc(APersistentVector.java:18)
Update 2:
I simply changed the smap to sreduce. How I should update, since I am newbie to this I am little bit confused about altering the code as per your suggestion
(let [email (mailer {"......"})]
(streams
(where (service "system_log")
(by :RefNo
(sreduce
(fn [events]
(let [count-of-failures (count (filter #(= "Failed" (:Status %)) events))]
(assoc (first events)
:status "Failure"
:metric count-of-failures
:total-fail (>= count-of-failures 2))))
(where (and (= (:status event) "Failure")
(:total-fail event))
(email "XXXXX#gmail.com"))prn)))))
Update 3:
I have updated my code using coalesce and smap has its child. Now its not showing any error but email didn't get triggered. I am getting count-of-failures as 0. I guess count function is not working.
(let [email (mailer {"......"})]
(streams
(where (service "system_log")
(by :RefNo
(coalesce
(smap
(fn [events]
(let [count-of-failures (count (filter #(= "Failed" (:status %)) events))]
(assoc (first events)
:status "Failure"
:metric count-of-failures
:total-fail (>= count-of-failures 2))))
(where (and (= (:status event) "Failure")
(:total-fail event))
(email "XXXXX#gmail.com"))))prn))))
Off the top of my hat, by accepts a vector not a symbol:
(by [:Refno] ...
As a side note, I recommend using REPL (e.g. https://github.com/aphyr/riemann/wiki/playing-with-the-REPL) so you can build your stream processing gradually while testing functions in the REPL. It worked great for me.
Update: I'm also not sure if you shouldn't nest the where inside smap because you're assigning "Failure" but the where runs in parallel to smap so unless I'm missing something, I think it won't see it.
Update 2: I've ran it through the REPL connected to Riemann like this:
(require '[riemann.streams :refer :all])
(def f (stream
(where (service "system_log")
(by :RefNo
(smap
(fn [events]
(let [count-of-failures (count (filter #(= "Failed" (:Status %)) events))]
(prn events)
(assoc (first events)
:status "Failure"
:metric count-of-failures
:total-fail (>= count-of-failures 2))))
#_(where (and (= (:status event) "Failure")
(:total-fail event)))
prn)))))
(f {:RefNo 4444 :service "system_log" :status "Failed"})
It produces the same error that you've got. The error is there because you're assuming that the function passed to smap receives a list of events. It doesn't, it receives a single event (see the prn there). Calling first on a hashmap produces a vector, then trying to assoc using a symbol as a key gives you the error because vectors support only integers.
You cannot count failures this way just like you wouldn't use a regular map in Clojure for this purpose because you need past events.
Here's what I think might be compatible with your smap example.
Either:
Use coalesce http://riemann.io/api/riemann.streams.html#var-coalesce and smap as its child; I think smap will receive a list of events just like you wanted originally. I haven't tried it but there's no reason it shouldn't work.
You can control the time window you need (let's say max 2 failures per hour) by sending events with 1 hour TTL and querying the index within the stream. Here's a complete example: http://riemann.io/howto.html#query-the-index-from-within-a-stream
Apart from the that, I believe :Status should be in lower case. I hope it helps.
I am trying to get into ClojureScript and Om. There is a specific case which has me running in circles.
I have a component that is first rendered without a key.
(defn model-view [data owner]
(reify
om/IWillMount
(will-mount [_]
(om/transact! data [:stats] (fn [] {}))
(go
(let [response ((<! (api/get-stats (data :id))) :body)
stats (:stats response)]
(om/update! data [:stats] stats))))
om/IRender
(render [_]
(dom/div nil
(dom/h3 nil (data :title))
;; Here I want to use the :stats key in data that I
;; queried for in IWillMount, but its not present
;; the first time this model is rendered. It's only present
;; AFTER IWillMount has ran.
(om/build model-stats-view (data :stats)))))
The first time this component is called, the :stats key is simply not present in data. That's why I do an API call to get its stats. But React still calls the render function, thus the component crashes.
How can I set an initial state in this component that gives data an empty map called :stats, thus preventing trying to render nil in the (om/build model-stats-view) call?
I prefer to do all of my initialization in init-state, then access it in render-state. And I put a go-loop in my did-mount. When you update your init-state (i.e. :e-map) in the go-loop, it forces a call to render/re-render of the component. I use this in all of my components for inter-component/intra-component messaging. Just push something into a pub/sub channel and we are off to the races.
;To update my state I use a function:
(defn set-owner-state! [owner old-map-key old-map new-map]
(om/set-state! owner {old-map-key (merge old-map new-map)}))
om/IInitState
(init-state [_]
(println "queue->init-state")
{:e-map {:active-fsm nil}})
om/IDidMount
(did-mount [_]
(go-loop []
(let [[v _] (alts! [server-fsm-events dispatcher-events])
current-state (om/get-state owner)
e-map (:e-map current-state)]
; what goes in here is the logic to respond to a message
; in my case I have a match, it could be a cond or a set of
; if's.
(set-owner-state! owner :e-map e-map {:active-fsm :active :task-paths nil})
...
om/IRenderState
(render-state [_ {:keys [e-map]}]
(println "e-map:" e-map)
...
I want to find the string in sentence using clojure in riemann. I have wrote the code using re-matches but I am facing some error while executing it.
(smap
(fn [events]
(let [count-of-failures (count (re-matches #("POST*" (:Webservice %)) events))]
This is the error I got
java.lang.ClassCastException: riemann.config$eval96$fn__97$fn__98 cannot be cast to java.util.regex.Pattern
at clojure.core$re_matcher.invoke(core.clj:4460)
at clojure.core$re_matches.invoke(core.clj:4497)
at riemann.config$eval96$fn__97.invoke(riemann_v1.config:31)
at riemann.streams$smap$stream__3695.invoke(streams.clj:161)
at riemann.streams$fixed_time_window_fn$stream__3946$fn__3979.invoke(streams.clj:381)
at riemann.streams$fixed_time_window_fn$stream__3946.invoke(streams.clj:381)
at riemann.config$eval96$stream__145$fn__150.invoke(riemann_v1.config:25)
at riemann.config$eval96$stream__145.invoke(riemann_v1.config:25)
at riemann.core$stream_BANG_$fn__5678.invoke(core.clj:19)
at riemann.core$stream_BANG_.invoke(core.clj:18)
at riemann.transport$handle.invoke(transport.clj:159)
at riemann.transport.tcp$tcp_handler.invoke(tcp.clj:93)
at riemann.transport.tcp$gen_tcp_handler$fn__5904.invoke(tcp.clj:65)
at riemann.transport.tcp.proxy$io.netty.channel.ChannelInboundHandlerAdapter$ff19274a.channelRead(Unknown Source)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
at io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
I often use a pattern like this for such things
(fixed-time-window 10
(smap
(fn [events]
(let [count-of-failures
(count (filter #(re-find #"POST*"
(:Webservice %))
events))]
(assoc (first events)
:service "failure-counts"
:metric count-of-failures))))
index
slack
prn
divine-intervention))
the top call to fixed-time-window takes in events one a time such as:
{:service "foo"} {:service "foo"} {:service "foo"}
and bundles them into groups by time:
[{:service "foo"} {:service "foo"}] [{:service "foo"}]
there is one child stream of that bundler which takes each group and counts things then passes them to it's child streams single events that look like this:
{:service "foo" :metric 2} ;; there where two in the first 10 second window
{:service "foo" :metris 1} ;; there was one in the second 10 second window.
the streams coming out of that are then passed to each of the child streams. In my example I called the example child streams:
index , a build in function that puts things into the index for the dash etc.
slack which is a function who's definition is not given here that forwads events to a slack channel. It's organizaion specific (channel names, tokesn etc) so there is no point in posting it.
prn: prints to the console for debugging
divine-intervention: another function, who's implementation is left as an exercise, which miraculously flips the appropriate bits in all systems to make the problem go away ;-)
I have to check the number of count appearing in an event at each interval of every 30 seconds. If the count is greater than 5 means, I need to trigger an email.
I am using the below code, but email didn't get triggered.
(let [userindex1 (default :ttl 300 (update-index (index)))]
(streams
prn
userindex1))
(streams
(where (and (service "system_log")
(not (expired? event)))
; fixed-time-window sends a vector of events out every 30 seconds
(fixed-time-window
30
; smap passes those events into a function
(smap
(fn [events]
;Calculate the no of count of events for failure
(let [numberofFailure (count (filter #(="IE" (:description %)) events))]
{:status "login failures"
:metric numberofFailure
:totalFail (boolean(numberofFailure > 5))}
(streams
prn
numberofFailure))))
;check if the variable status is true if condition satisfied then trigger an email
(let [email (mailer {:host "smtp.gmail.com"
:port 25
:user "aaaaa"
:pass "bbbbb"
:auth "true"
:subject (fn [events]
(clojure.string/join ", "
(map :service events)))
:from "abc#gmail.com"})]
(streams
(where (and (:status "login failures")
(:totalFail true))
(email "123#gmail.com")))))))
Where am I going wrong?
There are a couple of issues here. I'll try to address some of them, then post a minimal working example:
The first fn passed to smap should return an event. That event can be created with event or by assoc'ing into one of the received events. In your sample a plain map is created (which would not work, it's not a proper event), but that's even lost because then streams is called (which AFAIK should only be called at the top level). So instead of:
(smap
(fn [events]
(let [numberofFailure ...]
{:status "login failures"
:metric numberofFailure
:totalFail (boolean ...)}
(streams
prn
numberofFailure)))
...)
You should do something like:
(smap
(fn [events]
(let [numberofFailure ...]
(event {:status "login failures"
:metric numberofFailure
:totalFail (boolean ...)}))
...)
To calculate totalFail remember that you need to use prefix notation to call >, so it must be (> totalFail 5). And boolean is not needed, as > will already return a boolean.
I would initialize the mailer out of the top-level streams call, as an enclosing scope using let or with a def. But it should work as it is.
You should pass the last where as a children stream to smap, so it must be the second argument to smap. Let's recall the smap docs:
(smap f & children)
Streaming map. Calls children with (f event), whenever (f event) is non-nil.
Prefer this to (adjust f) and (combine f). Example:
(smap :metric prn) ; prints the metric of each event.
(smap #(assoc % :state "ok") index) ; Indexes each event with state "ok"
The last where should not be enclosed by streams, and the and sentence must work on the event, so it must be:
(where (and (= (:status event) "login failures")
(:total-fail event))
(email "123#gmail.com"))
The :subject fn for mailer should be passed as part of a second map, as explained in the mailer documentation
There's an open issue on fixed-time-window which makes it a bit unreliable: it doesn't fire as soon as the time window is due but waits until a new event is fired, so you might want to use a different windowing strategy until that get's fixed.
Here goes a full minimal working example based on yours:
(let [email (mailer {:host "localhost"
:port 1025
:from "abc#gmail.com"})]
(streams
(where (and (service "system_log")
(not (expired? event)))
(fixed-time-window
5
(smap
(fn [events]
(let [count-of-failures (count (filter #(= "IE" (:description %)) events))]
(event
{:status "login failures"
:metric count-of-failures
:total-fail (>= count-of-failures 2)})))
(where (and (= (:status event) "login failures")
(:total-fail event))
(email "hello123#gmail.com")))))))