Event count at certain time interval in riemann - clojure

Question

I have to check the number of count appearing in an event at each interval of every 30 seconds. If the count is greater than 5 means, I need to trigger an email.
I am using the below code, but email didn't get triggered.
(let [userindex1 (default :ttl 300 (update-index (index)))]
(streams
prn
userindex1))
(streams
(where (and (service "system_log")
(not (expired? event)))
; fixed-time-window sends a vector of events out every 30 seconds
(fixed-time-window
30
; smap passes those events into a function
(smap
(fn [events]
;Calculate the no of count of events for failure
(let [numberofFailure (count (filter #(="IE" (:description %)) events))]
{:status "login failures"
:metric numberofFailure
:totalFail (boolean(numberofFailure > 5))}
(streams
prn
numberofFailure))))
;check if the variable status is true if condition satisfied then trigger an email
(let [email (mailer {:host "smtp.gmail.com"
:port 25
:user "aaaaa"
:pass "bbbbb"
:auth "true"
:subject (fn [events]
(clojure.string/join ", "
(map :service events)))
:from "abc#gmail.com"})]
(streams
(where (and (:status "login failures")
(:totalFail true))
(email "123#gmail.com")))))))
Where am I going wrong?

Answer 1

There are a couple of issues here. I'll try to address some of them, then post a minimal working example:
The first fn passed to smap should return an event. That event can be created with event or by assoc'ing into one of the received events. In your sample a plain map is created (which would not work, it's not a proper event), but that's even lost because then streams is called (which AFAIK should only be called at the top level). So instead of:
(smap
(fn [events]
(let [numberofFailure ...]
{:status "login failures"
:metric numberofFailure
:totalFail (boolean ...)}
(streams
prn
numberofFailure)))
...)
You should do something like:
(smap
(fn [events]
(let [numberofFailure ...]
(event {:status "login failures"
:metric numberofFailure
:totalFail (boolean ...)}))
...)
To calculate totalFail remember that you need to use prefix notation to call >, so it must be (> totalFail 5). And boolean is not needed, as > will already return a boolean.
I would initialize the mailer out of the top-level streams call, as an enclosing scope using let or with a def. But it should work as it is.
You should pass the last where as a children stream to smap, so it must be the second argument to smap. Let's recall the smap docs:
(smap f & children)
Streaming map. Calls children with (f event), whenever (f event) is non-nil.
Prefer this to (adjust f) and (combine f). Example:
(smap :metric prn) ; prints the metric of each event.
(smap #(assoc % :state "ok") index) ; Indexes each event with state "ok"
The last where should not be enclosed by streams, and the and sentence must work on the event, so it must be:
(where (and (= (:status event) "login failures")
(:total-fail event))
(email "123#gmail.com"))
The :subject fn for mailer should be passed as part of a second map, as explained in the mailer documentation
There's an open issue on fixed-time-window which makes it a bit unreliable: it doesn't fire as soon as the time window is due but waits until a new event is fired, so you might want to use a different windowing strategy until that get's fixed.
Here goes a full minimal working example based on yours:
(let [email (mailer {:host "localhost"
:port 1025
:from "abc#gmail.com"})]
(streams
(where (and (service "system_log")
(not (expired? event)))
(fixed-time-window
5
(smap
(fn [events]
(let [count-of-failures (count (filter #(= "IE" (:description %)) events))]
(event
{:status "login failures"
:metric count-of-failures
:total-fail (>= count-of-failures 2)})))
(where (and (= (:status event) "login failures")
(:total-fail event))
(email "hello123#gmail.com")))))))