I'm new to riemann and clojure as well. what is want is, when new event comes in, it checks its state and service, if both things got match it print some thing in console. here is my config file.
(let [index (default :ttl 300 (update-index (index)))]
; Inbound events will be passed to these streams:
(streams
; Index all events immediately.
index
; Calculate an overall rate of events.
(with {:metric 1 :host nil :state "ok" :service "events/sec"}
(rate 5 index))
; my code starts from here
; _______________________
(where (and (= :service "foo")
(= :state "ok"))
(println "It works"))
;________ENDS HERE ______________
; Log expired events.
(expired
(fn [event] (info "expired" event)))
))
when I start riemann, I can see "It Works" in console. but never afterwards.
tell me where i'm doing wrong.?
The problem looks to be your use of keywords in the where expression. The where function will re-write the conditions to use keywords internally, though this behaviour doesn't appear to be clearly stated outside the API documentation. If you look at the example in the howto the conditions in where expressions don't have colons on field names.
I have tested the following config:
(where (and (service "foo")
(state "ok"))
prn)
Related
I have an app that reads CSV files and pushes it to BQ tables, checks for the status of that job before doing the next CSV file and so on. This was working fine while my datasets were in the US region, however we recently moved our datasets to Australia region and now I get
#error { :cause 404 Not Found { "code" : 404, "errors" : [ { "domain" : "global", "message" : "Not found: Job load-csv-job123", "reason" : "notFound" }
While I can run the job fine against this dataset but I cannot call the BQ get API in my Clojure code to get the status. While calling the insert job API I am setting the location in the jobReference
job-reference (doto (JobReference.) (.setLocation "australia-southeast1") (.setJobId job-id) )
and then call my insert like this
status (->> bq
(.jobs)
(#(.insert % project-id job-spec content))
(.execute)
(.getStatus))]
The status above works when I do (->> status (.getState)
I know I have to be setting the location somewhere for non-US/non-EU regions for the GET call on the job, but just can't figure how to from the Google Docs using the GET API.
https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/get
The API/jar I am using in the below code
[com.google.apis/google-api-services-bigquery "v2-rev459-1.25.0"]
The code I have for getting status in a loop with recur
(loop [status status] ;; Waiting until successfully processed
(log/info job-id " : " (->> status (.getState)))
(if (= "DONE" (->> status (.getState)))
(do (log/info "Status seems done?")
(if-let [errors (.getErrors status)]
(do
(log/info "seems like we have errors")
(vec (map #(.getMessage %) errors)))
nil))
(do
(log/info "status is pending let's wait and check...job spec" job-spec)
(Thread/sleep 3000)
(recur (->> bq
(.jobs)
(#(.get % project-id job-id))
(.execute)
(.getStatus))
))))))
Can you tell what I am missing, my attempts to setLocation on the .get
(#(.get % project-id job-id))(.setLocation "australia-southeast1")
comes back with
CompilerException java.lang.IllegalArgumentException: No matching field found: setLocation for class java.lang.String, compiling:```
The thing that's missing here appears to be the details about which clojure library you're using. That's not a language supported by first party libraries, so this may come down to how the library is assembled, and whether it's maintained.
For a jobs get call, the thing that's needed is for the request to route correctly is to add a location URL param to the request, e.g. GET https://bigquery.googleapis.com/bigquery/v2/projects/yourprojectid/jobs/yourjobid?location=australia-southeast1
I managed to figure out the Clojure code for how to set location when dataset has moved from one region to another, this is need to get the status of the jobs that are already run. Note that I had to make sure in my case that the previous BQ job(insert into table) has finished before I run the next one. I loop through the status and recur on it till I get a DONE. Note that DONE here doesn't mean success of the job, just that the job has finished. Which is why I fetch the err vec below if there were any errors and return that.
I was using a threaded form ->> for this initially but couldnt figure how to setLocation that way, so used a plain form now. I will update with a threaded form later.
(loop [status status] ;; Waiting until successfully processed
(log/info job-id " : " (->> status (.getState)))
(if (= "DONE" (->> status (.getState)))
(do (if-let [errors (.getErrors status)]
(do
(log/debug "Errors in job: " job-id)
(vec (map #(.getMessage %) errors)))
nil))
(do
(Thread/sleep 3000)
(recur
(let [jobsobj (.jobs bq)
getobj (.get jobsobj project-id job-id)
_ (.setLocation getobj "australia-southeast1")
]
(.getStatus (.execute getobj)))
))))
a new to Clojure here.
I would like to share a behaviour which seems strange to me, but may be it's totally ok.
I followed the tutorial on github https://gist.github.com/daveray/1441520#file-seesaw-repl-tutorial-clj-L381 , and more precisely the part where I am supposed to add a Listener to a Label. Let's make a constructor and display helper:
(defn make-lb [s]
(listbox :model (-> (symbol s) ns-publics keys sort)))
(defn display [content frame]
(config! frame :content content)
content)
This works perfectly:
(def lb (make-lb "clojure.core"))
(display (scrollable lb) f)
(listen lb :selection (fn [e] (println "Selection is " (selection e))))
Howevever, this doesn't:
(def lb (scrollable (make-lb "clojure.core")))
(display lb f)
(listen lb :selection (fn [e] (println "Selection is " (selection e))))
Notice the different "Scrollable" emplacement.
In the second case, compilier tells me "Unknown event type :selection seesaw.util/illegal-Argument (utils.clj:19)"
I don't see any reason why the first snippet works, and the second doesn't. I don't have any knowledge of Swing and/or other Java libraries
Why doesn't this work? (implied)
tl;dr
listbox and scrollable return different things
Details
Check out the return values of the different calls (original make-lb included for clarity):
(defn make-lb [s]
(listbox :model (-> (symbol s) ns-publics keys sort)))
(class (make-lb "clojure.core"))
;;=> seesaw.core.proxy$javax.swing.JList$Tag$fd407141
(class (scrollable (make-lb "clojure.core")))
;;=> seesaw.core.proxy$javax.swing.JScrollPane$Tag$fd407141
For our purposes, we'll just say that listbox returns a JList and scrollable returns a JScrollPane
Given that, the calls to display are equivalent
However, the calls to listen are not equivalent
In the first case, lb resolves to a JList, and in the second case, lb resolves to a JScrollPane
More details
If we look in the source for seesaw.event, we'd see the following:
; :selection is an artificial event handled specially for each class
; of widget, so we hack...
What I'll call a "real selection type" is resolved in resolve-event-aliases
You'll notice that there's a case for JList, but not for JScrollPane
In the JScrollPane case, the artificial :selection is simply handed back from the call to resolve-event-aliases
Since this isn't a "real selection type", it's only a matter of time before things go pear-shaped
And sure enough, get-or-install-handlers attempts to look up :selection, gets nothing back, and calls (illegal-argument "Unknown event type %s" event-name) where event-name is bound to :selection, which matches the exception that you were receiving
I'm new to Riemann and clojure. I'm trying to configure alerts based on changed states. But the states never seem to be updated/indexed. So when I get to the changed-state block, state is nil. I can add the alerts within the splitp block, but that seems redundant. Maybe we will want different types of notifications between critical and warnings, but for now, I'd like to see this work (if possible).
(let [index (default :ttl 20 (index))]
(streams
index
(where (not (state "expired"))
(where (service "load/load/shortterm")
(splitp < metric
0.05 (with :state "critical" index)
0.02 (with :state "warning" index)
(with :state "ok" index))
)
#(info %)
(changed-state {:init "ok"}
(stable 60 :state
#(info "CHANGED STATE" %)
(email "user#host.com")))
)
)
)
Thanks for any help!
Riemann v0.2.9, collectd v5.5.0, OS CentOS 6.5
I want to find the string in sentence using clojure in riemann. I have wrote the code using re-matches but I am facing some error while executing it.
(smap
(fn [events]
(let [count-of-failures (count (re-matches #("POST*" (:Webservice %)) events))]
This is the error I got
java.lang.ClassCastException: riemann.config$eval96$fn__97$fn__98 cannot be cast to java.util.regex.Pattern
at clojure.core$re_matcher.invoke(core.clj:4460)
at clojure.core$re_matches.invoke(core.clj:4497)
at riemann.config$eval96$fn__97.invoke(riemann_v1.config:31)
at riemann.streams$smap$stream__3695.invoke(streams.clj:161)
at riemann.streams$fixed_time_window_fn$stream__3946$fn__3979.invoke(streams.clj:381)
at riemann.streams$fixed_time_window_fn$stream__3946.invoke(streams.clj:381)
at riemann.config$eval96$stream__145$fn__150.invoke(riemann_v1.config:25)
at riemann.config$eval96$stream__145.invoke(riemann_v1.config:25)
at riemann.core$stream_BANG_$fn__5678.invoke(core.clj:19)
at riemann.core$stream_BANG_.invoke(core.clj:18)
at riemann.transport$handle.invoke(transport.clj:159)
at riemann.transport.tcp$tcp_handler.invoke(tcp.clj:93)
at riemann.transport.tcp$gen_tcp_handler$fn__5904.invoke(tcp.clj:65)
at riemann.transport.tcp.proxy$io.netty.channel.ChannelInboundHandlerAdapter$ff19274a.channelRead(Unknown Source)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
at io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
I often use a pattern like this for such things
(fixed-time-window 10
(smap
(fn [events]
(let [count-of-failures
(count (filter #(re-find #"POST*"
(:Webservice %))
events))]
(assoc (first events)
:service "failure-counts"
:metric count-of-failures))))
index
slack
prn
divine-intervention))
the top call to fixed-time-window takes in events one a time such as:
{:service "foo"} {:service "foo"} {:service "foo"}
and bundles them into groups by time:
[{:service "foo"} {:service "foo"}] [{:service "foo"}]
there is one child stream of that bundler which takes each group and counts things then passes them to it's child streams single events that look like this:
{:service "foo" :metric 2} ;; there where two in the first 10 second window
{:service "foo" :metris 1} ;; there was one in the second 10 second window.
the streams coming out of that are then passed to each of the child streams. In my example I called the example child streams:
index , a build in function that puts things into the index for the dash etc.
slack which is a function who's definition is not given here that forwads events to a slack channel. It's organizaion specific (channel names, tokesn etc) so there is no point in posting it.
prn: prints to the console for debugging
divine-intervention: another function, who's implementation is left as an exercise, which miraculously flips the appropriate bits in all systems to make the problem go away ;-)
I have the following function which gets a map with service name and threshold. It checks if the service crossed a defined threshold and then calls multiple downstream children on the event.
(defn tc
[s & children]
(where
(and (service (:service_name s)) (not (expired? event)))
(by [:host :service]
(where (> metric (:threshold s)
(with :state "critical"
(apply sdo children)))))))
I would like to build a stream dynamically using a vector of maps:
(def services [{:service "cpu/usage" :threshold 90}
{:service "memory/usage" :threshold 90}])
When trying to run it in a stream i'm getting the following warning:
(streams
(doseq [s services] (tc s prn)))
WARN [2015-01-05 14:27:07,187] Thread-15 - riemann.core - instrumentation service caught
java.lang.NullPointerException
at riemann.core$stream_BANG_$fn__11140.invoke(core.clj:19)
at riemann.core$stream_BANG_.invoke(core.clj:18)
at riemann.core$instrumentation_service$measure__11149.invoke(core.clj:57)
at riemann.service.ThreadService$thread_service_runner__8782$fn__8783.invoke(service.clj:66)
at riemann.service.ThreadService$thread_service_runner__8782.invoke(service.clj:65)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:701)
It works, if i run the streams function inside the doseq.
This one works and gives the following output:
(doseq [s services]
(streams (tc s prn)))
#riemann.codec.Event{:host "testhost", :service "memory/usage", :state "critical", :description nil, :metric 91.0, :tags nil, :time 1420460856, :ttl 60.0}
It seems to blow up if your events don't have all the required fields, here's a sample from a similar project where I build an event from a sequence of events (reducing) It's not exactly what you are doing though I'm generating events in the same way:
{:service (:service (first events))
:metric (->> events count)
:host "All-counts"
:state "OK"
:time (:time (last events))
:ttl default-interval}
I got NPE specifically when time was missing. If you can't inherit it form somewhere, just make it up (use now for instance) without a reasonable value here, event expiration will not work and you'll run out of RAM