I'm experiencing a strange problem with Java interop. I wrote a small specialized wrapper around the BDB JE API. When I'm in the repl (cider-repl) everything works fine: I can open the database, add entries, etc. But if I call the add-record function from another function in my app I get an NPE,
NullPointerException clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:26)
I'm starting to think I'm losing my mind. This opens the database:
(defn opendb [envpath dbname]
(let [envconf (.setAllowCreate (EnvironmentConfig.) true)
env (Environment. (ensure-directory envpath) envconf)
dbconf (-> (DatabaseConfig.) (.setAllowCreate true) (.setDeferredWrite true))
database (.openDatabase env nil dbname dbconf)]
{:environment env :database database}))
Here is add-record:
(defn add-record [dbm k v]
(let [key (DatabaseEntry. (.getBytes k "UTF-8"))
value (DatabaseEntry. (.getBytes v "UTF-8"))]
(.put (:database dbm) nil key value)))
There really isn't anything special going on here. The usage where this yields the NPE is similar to
(let [cache (bdb/opendb "local-cache" "subject-map")]
;; do stuff
(doseq [node (function "that returns a sequence of maps")]
(bdb/add-record cache (:foo node) (:bar node)))
(bdb/closedb cache))
I've added prnS inside bb/add-record and all the values are non-nil, yet I still get the NPE.
Something subtle is going on here, and it's eluding me. Anyone have an idea?
Thanks inadvance.
EDIT: here's the full stack trace:
Exception in thread "main" java.lang.NullPointerException
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:26)
at sterling.bdbje$add_record.invoke(bdbje.clj:33)
at sterling.loaders$load_subject_batch.invoke(loaders.clj:214)
at sterling.loaders$load_subject_headings.invoke(loaders.clj:224)
at sterling.loaders$_main.doInvoke(loaders.clj:257)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at sterling.loaders.main(Unknown Source)
As I said, this makes no sense to me, but is completely reproducible.
Related
So I use re-graph version 0.1.11 and I try fetching data from the endpoint. After fetching data, I checked network tab in the browser and I found the expected data after that it should activate my callback function, but it doesn't seem to work (but sometimes it works and after refreshing the page a few times it doesn't work again). Here is the code.
;; how I init re-graph
(rf/dispatch [::re-graph/init
::conf/graphql-client-name
{:ws-url url
:http-url url
:ws-reconnect-timeout 500
:resume-subscriptions? true}])
(re-frame.core/reg-event-fx
::fetch-expected-data
(fn [cofx event]
(let [app-db (:db cofx)
some-params (-> event second (cljs.core/js->clj :keywordize-keys true))
token (-> app-db (lens/get-in (auth-db/lens-token :group-level-x)))]
(re-frame.core/dispatch
[:re-graph.core/query
::conf/graphql-client-name
"query findExpectedData($query: FetchExpectedDataInput!, $token: String!) {
findExpectedData(query: $query, token: $token){
value1
value2
...
}
}"
{:query some-params
:token token}
;; this is where the problem occurs
;; even though i found the data in the network tab, but
;; this callback doesn't seem to work (sometimes it works sometimes it doens't)
[::fetched-data-completed]]))))
(re-frame.core/reg-event-fx
::fetched-data-completed
(fn [cofx [_ {:keys [data errors] :as payload}]]
(let [app-db (:db cofx)
error-message (-> errors :errors first :message)]
(if (or (nil? errors) (empty? errors))
(do (bla bla when success))
(pr error-message)))))
I'm stuck with this problem for a few months. maybe because I fetch a lot of data at the same time? or could be something else anyone knows?. By the way the actual code I use defmacro, but it works the same way as the above code.
So I managed to find the answer to my own question. It seems like app-db has not been initialized properly so I fixed that problem and everything works fine. hope it helps someone who struggle with this problem.
I am using Timbre as a logging library but I am having issues testing it with Midje. I've got the following method and all I'm trying to do is asserting that Will be printed is printed.
(defn init [level]
(timbre/merge-config! {:output-fn my-output-fn})
(timbre/set-level! level)
(timbre/info "Will be printed")
(timbre/debug "Won't be printed"))
By wrapping (timbre/default-output-fn) around a function of my own my-output-fn, I am able to assert that only 1 out of 2 logs are printed, which is true:
(fact "print info but not debug"
(core/init :info) => nil
(provided
(core/my-output-fn anything) => irrelevant :times 1))
However I would like to make sure that the message being printed is Will be printed. I can't find any way of doing this, what would you do?
I am doing experiments in the following project https://github.com/stephanebruckert/timbre-midje
It is possible to validate the input of a function using Midje's as-checker prerequisite.
When you print the data that come into timbre/default-output-fn (I discovered this via your output-fn) you see it has the following structure:
{:hash_ #delay[{:status :pending, :val nil} 0x5ed805b1], :instant #inst "2016-10-14T17:07:16.779-00:00", :config {:level :info, ....
So the log-level is available in data. With as-checker the log level can be validated as follows:
(fact "print info but not debug"
(core/init :info) => nil
(provided
(timbre/default-output-fn anything ; opts is first argument
(as-checker (fn [data]
(= (-> data :config :level) ; get log level
:info)))) ; validate that it is info
=> anything :times 1)
(provided
(timbre/default-output-fn anything
(as-checker (fn [data]
(= (-> data :config :level)
:debug))))
=> anything :times 0))
Now, you wanted to check the content of the message. Not only the log level.
The actual message is not available in the opts or data passed toward default-output-fn. Looking around in the the Timbre code I saw it is available in a private function vargs->margs that takes an ?err, a msg-type and vargs. The vargs contain the message (in your case what comes in is for example :auto :p ["The message"]).
Via the as-checker approach the messages can be validated follows:
(fact "print Will be printed, not Won't be printed"
(core/init :info) => nil
(provided
(#'timbre/vargs->margs anything anything ["Will be printed"]) => {} :times 1)
(provided
(#'timbre/vargs->margs anything anything ["Won't be printed"]) => {} :times 0) )
Note that vargs->margs has to return a map, otherwise Timbre throws an exception in a later function.
This way it is validated that "Will be printed" is printed once, and "Won't be printed" is never printed.
#ErwinRooijakkers' idea worked but #ptaoussanis on Github has a good reason not to do it.
Please note that timbre/vargs->margs is private, and an implementation
detail. Its behaviour can change at any time without notice, would
strongly recommend not depending on it in any way.
It'd be possible to use a custom appender that sends output somewhere that are handy for your tests to analyse. You could setup these appenders as part of your test setup, and/or use something like timbre/with-config for the appropriate test calls.
So we can add an appender that passes the parameters to check (level message) to a stub.
core.clj
(defn init
([level]
(init level {}))
([level config]
(timbre/merge-config! config)
(timbre/set-level! level)
(timbre/info "will be printed")
(timbre/debug "won't be printed")))
core_test.clj
(:require [timbre-midje.core :as core]
[midje.sweet :refer :all]))
(defn log-stub [level message])
(def log-stub-appender
{:appenders
{:test-appender
{:enabled? true
:fn (fn [data] (log-stub (:level data) (:vargs data)))}}})
(fact "print info but not debug"
(core/init :info log-stub-appender) => nil
(provided
(log-stub :info ["will be printed"]) => irrelevant :times 1
(log-stub :debug ["won't be printed"]) => irrelevant :times 0))
I have the following ClojureScript code to make a POST request:
(defn perform-post [resource]
"Performs a post and returns the body :)"
(go (let [response (<! (http/post resource))]
(:body response))))
When I make a call to a resource which returns a number
(js/console.log (perform-post post-create-recipe-url))
This prints:
bufObject { buf={...}, n=1, cljs$lang$protocol_mask$partition0$=2,
more...}
bufObject { head=1, tail=0, length=1, meer...}
arr
["6276677237104933520", undefined]
I want to obtain the "6276677237104933520" (the post body) information as a "return" value.
How can I accomplish this? I tried <!! but it does not work since it is not defined.
Blocking semantics (<!!) is not available on ClojureScript platform.
You can retrieve value from a channel only within go block:
(go (js/console.log (<! (perform-post post-create-recipe-url))))
I am using Clojure and Monger
It works fine, and I group functions by the collection they relate to.
Therefore, every file begins like this :
(ns img-cli.model.mycollectionname
(:require [monger.core :as mg]
[monger.collection :as mc]
[edn-config.core :refer [env]])
(:import [com.mongodb MongoOptions ServerAddress DB WriteConcern]
[org.bson.types ObjectId]))
(def config (get-in env [:mongo]))
;; using MongoOptions allows fine-tuning connection parameters,
;; like automatic reconnection (highly recommended for production
;; environment)
(def ^MongoOptions opts (mg/mongo-options { :threads-allowed-to-block-for-connection-multiplier 300}))
(def ^ServerAddress sa (mg/server-address (:url config) (:port config)))
(def conn (mg/connect sa opts))
(def db (mg/get-db conn (:db config)))
(def collection-name "asset")
;; I have to write one like this every time
(defn find-one-as-map
"fetch asset by Id"
[^String id]
(mc/find-one-as-map db collection-name {:_id (ObjectId. id)}))
Code duplication has of course several disadvantages in itself.
Also I'm not sure if connections are properly pooled afterwards ?
How can I avoid doing this ?
I sense I could pass an additional "db" parameter to each function, but then where would it come from ?
If I create the db connection in the "entry" file of my program, then how could it be passed to every function from there ?
For instance let's says I have Compojure routes in different files :
;; in the main handler file
(def db ...) ;; if I move the previous db configuration
;; in here, it could be the only place where this is set
;; importing Compojure routes from different files
(defroutes routes-from-file1
routes-from-file2...)
Let's say that some functions called from some of the routes in "file2" need access to the db, how can I pass this variable to them ?
I also have a lot of repetitive code afterwards, for instance to get data by Id for every collection...
I feel this could be simplified, but I'm not sure how.
Just refer to it by its namespace
(ns foo
(:require [handler :as h]))
(println h/db)
I'm writing some middleware for Ring and I'm really confused as to why I have to reverse the order of the middleware.
I've found this blog post but it doesn't explain why I have to reverse it.
Here's a quick excerpt from the blog post:
(def app
(wrap-keyword-params (wrap-params my-handler)))
The response would be:
{; Trimmed for brevity
:params {"my_param" "54"}}
Note that the wrap keyword params didn't get called on it because the params hash didn't exist yet. But when you reverse the order of the middleware like so:
(def app
(wrap-params (wrap-keyword-params my-handler)))
{; Trimmed for brevity
:params {:my_param "54"}}
It works.
Could somebody please explain why you have to reverse the order of the middleware?
It helps to visualize what middleware actually is.
(defn middleware [handler]
(fn [request]
;; ...
;; Do something to the request before sending it down the chain.
;; ...
(let [response (handler request)]
;; ...
;; Do something to the response that's coming back up the chain.
;; ...
response)))
That right there was pretty much the a-ha moment for me.
What's confusing at first glance is that middleware isn't applied to the request, which is what you're thinking of.
Recall that a Ring app is just a function that takes a request and returns a response (which means it's a handler):
((fn [request] {:status 200, ...}) request) ;=> response
Let's zoom out a little bit. We get another handler:
((GET "/" [] "Hello") request) ;=> response
Let's zoom out a little more. We find the my-routes handler:
(my-routes request) ;=> response
Well, what if you wanted to do something before sending the request to the my-routes handler? You can wrap it with another handler.
((fn [req] (println "Request came in!") (my-routes req)) request) ;=> response
That's a little hard to read, so let's break out for clarity. We can define a function that returns that handler. Middleware are functions that take a handler and wrap it another handler. It doesn't return a response. It returns a handler that can return a response.
(defn println-middleware [wrapped-func]
(fn [req]
(println "Request came in!")
(wrapped-func req)))
((println-middleware my-route) request) ;=> response
And if we need to do something before even println-middleware gets the request, then we can wrap it again:
((outer-middleware (println-middleware my-routes)) request) ;=> response
The key is that my-routes, just like your my-handler, is the only named function that actually takes the request as an argument.
One final demonstration:
(handler3 (handler2 (handler1 request))) ;=> response
((middleware1 (middleware2 (middleware3 handler1))) request) ;=> response
I write so much because I can sympathize. But scroll back up to my first middleware example and hopefully it makes more sense.
The ring middleware is a series of functions which when stacked up return a handler function.
The section of the article that answers your question:
In case of Ring wrappers, typically we have “before” decorators that
perform some preparations before calling the “real” business function.
Since they are higher order functions and not direct function calls,
they are applied in reversed order. If one depends on the other, the
dependent one needs to be on the “inside”.
Here is a contrived example:
(let [post-wrap (fn [handler]
(fn [request]
(str (handler request) ", post-wrapped")))
pre-wrap (fn [handler]
(fn [request]
(handler (str request ", pre-wrapped"))))
around (fn [handler]
(fn [request]
(str (handler (str request ", pre-around")) ", post-around")))
handler (-> (pre-wrap identity)
post-wrap
around)]
(println (handler "(this was the input)")))
This prints and returns:
(this was the input), pre-around, pre-wrapped, post-wrapped, post-around
nil
As you may know the ring app is actually just a function that receives a request map and returns a response map.
In the first case the order in which the functions are applied is this:
request -> [wrap-keyword-params -> wrap-params -> my-handler] -> response
wrap-keyword-params looks for the key :params in the request but it's not there since wrap-params is the one who adds that key based on the "urlencoded parameters from the query string and form body".
When you invert the order of those two:
request -> [wrap-params -> wrap-keyword-params -> my-handler] -> response
You get the desired result since once the request gets to wrap-keyword-params, wrap-params has already added the corresponding keys.
The answer by danneu is nice, but it only really "clicked" for me after I visualized it in code to see how the chaining of middleware really looks like without the "->" threading macro magic (here's a link if you're not familiar with it). This is what I ended up with:
Let's say you have a request handler that looks like this:
(def amazing-handler
(-> #'some-amazing-fn
some-mware
another-mware
one-more-mware))
^^ The above handler, written without using a threading macro, would look like this (and I'm extending the indentation on purpose, so it is visually easier to understand which request belongs to which handler):
(def amazing-handler
(one-more-mware
(another-mware
((some-mware #'some-amazing-fn) request-from-another-mware)
request-from-one-more-mware)
original-request))
^^ The above is a style of code that requires us to read it from inside out (which sometimes is hard to follow), the threading macros (-> and ->>) allow us to read code in a natural left-to-right way, but it requires understanding on our part of how exactly it allows us to compose code in this "natural" way behind the scene.
Here's a more complete example:
;; For reference: this is how the end result of the entire "threading" looks like:
;; (((#'some-amazing-fn req-from-up-passed-down) req-from-up-passed-down) original-request)
(defn some-amazing-fn [req] ;; this "req" is the one that will get passed to this function from "some-mware"
(println "this is the final destination of the req", req)
(ring.util.http-response/ok {:body "some funny response"}))
(defn one-more-mware [some-argument] ;; the "some-argument" in this case is (another-mware (some-mware #'some-amazing-fn))
(fn [req] ;; the "req" here is the original request generated by the ring adaptors and passed to this chain of middleware
(println "|--> from fn inside one-more-mware")
(some-argument req))) ;; here we provide the another-mware with the request that it will then pass down the chain of middleware, you can imagine that chain, at this point in time, to look like this:
;; ((another-mware (some-mware #'some-amazing-fn)) req)
(defn another-mware [dunno-something] ;; the "dunno-something" in this case is (some-mware #'some-amazing-fn)
(fn [req] ;; the "req" here is passed from one-more-mware function
(println "|--> from fn inside another-mware")
(dunno-something req))) ;; here we are passing the "req" down the line to the (some-mware #'some-amazing-fn), so the entire thing behind the scenes, at this point in time, looks like this:
;; ((some-mware #'some-amazing-fn) req)
(defn some-mware [some-handler] ;; the "some-handler" arg here refers to #'some-amazing-fn
(fn [req] ;; the "req" here is passed to this function from another-mware function
(println "|--> from fn inside some-mware")
(some-handler req))) ;; here is where we are passing a "req" argument to the #'some-amazing-fn, so behind the scenes it could be thought of as looking like this:
;; (#'some-amazing-fn req)
(def amazing-handler
(-> #'some-amazing-fn
some-mware
another-mware
one-more-mware))
;; |--> from fn inside one-more-mware
;; |--> from fn inside another-mware
;; |--> from fn inside some-mware
;; |--> this is the final destination of the req {.. .. ..}