The ClojureScript cljs-ajax client library converts {:b [1 2]} to b[0]=1&b[1]=2
For example:
(http/get "http://example.com" {:b [1 2]})
results in a request to:
"http://example.com?b[0]=1&b[1]=2"
How can I setup my ring middleware to handle this format on the server side? I would like to convert it back to the original structure:
{:b [1 2]}
I am using the middleware below, but it does not work properly:
(ring.middleware.keyword-params/wrap-keyword-params)
(ring.middleware.params/wrap-params :encoding encoding)
(ring.middleware.nested-params/wrap-nested-params)
There is no issue in middleware side. The issue is in cljs-ajax's ajax.core/params-to-str api. It is generating duplicate URL for different data format.
(ajax.core/params-to-str {:b [1 3]})
;; => "b[0]=1&b[1]=3"
(ajax.core/params-to-str {:b {0 1 1 3}})
;; => "b[0]=1&b[1]=3"
For array, format should be b[]=1&b[]=3.
I would suggest the middleware is working fine, but perhaps there is a misalignment between what it does and your expectations. I'm assuming that what you have above is just a list of the middleware and not how you are calling/using it. If not, your way off track.
Strictly speaking, what you are trying to pass is not a nested parameter. What you really have are parameters with names like "b[0]" and "b[1]", which each have a value. This is because you are using get rather than post and cljs-ajax needs to translate your clojure data structure to normal query parameter format. Unless there is a strong reason to do this, you will find life much easier if you use a post method rather than get and embed the data in the body as json/edn/transit whatever. It also has the added benefit that your data won't be sent 'public' as part of the URL and captured by logs all over the place.
A useful server side package to use with cljs-ajax and post commands is ring.middleware.format. This will simplify the parsing of the data in the body of your request and supports multiple different data encoding methods.
Related
I am getting the body and headers from the request like this:
(POST "/api/item" {body :body headers :headers} (create-item body headers))
The body is wrapped, so I get a keyword map and I can easily take values from the that:
(def app
(-> (handler/api app-routes)
(middleware/wrap-json-body {:keywords? true})
(middleware/wrap-json-response)))
As simple as:
(:item-name body)
How can I achieve the same with the headers, or just simply take a specific header value? Do I have to map the headers into a Clojure data structure first?
If I print headers I get something like this:
({host localhost:3000, user-agent Mozilla/5.0})
The headers are already in a Clojure data structure. If you want a better idea of the data types present, use prn instead of println, and you will see that it is a a hash-map with strings as keys.
(:foo x) is a shortcut for (get x :foo). For a hash-map with string keys you can get a value with eg. (get headers "host"). There is a function in clojure.walk, clojure.walk/keywordize-keys that will turn keys of a data structure into keywords, recursively through a nested structure. IMHO this is a bit silly, and one is better off using get and the string keys in most cases.
In my handler I have the following defined:
(def reg (new-registry))
(def app (-> app-routes
(expose-metrics-as-json)
(instrument reg)))
If I go to /metrics I only see an empty json object. What am I missing? Also, the documetation is not clear as to how I can report on the console, it seems report-to-console which is in the documentation is not used any more, instead there are reporters but I cannot find any documentation as to how you use reporters.
EDIT:
I have seen that metrics are being added and recorded by looking at this:
(meters/rates ((all-metrics reg) "ring.responses.rate.2xx"))
And the results are as expected, something like:
{1 0.06325196458537237, 5 0.01824839591203641, 15 0.006466051961211013, :total 6}
So it seems that although expose-metrics-as-json is creating an endpoint, the metrics are not getting added to it.
You are not passing the registry to expose-metrics-as-json. It falls back to the default registry.
In your threading expression, try changing the second line so that it reads:
(def app (-> app-routes
(expose-metrics-as-json "/metrics" reg)
(instrument reg)))
Ha! I found that this is actually a bug in the code. https://github.com/sjl/metrics-clojure/blob/master/metrics-clojure-ring/src/metrics/ring/expose.clj
([handler uri registry]
(expose-metrics-as-json handler uri default-registry {:pretty-print? false}))
This ignores the registry parameter. I got things to work by using
(expose-metrics-as-json "/metrics" reg {:pretty-print? false})
I will fix and create a pull request.
NOTE: I resolved my issue. However, it took a number of incremental changes. If you happen upon this page, feel free to checkout my github below to see how I made this application work.
I am using http-kit to post a request to btc-china. I want to use their trading api. I am able to do this just fine with python, but for some reason I keep getting 401s with clojure and http-kit. I've posted a snippit of code below which may show that I am not using http-kit correctly. In addition to that, here is a github for my full code if you wish to look at that: https://github.com/gilmaso/btc-trading
Here are the btc-china api docs: http://btcchina.org/api-trade-documentation-en
(def options {:timeout 2000 ; ms
:query-params (sorted-map :tonce tonce
:accesskey access-key
:requestmethod request-method
:id tonce
:method method
:params "")
:headers {"Authorization" auth-string
"Json-Rpc-Tonce" tonce}})
(client/post (str "https://" base-url) options
(fn [{:keys [status headers body error]}] ;; asynchronous handle response
(if error
(println "Failed, exception is " error)
(println "Async HTTP GET: " status))))
quoting from the example on the bttchina site:
# The order of params is critical for calculating a correct hash
clojure hash maps are unordered, and you cannot use a clojure hash map literal to provide the input if order is significant
I had very similar problem with bitstamp api. The solution was to replace :query-params with :form-params. Then the parameters are sent in the body. I noticed that in your api you are manually sending then in the body. It looks like using :form-params might help in your case as well.
I am trying to write Webdriver checks using Clojure. If I was using an object oriented language, I would use the Page Object Pattern. I think modeling a page as an object makes sense, I could create some java classes for the page objects and all would be well.
I want to know if there are any alternatives to the page object pattern using a functional style that maintain the same level of clarity.
A page (especially a RESTful one), can be thought of as a function from request to render (and, if you want to take the next step, the render exposes some set of new requests).
The translation from sending a request to page to applying a function to arguments is simple, and also quite comprehensive.
If you are providing a complex webapp, try taking a functional view of requests. GET can retrieve data, but should not modify server side state, use PUT to create a resource, use POST for mutation.
Once you write your controllers in this way, you can do quite a bit of testing without webdrivers. It should mostly suffice to provide a mockup of the request input to the controller, and verify some properties of the rendered result (for GET) or the storage state (for POST AND PUT).
I have even found it useful to break off request parsing and rendering into separate functions, in order to simplify the testing of the data processing that should happen in the middle, ie.:
(defn parse-home
[request]
(let [user-id (-> request :params :session :id)
account (get-user user-id)]
{:user-id user-id
:account account}))
(defn do-home
[user-id account]
(let [messages (get-messages account)
feed (generate-feed user-id)]
(update-user user-id :last-visited (java.util.Date.))
{:messages messages
:feed feed}))
(defn render-home
[request messages feed account]
(let [messages (mapv summarize (filter important messages))
feed (sort-by :priority feed)
template (-> request :page :template)]
(render template (assoc request :data {:messages messages :feed feed :account account}))))
(defn home
[request]
(let [{:keys [user-id account]} (parse-home request)
{:keys [messages feed]} (do-home user-id account)
body (render-home request messages feed account)]
{:status 200
:content-type "text/html"
:body body}))
Each element of the home page logic can be verified by providing one of these functions with the right input, and verifying some properties of the output. There is no need to mock up state or simulate page interaction unless you are also using clojurescript on the front end (and even in that case the logic you want to verify can be abstracted from the interface of the browser to avoid the need for replay testing).
I am trying to write a middleware for converting all the string object ids in the request to ObjectId objects.
I am achieving this using the following:
(defn get-object-id
[id]
(when (and (string? id) (re-matches object-id-regex id))
(ObjectId. id)))
(defn maybe-obj->object-id [obj]
(or (get-object-id obj) obj))
(defn- convert-string->object-ids [obj]
(cwalk/postwalk
(partial pcommon/maybe-obj->object-id) obj))
(defn warp-params-string->objectid
"convert strings to object ids"
[handler]
(fn [request]
(handler (update-in request [:params] convert-string->object-ids))))
This is working for all the params coming for json, request params etc. But this is not applying to the route params, e.g. :fst for url "/:fst". I looked at the GET macro and the route params are being injected somewhere inside that macro. However since GET/POST etc are executed last, my middlewares do not have access to these. Any graceful way of achieving this.
Those /:foo/:bar-style parameters get bound as a result of pattern matching on URIs, with the patterns specified in the individual routes' definitions. Outer layers don't even know what the patterns look like. So, not really possible to lift processing of these to middleware.
Instead, you could write a macro, say with-preprocessed-params, to wrap your route handlers' bodies in. If it ends up being useful in many handlers, you can additionally provide your own versions of GET & Co., delegating to Compojure's macros with the body wrapped in your param-processing macro.
That's not really a good solution if you were hoping to use the results of this preprocessing in further layers of middleware. In that case, assuming you're happy to leave matching actual URI path segments to the core handler layer, you can perform your preprocessing of other parameter types in a piece of middleware, then use your GET & Co. variants to preprocess the route parameters only.