I have been struggling with signing requests for private GDAX endpoints for a while. Everything I have tried results in a 400 response with a message of "invalid signature." I have read their documentation on the matter several times, which can be found here. My current code is below. I'm using clj-http for making requests. I'm using their /time endpoint response for the timestamp, and I'm using pandect for the sha256 HMAC generation. I've tried converting the secret-decoded to a string using String. before passing it to sha256-hmac. I've also examined the request using clj-http's debug flag. It looks to me that I am following their directions precisely, but something must be wrong. I've done a lot of online searching before posting here. Any help would be greatly appreciated.
(defn get-time
[]
(-> (str (:api-base-url config) "/time")
(http/get {:as :json})
:body))
(defn- create-signature
([timestamp method path]
(create-signature timestamp method path ""))
([timestamp method path body]
(let [secret-decoded (b64/decode (.getBytes (:api-secret config)))
prehash-string (str timestamp (clojure.string/upper-case method) path body)
hmac (sha256-hmac prehash-string secret-decoded)]
(-> hmac
.getBytes
b64/encode
String.))))
(defn- send-signed-request
[method path & [opts]]
(let [url (str (:api-base-url config) path)
timestamp (long (:epoch (get-time)))
signature (create-signature timestamp method path (:body opts))]
(http/request
(merge {:method method
:url url
:as :json
:headers {"CB-ACCESS-KEY" (:api-key config)
"CB-ACCESS-SIGN" signature
"CB-ACCESS-TIMESTAMP" timestamp
"CB-ACCESS-PASSPHRASE" (:api-passphrase config)
"Content-Type" "application/json"}
:debug true}
opts))))
(defn get-accounts []
(send-signed-request "GET" "/accounts"))
(send-signed-request "GET" "/accounts")))
I've figured out the issue. Just in case anyone happens to have this very specific problem, I'm posting the solution. My error was that I was using the sha256-hmac function from pandect, which returns a string hmac, then I was converting that to a byte array, base64 encoding it, and converting it back to a string. Somewhere in those conversions, or perhaps in the pandect function's conversion, the value is altered in an erroneous way.
What works is using the sha256-hmac* function (note the asterisk) from pandect, which returns a raw byte array hmac, then base64 encoding that result directly, and converting it to a string. Below is the corrected, working code snippet, with which I was able to make a request to a private GDAX endpoint.
(defn create-signature
([timestamp method path]
(create-signature timestamp method path ""))
([timestamp method path body]
(let [secret-decoded (b64/decode (.getBytes (:api-secret config)))
prehash-string (str timestamp (clojure.string/upper-case method) path body)
hmac (sha256-hmac* prehash-string secret-decoded)]
(-> hmac
b64/encode
String.))))
Related
So I use re-graph version 0.1.11 and I try fetching data from the endpoint. After fetching data, I checked network tab in the browser and I found the expected data after that it should activate my callback function, but it doesn't seem to work (but sometimes it works and after refreshing the page a few times it doesn't work again). Here is the code.
;; how I init re-graph
(rf/dispatch [::re-graph/init
::conf/graphql-client-name
{:ws-url url
:http-url url
:ws-reconnect-timeout 500
:resume-subscriptions? true}])
(re-frame.core/reg-event-fx
::fetch-expected-data
(fn [cofx event]
(let [app-db (:db cofx)
some-params (-> event second (cljs.core/js->clj :keywordize-keys true))
token (-> app-db (lens/get-in (auth-db/lens-token :group-level-x)))]
(re-frame.core/dispatch
[:re-graph.core/query
::conf/graphql-client-name
"query findExpectedData($query: FetchExpectedDataInput!, $token: String!) {
findExpectedData(query: $query, token: $token){
value1
value2
...
}
}"
{:query some-params
:token token}
;; this is where the problem occurs
;; even though i found the data in the network tab, but
;; this callback doesn't seem to work (sometimes it works sometimes it doens't)
[::fetched-data-completed]]))))
(re-frame.core/reg-event-fx
::fetched-data-completed
(fn [cofx [_ {:keys [data errors] :as payload}]]
(let [app-db (:db cofx)
error-message (-> errors :errors first :message)]
(if (or (nil? errors) (empty? errors))
(do (bla bla when success))
(pr error-message)))))
I'm stuck with this problem for a few months. maybe because I fetch a lot of data at the same time? or could be something else anyone knows?. By the way the actual code I use defmacro, but it works the same way as the above code.
So I managed to find the answer to my own question. It seems like app-db has not been initialized properly so I fixed that problem and everything works fine. hope it helps someone who struggle with this problem.
Most examples for JWT token use clj-time which is now deprecated in favor of native java.time. I'm trying to use java-time along with buddy to sign/verify tokens but I'm stuck trying to pass the exp claim to my token. Here's an example of what I have:
(ns test-app.test-ns
(:require
[buddy.sign.jwt :as jwt]
[buddy.auth.backends.token :as bat]
[java-time :as t]))
(def secret "myfishysecret")
(def auth-backend (bat/jws-backend {:secret secret
:options {:alg :hs512}}))
(def token (jwt/sign {:user "slacker"
:exp (t/plus (t/local-date-time) (t/seconds 60))
} secret {:alg :hs512}))
When tyring to test if I can unsign the token
(jwt/unsign token secret {:alg :hs512})
I get the following error:
Execution error (JsonGenerationException) at
cheshire.generate/generate (generate.clj:152). Cannot JSON encode
object of class: class java.time.LocalDateTime:
2021-01-22T12:37:52.206456
So, I tried to pass the same by encapsulating the call to (t/plus ...) inside a (str) but then I get this error:
class java.lang.String cannot be cast to class java.lang.Number
(java.lang.String and java.lang.Number are in module java.base of
loader 'bootstrap')
So, I'm stuck since I don't really know how to generate a valid exp number value using java-time (according to this question, format should be in seconds since the epoch). Older examples using clj-time just passed the exp claim value as
(clj-time.core/plus (clj-time.core/now) (clj-time.core/seconds 3600))
Any help is highly appreciated.
EDIT: Alan Thompson's answer works perfectly, for what's worth this would be the equivalent using the java-time wrapper:
(t/plus (t/instant) (t/seconds 60))
Here are 2 ways to do it:
(let [now+60 (-> (Instant/now)
(.plusSeconds 60))
now+60-unixsecs (.getEpochSecond now+60)]
(jwt/sign {:user "slacker" :exp now+60 } secret {:alg :hs512})
(jwt/sign {:user "slacker" :exp now+60-unixsecs} secret {:alg :hs512}))
and we have the now results:
now+60 => <#java.time.Instant #object[java.time.Instant 0x7ce0054c "2021-01-22T19:04:51.905586442Z"]>
now+60-unixsecs => <#java.lang.Long 1611342291>
So you have your choice of methods. It appears that buddy knows how to convert from a java.time.Instant, so going all the way to unix-seconds is unnecessary.
You may also be interested in this library of helper and convenience functions for working with java.time.
I have the following code, in which I want to send an InputStream of a file in the function fetch-items, which handles the route /fetch-items.
(defn id->image [image-id]
(let [image (.getInputStream (gfs/find-by-id fs image-id))] image))
(defn item-resp [item]
(assoc item :_id (str (:_id item))
:images (into [] (map id->image (:image-ids item))))
)
(defn fetch-items [req]
(res/response
(map item-resp (find fs "items" {}))))
Here's my request in the client side, using cljs-ajax:
(ajax-request
{:uri "http://localhost:5000/fetch-items"
:method :get
:handler #(prn (into [] %))
:format (json-request-format)
:response-format (raw-response-format)
}
)
But the response I get on the client is this:
[:failure :parse] [:response nil] [:status-text "No reader function for tag object. Format should have been EDN"]
:original-text "{:_id \"5e63f5c591585c30985793cd\", :images [#object[com.mongodb.gridfs.GridFSDBFile$GridFSInputStream 0x22556652 \"com.mongodb.gridfs.GridFSDBFile$GridFSInputStream#22556652\"]]}{:_id \"5e63f5d891585c30985793d0\", :images [#object[com.mongodb.gridfs.GridFSDBFile$GridFSInputStream 0x266ae6c0 \"com.mongodb.gridfs.GridFSDBFile$GridFSInputStream#266ae6c0\"]]}{:_id \"5e63f5e891585c30985793d3\", ...
Why would the response say that the format should have been edn? How do I extract this file/image out in the client side?
--- EDIT ----
Doing the following:
(IOUtils/toString image "utf-8")
returns a string of size 1594 bytes, which is much smaller than the expected image size.
I think this is because it's converting the file object to base64 and not the actual chunk of data associated with it.
How do I make it convert the actual GridFS chunk to base64 string and not the file object?
It seems that you are building a response and directly putting an reference to an InputStream object into the response, without encoding the contents of the stream into an array of bytes and serializing the contents on the response.
You'll need to find a way to read the contents of the stream and encode it in the response (maybe send them encoded as base 64?)
On the other end, the client seems to be expecting an EDN response, and when it found the string #object, it complained that it didn't have a way to read an object with such a tag.
Here's a simple example of how to read an EDN string with a tagged literal, you can extend it so you decode the image in the client (note I'm using Java in the decoder, you'll need a different implementation on JS):
(defn b64decode [s]
(->> s .getBytes (.decode (java.util.Base64/getDecoder)) String.))
(def message "{:hello :world :msg #base64str \"SGV5LCBpdCB3b3JrcyE=\"}")
;; Now we can read the EDN string above adding our handler for #base64str
(clojure.edn/read-string {:readers {'base64str b64decode}} message)
;; => {:hello :world, :msg "Hey, it works!"}
I am using Rook framework for web services. I want to make API responses be pretty-printed. It seems that the response encoding is all handled by the wrap-restful-format function from ring.middleware.format. So I tried to replace the rook/wrap-with-standard-middleware function with my own version that passes different options through to ring.middleware.format.
(defn make-encoders-seq []
[(ring.middleware.format-response/make-encoder
(fn [s]
(json/generate-string s {:pretty true}))
"application/json")])
(defn wrap-with-standard-middleware-modified
[handler]
(-> handler
(ring.middleware.format/wrap-restful-format :formats [:json-kw :edn]
:response-options
[:encoders (make-encoders-seq)])
ring.middleware.keyword-params/wrap-keyword-params
ring.middleware.params/wrap-params))
(def handler (-> (rook/namespace-handler
["resource" 'my-app.resource])
(rook/wrap-with-injection :data-store venues)
wrap-with-standard-middleware-modified))
This compiles fine but it doesn't work to pretty print the responses, it seems like the custom encoder is never called.
Rook 1.3.9
ring-middleware-format 0.6.0
cheshire 5.4.0 (for json/generate-string in above)
Try to change your format/wrap-restful-format to:
(ring.middleware.format/wrap-restful-format :formats (concat (make-encoders-seq) [:edn])
I'm writing some middleware for Ring and I'm really confused as to why I have to reverse the order of the middleware.
I've found this blog post but it doesn't explain why I have to reverse it.
Here's a quick excerpt from the blog post:
(def app
(wrap-keyword-params (wrap-params my-handler)))
The response would be:
{; Trimmed for brevity
:params {"my_param" "54"}}
Note that the wrap keyword params didn't get called on it because the params hash didn't exist yet. But when you reverse the order of the middleware like so:
(def app
(wrap-params (wrap-keyword-params my-handler)))
{; Trimmed for brevity
:params {:my_param "54"}}
It works.
Could somebody please explain why you have to reverse the order of the middleware?
It helps to visualize what middleware actually is.
(defn middleware [handler]
(fn [request]
;; ...
;; Do something to the request before sending it down the chain.
;; ...
(let [response (handler request)]
;; ...
;; Do something to the response that's coming back up the chain.
;; ...
response)))
That right there was pretty much the a-ha moment for me.
What's confusing at first glance is that middleware isn't applied to the request, which is what you're thinking of.
Recall that a Ring app is just a function that takes a request and returns a response (which means it's a handler):
((fn [request] {:status 200, ...}) request) ;=> response
Let's zoom out a little bit. We get another handler:
((GET "/" [] "Hello") request) ;=> response
Let's zoom out a little more. We find the my-routes handler:
(my-routes request) ;=> response
Well, what if you wanted to do something before sending the request to the my-routes handler? You can wrap it with another handler.
((fn [req] (println "Request came in!") (my-routes req)) request) ;=> response
That's a little hard to read, so let's break out for clarity. We can define a function that returns that handler. Middleware are functions that take a handler and wrap it another handler. It doesn't return a response. It returns a handler that can return a response.
(defn println-middleware [wrapped-func]
(fn [req]
(println "Request came in!")
(wrapped-func req)))
((println-middleware my-route) request) ;=> response
And if we need to do something before even println-middleware gets the request, then we can wrap it again:
((outer-middleware (println-middleware my-routes)) request) ;=> response
The key is that my-routes, just like your my-handler, is the only named function that actually takes the request as an argument.
One final demonstration:
(handler3 (handler2 (handler1 request))) ;=> response
((middleware1 (middleware2 (middleware3 handler1))) request) ;=> response
I write so much because I can sympathize. But scroll back up to my first middleware example and hopefully it makes more sense.
The ring middleware is a series of functions which when stacked up return a handler function.
The section of the article that answers your question:
In case of Ring wrappers, typically we have “before” decorators that
perform some preparations before calling the “real” business function.
Since they are higher order functions and not direct function calls,
they are applied in reversed order. If one depends on the other, the
dependent one needs to be on the “inside”.
Here is a contrived example:
(let [post-wrap (fn [handler]
(fn [request]
(str (handler request) ", post-wrapped")))
pre-wrap (fn [handler]
(fn [request]
(handler (str request ", pre-wrapped"))))
around (fn [handler]
(fn [request]
(str (handler (str request ", pre-around")) ", post-around")))
handler (-> (pre-wrap identity)
post-wrap
around)]
(println (handler "(this was the input)")))
This prints and returns:
(this was the input), pre-around, pre-wrapped, post-wrapped, post-around
nil
As you may know the ring app is actually just a function that receives a request map and returns a response map.
In the first case the order in which the functions are applied is this:
request -> [wrap-keyword-params -> wrap-params -> my-handler] -> response
wrap-keyword-params looks for the key :params in the request but it's not there since wrap-params is the one who adds that key based on the "urlencoded parameters from the query string and form body".
When you invert the order of those two:
request -> [wrap-params -> wrap-keyword-params -> my-handler] -> response
You get the desired result since once the request gets to wrap-keyword-params, wrap-params has already added the corresponding keys.
The answer by danneu is nice, but it only really "clicked" for me after I visualized it in code to see how the chaining of middleware really looks like without the "->" threading macro magic (here's a link if you're not familiar with it). This is what I ended up with:
Let's say you have a request handler that looks like this:
(def amazing-handler
(-> #'some-amazing-fn
some-mware
another-mware
one-more-mware))
^^ The above handler, written without using a threading macro, would look like this (and I'm extending the indentation on purpose, so it is visually easier to understand which request belongs to which handler):
(def amazing-handler
(one-more-mware
(another-mware
((some-mware #'some-amazing-fn) request-from-another-mware)
request-from-one-more-mware)
original-request))
^^ The above is a style of code that requires us to read it from inside out (which sometimes is hard to follow), the threading macros (-> and ->>) allow us to read code in a natural left-to-right way, but it requires understanding on our part of how exactly it allows us to compose code in this "natural" way behind the scene.
Here's a more complete example:
;; For reference: this is how the end result of the entire "threading" looks like:
;; (((#'some-amazing-fn req-from-up-passed-down) req-from-up-passed-down) original-request)
(defn some-amazing-fn [req] ;; this "req" is the one that will get passed to this function from "some-mware"
(println "this is the final destination of the req", req)
(ring.util.http-response/ok {:body "some funny response"}))
(defn one-more-mware [some-argument] ;; the "some-argument" in this case is (another-mware (some-mware #'some-amazing-fn))
(fn [req] ;; the "req" here is the original request generated by the ring adaptors and passed to this chain of middleware
(println "|--> from fn inside one-more-mware")
(some-argument req))) ;; here we provide the another-mware with the request that it will then pass down the chain of middleware, you can imagine that chain, at this point in time, to look like this:
;; ((another-mware (some-mware #'some-amazing-fn)) req)
(defn another-mware [dunno-something] ;; the "dunno-something" in this case is (some-mware #'some-amazing-fn)
(fn [req] ;; the "req" here is passed from one-more-mware function
(println "|--> from fn inside another-mware")
(dunno-something req))) ;; here we are passing the "req" down the line to the (some-mware #'some-amazing-fn), so the entire thing behind the scenes, at this point in time, looks like this:
;; ((some-mware #'some-amazing-fn) req)
(defn some-mware [some-handler] ;; the "some-handler" arg here refers to #'some-amazing-fn
(fn [req] ;; the "req" here is passed to this function from another-mware function
(println "|--> from fn inside some-mware")
(some-handler req))) ;; here is where we are passing a "req" argument to the #'some-amazing-fn, so behind the scenes it could be thought of as looking like this:
;; (#'some-amazing-fn req)
(def amazing-handler
(-> #'some-amazing-fn
some-mware
another-mware
one-more-mware))
;; |--> from fn inside one-more-mware
;; |--> from fn inside another-mware
;; |--> from fn inside some-mware
;; |--> this is the final destination of the req {.. .. ..}