i am trying to retrieve a website with Clojure and the clj-http library.
I wanted to start slow with a simple example:
(:body (client/get (str "http://www.google.com") {:as :clojure}))
As far as i understand the behaviour of the library, this call should return the body of the website but all it returns is <!doctype.
When i try to call
(:body (client/get (str "http://www.google.com") {:as :json}))
i even get an exception:
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60))
I can not imagine the library to be broken but also i am not able to see an obvious error in my call. Has anybody of you experienced this behaviour?
To get the http response body as a string you can use the following:
(:body (client/get "http://www.google.com"))
The :as entry in the {:as :clojure} options is output-coercion, and is trying to convert the HTML body, from google.com, into a Clojure data structure. This will fail unless the response body actually contains Clojure code.
If you are trying to parse the HTML response, you might need to look into an additional library, like Enlive.
The problem is you're querying a URL that is not returning the data type you're coercing the result to.
For instance if you try with http://ip.jsontest.com/ this url which returns a proper json:
(require '[clj-http.client :as client])
(client/get "http://ip.jsontest.com/" {:as :json})
=> {:trace-redirects ["http://ip.jsontest.com/"], :request-time 1153,
:status 200,
:headers {"access-control-allow-origin" "*", "content-type" "application/json; charset=ISO-8859-1", "date" "Tue, 22 Oct 2013 19:50:36 GMT", "server" "Google Frontend", "cache-control" "private", "alternate-protocol" "80:quic,80:quic", "connection" "close"}, :body {:ip "186.54.233.167"}}
Response is properly parsed.
Checking the response body you can easily see there's a json indeed there:
(:body (client/get "http://ip.jsontest.com/"))
=> "{\"ip\": \"186.54.233.167\"}\n"
Related
I'm using the last available lacinia version: "0.36.0-alpha-3" with Luminus (Ring+reitit), but this version asks for a specific header:
$ curl 'http://localhost:3000/api/graphql' -X POST --data "{test_by_id(id: 5) { title } }" -H 'Content-Type: application/graphql'
that request works fine, but without "'Content-Type: application/graphql'" the request wouldn't work. So I need to define my re-graph init vector like:
[::re-graph/init
{:ws-url nil
:http-url "http://localhost:3000/api/graphql"
:http-parameters {:with-credentials? false
:headers {"Content-Type" "application/graphql"}
}
:ws-reconnect-timeout nil
:resume-subscriptions? false
:connection-init-payload {}}]
but putting that header makes re-graph unable to work properly:
{"errors":[{"message":"Failed to parse GraphQL query.","extensions":{"errors":[{"locations":[{"line":1,"column":null}],"message":"mismatched input '\"query\"' expecting {'query', 'mutation', 'subscription',
it looks like re-graph sends and receives data using "application/json" header, so lacinia asks for some type of header but re-graph can't work with that option.
I had the same problem, and I think I got a solution for it. re-frame requests follows the Apollo Specification, as stated by #aarkerio. Here is the code to keep the original endpoint working with the origina specification, and allow it to respond to re-frame requests. This will make the endpoint respond to Graphiql request (from your http://localhost:3000/graphiql route), and re-graph ones. Any comments or corrections are welcomed.
Replace the original function set on the /graphql route on src/clj/mem_learning/routes/services.clj:
["/graphql" {:post graphql-call}
Add the graphql-call function on that same file:
(defn graphql-call [req]
(let [body (:body-params req)
content-type (keyword (get-in req [:headers "content-type"]))]
(case content-type
:application/json (ok (graphql/execute-request-re-graph body))
:application/graphql (ok (graphql/execute-request (-> req :body slurp))))))
add the execute-request-re-graph to the src/clj/mem_learning/routes/services/graphql.clj file:
(defn execute-request-re-graph
"execute request with re-graph/apollo format"
[{:keys [variables query context]}]
(lacinia/execute compiled-schema query variables context)))
ANSWER:
It looks that Luminus creates a middleware configuration:
(defn service-routes []
["/api"
{:coercion spec-coercion/coercion
:muuntaja formats/instance
:swagger {:id ::api}
:middleware [;; query-params & form-params
parameters/parameters-middleware
;; content-negotiation
muuntaja/format-negotiate-middleware
;; encoding response body
muuntaja/format-response-middleware
;; exception handling
exception/exception-middleware
;; decoding request body
muuntaja/format-request-middleware
;; coercing response bodys
coercion/coerce-response-middleware
;; coercing request parameters
coercion/coerce-request-middleware
;; multipart
multipart/multipart-middleware
]}
commenting the line "muuntaja/format-negotiate-middleware" makes the "application/json" call possible.
SECOND UPDATE (four hours later)
Ok, that muuntaja middleware thing was not the problem at all, the real problem is that curl send the data with the format:
{ test_by_id(id: 7, archived: false) { title } }
meanwhile re-graph uses:
{"query":"query { test_by_id(id: 7, archived: false) { title } }","variables":null}
this is a normal java string btw not a data structure, so we need to do some changes, first a new function:
(defn graphql-call [req]
(let [body (-> req :body slurp)
full-query (json/read-str body :key-fn keyword)
_ (log/info (str ">>> **** full-query >>>>> " full-query))]
(ok (graphql/execute-request full-query))))
we set the function:
["/graphql" {:post graphql-call}]
and in my_app.routes.services.graphql file:
(defn execute-request [{:keys [variables query context]}]
(json/write-str (lacinia/execute compiled-schema query variables context)))
and now re-graph works!
(also now I can send and use variables in GraphQL)
It's necessary to set:
:http-parameters {:with-credentials? false
:oauth-token "ah4rdSecr3t"
:headers {"Content-Type" "application/graphql"}
btw. Also, maybe it's better:
(lacinia/execute compiled-schema query variables context)
than:
(json/write-str (lacinia/execute compiled-schema query variables context))
because it interferes with re-graph importing the data already as a native ClojureScript map.
I have POSTed data to a Pedestal endpoint "/my-post. I have routed that end point as such:
[[["/" {:get landing} ^:interceptors [(body-params/body-params) ...]
["/my-post {:post mypost-handler}
....
So to my mind this means that the body-params interceptor will fire for /my-post too.
In mypost-handler I have:
(defn mypost-handler
[request]
****HOW TO ACCESS THEN FORM DATA HERE ****
)
How do I now access the form data here? I can see from printing the request that I have a #object[org.eclipse.jetty.sever.HttpInputOverHTTP..] which will clearly need further processing before it is useful to me.
(I must say, the documentation for Pedestal is pretty sketchy at best...)
Something like this should work. Note the body-params interceptor on the mypost-handler route
(defn mypost-handler
[{:keys [headers params json-params path-params] :as request}]
;; json-params is the posted json, so
;; (:name json-params) will be the value (i.e. John) of name property of the posted json {"name": "John"}
;; handle request
{:status 200
:body "ok"})
(defroutes routes
[[["/mypost-handler" {:post mypost-handler}
^:interceptors [(body-params/body-params)]
]
]])
The mypost-handler is acting as a Ring handler, i. e. it should accept a Ring request map and return a Ring response map. Thus, you can expect a typical Ring request structure:
(defn mypost-handler
[{:keys [headers params json-params path-params] :as request}]
;; handle request
{:status 200
:body "ok"})
Here's more relevant info on defining such handlers in your route tables.
I am writing a small website in Clojure and Compojure. I would like to set the HTTP response status for each request based on the data found or not found.
The last call is the html5 macro that returns to the handler the html that needs to be sent back to the browser. Is it possible to set the HTTP response status somehow here?
(ns myapp.views.layout
(:require
[hiccup.page :refer (html5 include-css include-js)]))
(defn layout [title & content]
(html5
(head title)
(body content)))
If you only return text that text will be the body of the response. If you return a map, the map can describe other aspects of the response.
(defn layout [title & content]
{:status 200
:body (html5 (head title) (body content))})
If you return a map containing
{:status NNN
:body (my-code-here)}
then the contents of the :status key will be the http response code.
Just to add some details that might be helpful or interesting to others, each of the return values of your Compojure route handlers "... is treated intelligently" and that intelligence is encapsulated in the "compojure.response/render multimethod".
Based on a cursory examination of the render code, the reason why returning a map works is that the map you return is merge-d with the Ring response map that Compojure implicitly creates.
You might also want to include :headers {"Content-Type" "text/html"} (or whatever's appropriate) in the map for your handler return values. A Unicode character in the page title in my responses wasn't being rendered correctly because the content type header was missing.
org.clojure/clojure-contrib "1.2.0"
ring "1.1.8"
compojure "1.1.5"
clout "1.1.0"
(defroutes rest-routes
(GET "/" [] "<p> Hello </p>")
(POST "/api/v1/:stor/sync" [stor] (start-sync stor))
(POST ["/api/v1/:stor/:txn/data/:file" :file #".*"] [stor txn file] (txn-add stor txn file))
(ANY "*" [] "<p>Page not found. </p>"))
In the second POST, I also want to pass all http-headers to "txn-add" handler. I did lot of google and look through the code, but couldn't find anything useful.
I know, I can use the following to pass headers (but then it doesn't parse url request),
(POST "/api/v1"
{headers :headers} (txn-add "dummy stor" "dummy txn" headers))
Also, how do I pass the content (i.e. :body) of POST request to "txn-add" ?
If the second argument to GET, POST etc is not a vector, it's a destructuring binding form for request. That means you can do things like:
(GET "/my/path"
{:keys [headers params body] :as request}
(my-fn headers body request))
To pick out the parts of request you want. See the Ring SPEC and Clojure's docs on binding & destructuring
The whole request map can be specified in the bindings using :as keyword in bindings and then used to read headers or body :
(POST ["/api/v1/:stor/:txn/data/:file" :file #".*"]
[stor txn file :as req]
(my-handler stor txn file req))
I have a clojure / compojure webapp with the following routes
(defroutes my-routes
(GET "/app/preview" request (my-preview-function request))
(ANY "*" request (str "ANY page <br>" (request :params))))
The preview GET request is made with a couple of parameters. I find this works most of the time but sometimes the /ebook/preview is not found and processing drops to the ANY route, in which case the output is similar to this,
ANY page
{:* "/app/preview", :section "50", :id "48"}
Can anyone suggest what might cause the /ebook/preview request to be skipped? It is definitely a GET request being made; the HTML does not have a POST for the /app/preview URL and to be doubly sure I added a POST route for /app/preview and that was not being hit.
JAR versions:
Clojure 1.2
compojure-0.6.2
ring-core-0.3.7
jetty-6.1.14
ring-jetty-adapter-0.3.1
ring-servlet-0.3.1jar
servlet-api-2.5-6.1.14
Routes are wrapped as follows
(require '[compojure.handler :as handler])
(defn wrap-charset [handler charset]
(fn [request]
(if-let [response (handler request)]
(if-let [content-type (get-in response [:headers "Content-Type"])]
(if (.contains content-type "charset")
response
(assoc-in response
[:headers "Content-Type"]
(str content-type "; charset=" charset)))
response))))
(def app (-> my-routes
handler/site
wrap-stateful-session
(wrap-charset "utf-8")
(wrap-file "public")))
(defn run []
(run-jetty (var app) {:join? false :port 8080}))
If you're trying to figure out what request is causing the problems, stop throwing away the request map with (request :params) and just have a look at request. That will give you a map with all the information Compojure has; you can inspect it, and pass it back into your routes later to observe what happens (after you make some changes, say).
If
(my-preview-function request)
returns nil, then the routing will try the next route. Take a look at (source GET) and see how it matches (or doesn't) your route.