How do I send a gzipped request using the dakrone/clj-http client? So far I have:
(http/post <<REDACTED>>
{:body (->> <<REDACTED>>
cheshire.core/generate-string
.getBytes
clj-http.util/gzip)
:content-type "application/json"
:content-encoding "gzip"
:as :json})
But elasticsearch (the server in my case) is giving 500 errors Illegal character ((CTRL-CHAR, code 31)): only regular white space.
Any ideas?
I guess that you need to enable HTTP compression on the server, e. g. in the Elasticsearch config:
http.compression: true
Related
How to get the content of an incoming POST http request's :body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x42c3599b "HttpInputOverHTTP#42c3599b"] in a Compojure/Ring project?
I know that this :body is composed of a part named data whose MIME-type is text-plain and another part named excel whose MIME-type is application/excel.
I slurped the content of :body and it shows:
Parsing a binary stream manually would be difficult. Wrap your handler as follows:
(wrap-multipart-params handler options)
This middleware parses the body and populates :params parameters with parsed data as well.
See ring.middleware.multipart-params documentation for more details.
I was seeing it in Reitit, what fixed for me was to change the order of the middlewares so the exception-middleware is after the multipart/multipart-middleware.
:middleware [;; multipart
multipart/multipart-middleware
;; exception handling
exception-middleware]
You can find a basic example in the Clojure Cookbook (O'Reilly), which I highly recommend:
(ns ringtest
(:require
[ring.adapter.jetty :as jetty]
clojure.pprint))
;; Echo (with pretty-print) the request received
(defn handler [request]
{:status 200
:headers {"content-type" "text/clojure"}
:body (with-out-str (clojure.pprint/pprint request))})
(defn -main []
;; Run the server on port 3000
(jetty/run-jetty handler {:port 3000}))
I am playing with compojure-api and am blocked at trying to manage Content-Type for my simple webapp. What I want is to emit an HTTP response that is just plain/text, but somehow Compojure-API keeps setting it to "application/json".
(POST "/echo" []
:new-relic-name "/v1/echo"
:summary "info log the input message and echo it back"
:description nil
:return String
:form-params [message :- String]
(log/infof "/v1/echo message: %s" message)
(let [resp (-> (resp/response message)
(resp/status 200)
(resp/header "Content-Type" "text/plain"))]
(log/infof "response is %s" resp)
resp))
but curl shows the server responded Content-Type:application/json.
$ curl -X POST -i --header 'Content-Type: application/x-www-form-urlencoded' -d 'message=frickin compojure-api' 'http://localhost:8080/v1/echo'
HTTP/1.1 200 OK
Date: Fri, 13 Jan 2017 02:04:47 GMT
Content-Type: application/json; charset=utf-8
x-http-request-id: 669dee08-0c92-4fb4-867f-67ff08d7b72f
x-http-caller-id: UNKNOWN_CALLER
Content-Length: 23
Server: Jetty(9.2.10.v20150310)
My logging shows that the function requested "plain/text", but somehow the framework trumped it.
2017-01-12 18:04:47,581 INFO [qtp789647098-46]kthxbye.v1.api [669dee08-0c92-4fb4-867f-67ff08d7b72f] - response is {:status 200, :headers {"Content-Type" "text/plain"}, :body "frickin compojure-api"}
How do I gain control over Content-Type in a Compojure-API Ring application?
compojure-api serves response in format requested by HTTP client which is indicated using HTTP Accept header.
With curl you need to add:
-H "Accept: text/plain"
You can also provide a list of acceptable formats and the server will serve the response in the first supported format from that list:
-H "Accept: text/plain, text/html, application/xml, application/json, */*"
I never tried compojure so here goes nothing:
1.) your local val reps has the same name as the aliased namespace - kind of confusing
2.) to get access to the params - it seems - you have to apply ring.middleware.params/wrap-params to your routes
3.) ah yes the Content-Type: since you required :form-params, which didn't get delivered due to missing wrap-params you ended up in some sort of default route - hence not text/plain. Thats what I think happend, at least.
with
lein try compojure ring-server
demo/paste into repl:
(require '[compojure.core :refer :all])
(require '[ring.util.response :as resp])
(require '[ring.server.standalone :as server])
(require '[ring.middleware.params :refer [wrap-params]])
(def x
(POST "/echo" [message]
:summary "info log the input message and echo it back"
:description nil
:return String
:form-params [message :- String]
(let [resp (-> (resp/response (str "message: " message))
(resp/status 200)
(resp/header "Content-Type" "text/plain"))]
resp)))
(defroutes app (wrap-params x))
(server/serve app {:port 4042})
test:
curl -X POST -i --header 'Content-Type: application/x-www-form-urlencoded' -d 'message=frickin' 'http://localhost:4042/echo'
HTTP/1.1 200 OK
Date: Fri, 13 Jan 2017 17:32:03 GMT
Content-Type: text/plain;charset=ISO-8859-1
Content-Length: 14
Server: Jetty(7.6.13.v20130916)
message: frickin
I'm using reagent and compojure to make a toy webapp and I can't figure out why my server isn't sending out a CSRF cookie. Other answers and several blog posts seem to imply that the default settings for compojure now send the CSRF token and that manually resending it is actually a bug. When I try to hit the POST /art endpoint I get back a 403 Forbidden response. None of the pages get the cookie with the CSRF token in it so I can't send it with the POST request. Any advice?
;;server.clj
(ns my-app.server
(:require [my-app.handler :refer [app]]
[environ.core :refer [env]]
[ring.adapter.jetty :refer [run-jetty]])
(:gen-class))
(defn -main [& args]
(let [port (Integer/parseInt (or (env :port) "3000"))]
(run-jetty app {:port port :join? false})))
;; handler.clj
(ns my-app.handler
(:require [compojure.core :refer [GET POST defroutes]]
[compojure.route :refer [not-found resources]]
[hiccup.page :refer [include-js include-css html5]]
[my-app.middleware :refer [wrap-middleware]]
[environ.core :refer [env]]))
(defroutes routes
(GET "/" [] loading-page)
(GET "/about" [] loading-page)
(GET "/art" [] loading-page)
(POST "/art" request {:sent (:body request) :hello "world"})
(resources "/")
(not-found "Not Found"))
(def app (wrap-middleware #'routes))
;;middleware.clj
(ns stagistry.middleware
(:require [ring.middleware.defaults :refer [site-defaults wrap-defaults]]
[prone.middleware :refer [wrap-exceptions]]
[ring.middleware.reload :refer [wrap-reload]]))
(defn wrap-middleware [handler]
(-> handler
(wrap-defaults site-defaults)
wrap-exceptions
wrap-reload))
I threw the code itself on github here since I still can't see what's wrong.
Other answers and several blog posts seem to imply that the default settings for compojure now send the CSRF token and that manually resending it is actually a bug.
(wrap-defaults site-defaults) applies the ring-anti-forgery middleware. This will only add a CSRF token to each ring browser session and look for the token on POST requests. If the token is missing the middleware will return a 403 for the request. Adding the token to your form or ajax/whatever post requests is up to you, the price of freedom. :)
From the ring-anti-forgery docs:
By default, the token is expected to be in a form field named '__anti-forgery-token', or in the 'X-CSRF-Token' or 'X-XSRF-Token' headers.
For example try adding this route:
(GET "/someform" request (html5 (ring.util.anti-forgery/anti-forgery-field)))
The anti-forgery-field helper will add a hidden input field with the CSRF token as its value, which is picked up by the middleware if the form is posted. To access the token directly you can either use ring.middleware.anti-forgery/*anti-forgery-token* or look it up in the session of the current request map:
(-> request :session :ring.middleware.anti-forgery/anti-forgery-token)
The global var (and by extension the helper) is bound to the handler context though, you can't access it from outside or from another thread in the same context.
Simple curl header example:
Get the CSRF token:
$ curl -v -c /tmp/cookiestore.txt http://localhost:3000/someform
Set the token via header and post some stuff:
$ curl -v -b /tmp/cookiestore.txt --header "X-CSRF-Token: ->token from prev. req<-" -X POST -d '{:foo "bar"}' localhost:3000/art
So I found an answer on a related question that's good enough for me at the moment. I switched the middleware.clj site-defaults to api-defaults which doesn't use CSRF. Still curious how to make CSRF work here, but it's not critical for what I'm doing. If anyone latter suggests a fix that works I'll mark that as correct.
When I try and request a resource from a cljs app (running on http://localhost:3000) to my Pedestal server (running on http://localhost:8080) I get the below error. I would like to allow CORS from http://localhost:3000:
XMLHttpRequest cannot load http://localhost:8080/db/query. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access.
I am using cljs-http to send the request from the client. The request looks something like this:
(defn load-server-data
[]
(go
(let [q (<! (http/post "http://localhost:8080/db/query"
{:edn-params {:query '[:find ?rep ?last
:where
[?rep :sales-rep/first-name ?last]]}}))]
(println "q" q))))
The route for /db/query looks like this:
(defroutes routes
[[["/db"
{:post handlers/db-post}
["/query" {:post handlers/db-query}
^:interceptors [interceptors/edn-interceptor]]]]])
This is the handler for /db/query:
(defn db-query
[req]
(let [edn-params (:edn-params req)
q (:query edn-params)
args (:args edn-params)
q-result (apply d/q q (d/db conn) args)]
{:status 200
:body (pr-str q-result)}))
To run the server I execute this function in the REPL.
(defn run-dev
"The entry-point for 'lein run-dev'"
[& args]
(println "\nCreating your [DEV] server...")
(-> service/service
(merge {:env :dev
::server/join? false
::server/routes #(deref #'service/routes)
::server/allowed-origins {:creds true :allowed-origins (constantly true)}})
server/default-interceptors
server/dev-interceptors
server/create-server
server/start))
There does not seem to be much information around CORS for Pedestal. I have looked at the cors example but it seems to just work while mine does not. Is there another interceptor I need to add to my routes or some sort of configuration setting that I am missing here?
I have figured out the problem. It turns out that an error was being thrown, however, it was getting swallowed and hidden from my debugger. Simply adding a try catch around my handler function fixes the problem.
(defn db-query
[req]
(try
(let [edn-params (:edn-params req)
q (:query edn-params)
args (:args edn-params)
q-result (apply d/q q (d/db conn) args)]
{:status 200
:body (pr-str q-result)})
(catch Exception ex
{:status 400
:body "Not authorized"})))
My original response:
The purpose of CORS is to limit the origin of the requests. You have
to purposely tell it where requests can come from. This will fix it.
(def service {;other config stuff
io.pedestal.http/allowed-origins ["http://localhost:3000"]}
It appears this is a duplicate question. Apparently javascript ajax requests are by definition limited to single origin. That code would work in production only if the GET request is made by clj-http or http-kit on the ring server that spawn http://localhost:3000 and then a cljs-http ajax request is made to that same ring server on port 3000. I still don't know why your run-dev doesn't work, but if you're calling lein with run, this is definitely what's happening.
According to Compojure:
Compojure does not serve static files by default, nor does it
automatically deal out 404s when no route matches.
So to deal with that, we have to set these Public Vars
files
not-found
resources
And this is how we currently set it:
(defroutes app-routes
(route/files "/" {:root "path/to/public"})
(route/resources "/")
(route/not-found "Not Found"))
It worked as expected when most of the static files are accessed through the web browser.
e.g.
http://localhost:3000/img/icon.png
But the problem is, it doesn't work on favicon files.
e.g.
http://localhost:3000/img/favicon.ico
It treats this as a different call which it should be serve as a static file.
Response to to the CURL I run:
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> GET /img/favicon.ico HTTP/1.1
> User-Agent: curl/7.38.0
> Host: localhost:3000
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 28 Jul 2015 19:05:24 GMT
< Last-Modified: Thu, 28 May 2015 14:51:16 +0000
< Content-Length: 1106
< Content-Type: image/x-icon
* Server Jetty(7.6.13.v20130916) is not blacklisted
< Server: Jetty(7.6.13.v20130916)
files, resources, and not-found are functions, not variables that you set. When you call (route/files "/" {:root "path/to/public"}), a route for resolving URLs under "/" as static files under "path/to/public" is returned.
defroutes defines a collection of routes. These routes are tried in the order they are listed, until the first one that returns a response.
If you add a route (GET "/:slug" [* :as req slug] (search req slug)) before the others, then any URL other than "/" will be handled by this new route — including the favicon request. On the other hand, if you add it just before the not-found route, then it should work.
Also, if there isn't a static file that matches the request, then the files route will fail and the next one will be tried. So you should also check that favicon.ico actually exists and is in the img sub-directory.
umm...
This works for me, (wherever you define/declare your routes):
In the namespace section:
[ring.util.response :refer [resource-response]]
Down below (at least above (def app ):
(defn wrap-return-favicon [handler]
(fn [req]
(if (= [:get "/favicon.ico"] [(:request-method req) (:uri req)])
(resource-response "favicon.ico" {:root "public/img"})
(handler req))))
then in:
(def app
(-> routes
...
wrap-return-favicon
wrap-stacktrace))