I have a page with a login form, and a server that accepts POST requests. Here is the server:
(ns clj_server.core
(:require [org.httpkit.server :refer [run-server]]
[compojure.core :refer [defroutes POST]]
[compojure.route :as route]
[ring.middleware.params :refer [wrap-params]]))
(defn printPostBody [request]
{:status 200
:headers {"Content-Type" "text/html"}
:body request})
(defroutes routes
(POST "/login" request (printPostBody request))
(route/not-found {:status 404 :body "<h1>Page not found</h1"}))
(def app (wrap-params routes))
(defn -main [& args]
(run-server app {:port 8000})
(println "Server started on port 8000"))
When I make a login request, this is printed out:
[:remote-addr "0:0:0:0:0:0:0:1"][:params {"username" "asdf", "password" "ghkj"}][:route-params {}][:headers {"origin" "http://localhost:3449", "host" "localhost:8000", "user-agent" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.172 Safari/537.36 Vivaldi/2.5.1525.46", "content-type" "application/x-www-form-urlencoded", "content-length" "27", "referer" "http://localhost:3449/", "connection" "keep-alive", "upgrade-insecure-requests" "1", "accept" "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3", "accept-language" "en-US,en;q=0.9", "accept-encoding" "gzip, deflate, br", "cache-control" "max-age=0"}][:async-channel #object[org.httpkit.server.AsyncChannel 0x2212125a "/0:0:0:0:0:0:0:1:8000<->/0:0:0:0:0:0:0:1:50592"]][:server-port 8000][:content-length 27][:form-params {"username" "asdf", "password" "ghkj"}][:compojure/route [:post "/login"]][:websocket? false][:query-params {}][:content-type "application/x-www-form-urlencoded"][:character-encoding "utf8"][:uri "/login"][:server-name "localhost"][:query-string nil][:body #object[org.httpkit.BytesInputStream 0x4e67c6c0 "BytesInputStream[len=27]"]][:scheme :http][:request-method :post]
So I'm wondering, what kind of data structure is it? It doesn't look like a hash map, yet when I print out (:params request) instead of request, I get
["username" "asdf"]["password" "ghkj"]
Is it a hash-map of a list of vectors? I don't understand what kind of data structure I'm dealing with here.
Also, why is {"username" "asdf", "password" "ghkj"} being converted into ["username" "asdf"]["password" "ghkj"] when I only ask for the params instead of the whole request?
I then tried printing out (get (:params request) "username") and I got "asdf". Which makes sense, but how is it allowing me to use get on a collection of multiple vectors?
Finally, how would I process JSON in my post requests? Is it just the same thing, or would I have to handle it differently?
I'm making a toy API using the Yada library in Clojure. It searches a database for city names starting with the given characters and returns some info about it.
I want a URI of the form: /cities/:name?count=:count so for example /cities/ber?count=4 will return the top 4 matches. But I also want /cities/ber without the ?count= parameter to return a default number of results (say just the first).
I've defined my route and yada handler like this:
(defn city-search-fn
[ctx]
(let [name (get-in ctx [:parameters :path :name])
count (get-in ctx [:parameters :query :count] 1)]
(city->geoposition name count)))
(def cities (yada/handler (yada/resource
{:methods
{:get
{:parameters {:path {:name String}
:query {:count Long}}
:produces ["application/json"
"application/edn"]
:response city-search-fn}}})))
(def routes
[["/cities/" :name] cities])
(def server
(yada/listener routes {:port 30000}))
This works fine if I supply the ?count= query parameter:
$ curl -i 'http://localhost:30000/cities/ber?count=2'
HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Length: 259
Content-Type: application/json
Vary: accept
Server: Aleph/0.4.4
Connection: Keep-Alive
Date: Mon, 09 Sep 2019 16:01:45 GMT
[{"name":"Berlin","state":"Berlin","countrycode":"DE","timezone":"Europe/Berlin","latitude":52.52437,"longitude":13.41053},{"name":"Berbera","state":"Woqooyi Galbeed","countrycode":"SO","timezone":"Africa/Mogadishu","latitude":10.43959,"longitude":45.01432}]
But I get status 400 ({:status 400, :errors ([:query {:error {:count missing-required-key}}])}) if I don't supply it:
$ curl -i 'http://localhost:30000/cities/ber'
HTTP/1.1 400 Bad Request
Content-Length: 77
Content-Type: text/plain;charset=utf-8
Server: Aleph/0.4.4
Connection: Keep-Alive
Date: Mon, 09 Sep 2019 16:06:56 GMT
{:status 400, :errors ([:query {:error {:count missing-required-key}}])}
The documentation of yada says it supports optional query parameters using the "schema" library. So I found in schema's documentation that there exists a schema.core/maybe function. I tried to modify my yada resource as follows:
:parameters {:path.....
:query (schema/maybe {:count Long})}
this doesn't work (same 400 error).
Then I tried:
:parameters {:path.....
:query {:count (schema/maybe Long)}}
this also didn't work.
So my question is: what is the correct way to have an optional query parameter in yada?
To answer my own question, digging more into Schema documentation, here is the correct way:
:parameters {:path.....
:query {(schema/optional-key :count) Long}}
The key itself needs to be marked as optional.
What's the correct way to send a POST request with content-type application/json?
When I send the request to my application, I can't retrive the parameters.
curl -X POST http://localhost:3000/agents --data '{"name":"verma"}' --header "Content-type:application/json"
I'm using:
[org.clojure/clojure "1.8.0"]
[compojure "1.5.1"]
[ring/ring-defaults "0.2.1"]]
And my handler.cljs is:
▾| 1 (ns sof.rest.handler
| 2 (:require [compojure.core :refer :all]
| 3 [compojure.route :as route]
| 4 [clojure.data.json :as json]
| 8 [ring.middleware.defaults :refer [wrap-defaults api-defaults]]))
| 9
| 10 (defn json-response [data & [status]]
| 11 {:status (or status 200)
| 12 :headers {"Content-Type" "application/json"}
| 13 :body (json/write-str data)})
| 14
| 15 (defroutes app-routes
|~ 16 (POST "/agents" params (println params))
| 21
~| 22 (def app
~| 23 (wrap-defaults app-routes api-defaults))
Those are the params logged when I run the CURL command:
{:ssl-client-cert nil, :remote-addr 0:0:0:0:0:0:0:1, :params {}, :route-
params {}, :headers {accept */*, user-agent curl/7.54.0, content-type
application/json, content-length 16, host localhost:3000}, :server-port
3000, :content-length 16, :form-params {}, :compojure/route [:post
/agents], :query-params {}, :content-type application/json, :character-
encoding nil, :uri /agents, :server-name localhost, :query-string nil,
:body #object[org.eclipse.jetty.server.HttpInput 0x70504fbe
org.eclipse.jetty.server.HttpInput#70504fbe], :scheme :http, :request-
method :post}
You can slurp the :body input stream (read it into a string) and parse its JSON. Something like this:
(POST "/agents" request
(let [body (json/read-str (slurp (:body request)))]
(println body)))
But you should use the Ring middleware to do this automatically.
I am playing with compojure-api and am blocked at trying to manage Content-Type for my simple webapp. What I want is to emit an HTTP response that is just plain/text, but somehow Compojure-API keeps setting it to "application/json".
(POST "/echo" []
:new-relic-name "/v1/echo"
:summary "info log the input message and echo it back"
:description nil
:return String
:form-params [message :- String]
(log/infof "/v1/echo message: %s" message)
(let [resp (-> (resp/response message)
(resp/status 200)
(resp/header "Content-Type" "text/plain"))]
(log/infof "response is %s" resp)
resp))
but curl shows the server responded Content-Type:application/json.
$ curl -X POST -i --header 'Content-Type: application/x-www-form-urlencoded' -d 'message=frickin compojure-api' 'http://localhost:8080/v1/echo'
HTTP/1.1 200 OK
Date: Fri, 13 Jan 2017 02:04:47 GMT
Content-Type: application/json; charset=utf-8
x-http-request-id: 669dee08-0c92-4fb4-867f-67ff08d7b72f
x-http-caller-id: UNKNOWN_CALLER
Content-Length: 23
Server: Jetty(9.2.10.v20150310)
My logging shows that the function requested "plain/text", but somehow the framework trumped it.
2017-01-12 18:04:47,581 INFO [qtp789647098-46]kthxbye.v1.api [669dee08-0c92-4fb4-867f-67ff08d7b72f] - response is {:status 200, :headers {"Content-Type" "text/plain"}, :body "frickin compojure-api"}
How do I gain control over Content-Type in a Compojure-API Ring application?
compojure-api serves response in format requested by HTTP client which is indicated using HTTP Accept header.
With curl you need to add:
-H "Accept: text/plain"
You can also provide a list of acceptable formats and the server will serve the response in the first supported format from that list:
-H "Accept: text/plain, text/html, application/xml, application/json, */*"
I never tried compojure so here goes nothing:
1.) your local val reps has the same name as the aliased namespace - kind of confusing
2.) to get access to the params - it seems - you have to apply ring.middleware.params/wrap-params to your routes
3.) ah yes the Content-Type: since you required :form-params, which didn't get delivered due to missing wrap-params you ended up in some sort of default route - hence not text/plain. Thats what I think happend, at least.
with
lein try compojure ring-server
demo/paste into repl:
(require '[compojure.core :refer :all])
(require '[ring.util.response :as resp])
(require '[ring.server.standalone :as server])
(require '[ring.middleware.params :refer [wrap-params]])
(def x
(POST "/echo" [message]
:summary "info log the input message and echo it back"
:description nil
:return String
:form-params [message :- String]
(let [resp (-> (resp/response (str "message: " message))
(resp/status 200)
(resp/header "Content-Type" "text/plain"))]
resp)))
(defroutes app (wrap-params x))
(server/serve app {:port 4042})
test:
curl -X POST -i --header 'Content-Type: application/x-www-form-urlencoded' -d 'message=frickin' 'http://localhost:4042/echo'
HTTP/1.1 200 OK
Date: Fri, 13 Jan 2017 17:32:03 GMT
Content-Type: text/plain;charset=ISO-8859-1
Content-Length: 14
Server: Jetty(7.6.13.v20130916)
message: frickin
I think I'm doing things right but I can't get my EDN out of the :body input stream. The Ring and Compojure handler is this:
dependencies:
[ring.middleware.params :as params]
[ring.middleware.edn :as edn]; https://github.com/tailrecursion/ring-edn
Handler:
(defroutes ajax-example
(PUT "/ajax-example" r
(println r)
{:status 200
:headers {"Content-Type" "application/edn"}
:body "yo"}))
I wrap it like:
(def ajax-wrapped
(-> ajax-example
edn/wrap-edn-params
params/wrap-params))
The println'd response correctly reveals that it is EDN and the content length is correct based on the simple test map I send in, but the map itself is nowhere to be found, it is forever trapped in the input stream of the :body... how to get at it?
Here is the response println:
{:ssl-client-cert nil, :remote-addr 0:0:0:0:0:0:0:1, :params {}, :route-params {}, :headers {origin http://localhost:6542, host localhost:6542, user-agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36, content-type application/edn, cookie _ga=GA1.1.1354518981.1429622648; content-length 20, referer http://localhost:6542/, connection keep-alive, accept /, accept-language en-US,en;q=0.8,sq;q=0.6, accept-encoding gzip, deflate, sdch, cache-control max-age=0}, :server-port 6542, :content-length 20, :form-params {}, :query-params {}, :content-type application/edn, :character-encoding nil, :uri /ajax-example, :server-name localhost, :query-string nil, :body #, :edn-params nil, :scheme :http, :request-method :put}
The :body is not pasting correctly above, it looks like this:
[open corner bracket] HttpInput org.eclipse.jetty.server.HttpInput#5d969109 [end corner bracket]
The client-side code sent from the browser using cljs-ajax lib is:
(defn ajax-send
[]
(let [push {:adder1 2 :add2 3}]
(PUT "/ajax-example"
{:format :edn
:params push
:handler ajax-result
:error-handler error-handler})))
Here is the output of a test suggested by one of the answers:
hf.examples> ((-> ajax-example params/wrap-params edn/wrap-edn-params) (-> (mock/request :put "/ajax-example")
(mock/content-type "application/edn")
(mock/body "{:hello \"there\"}")))
{:status 200,
:headers {"Content-Type" "application/edn"},
:body
"{:remote-addr \"localhost\", :params {:hello \"there\"}, :route-params {}, :headers {\"content-length\" \"16\", \"content-type\" \"application/edn\", \"host\" \"localhost\"}, :server-port 80, :content-length 16, :form-params {}, :query-params {}, :content-type \"application/edn\", :uri \"/ajax-example\", :server-name \"localhost\", :query-string nil, :edn-params {:hello \"there\"}, :scheme :http, :request-method :put}"}
hf.examples>
I also tried this:
(defroutes ajax-example
(PUT "/ajax-example" r
{:status 200
:headers {"Content-Type" "application/edn"}
:body (pr-str (dissoc r :body))}))
Curl result independent of the front end:
curl -X PUT -H "Content-Type: application/edn" -d '{:name :barnabas}' http://localhost:6542/ajax-example
{:ssl-client-cert nil, :remote-addr "0:0:0:0:0:0:0:1", :params {}, :route-params {}, :headers {"host" "localhost:6542", "content-length" "17", "content-type" "application/edn", "user-agent" "curl/7.37.1", "accept" "*/*"}, :server-port 6542, :content-length 17, :content-type "application/edn", :character-encoding nil, :uri "/ajax-example", :server-name "localhost", :query-string nil, :edn-params nil, :scheme :http, :request-method
The content-length of 17 matches the number of characters in the map passed via Curl. But the edn-params is nil! Where is the content?
EDIT: As an answer to the updated question, the wrap-edn-params function consumes the body of the request by fully reading the :body InputStream. Compojure routes passes the request to each parameter handler until a non nil value is returned. In this case, whichever handler is passed to routes as the first handler will consume :body and there will be no :body value for the 2nd handler to consume, resulting in a nil body value read by wrap-edn-params.
The request that is being passed to the ring handler probably does not have its content-type set to edn. The wrap-edn-params function will only parse the edn if the request content-type is set to edn.
In addition the parsed edn parameters will only be placed in the :params and :edn-params keys of the request map by the wrap-edn-params function, and therefore :body should not be used to access the parsed edn.
(require '[ring.mock.request :as mock])
(require '[ring.middleware.edn :as edn])
((-> ajax-example params/wrap-params edn/wrap-edn-params) (-> (mock/request :put "/ajax-example")
(mock/content-type "application/edn")
(mock/body "{:hello \"there\"}")))
{:remote-addr "localhost",
:params {:hello "there"},
:route-params {},
:headers
{"content-length" "16",
"content-type" "application/edn",
"host" "localhost"},
:server-port 80,
:content-length 16,
:form-params {},
:query-params {},
:content-type "application/edn",
:uri "/ajax-example",
:server-name "localhost",
:query-string nil,
:body #<ByteArrayInputStream java.io.ByteArrayInputStream#171788d8>,
:edn-params {:hello "there"},
:scheme :http,
:request-method :put}
I have fixed this but I do not know why the problem exists. I had another independent route that also wrapped EDN middle ware. Here is the reproducible example:
(defroutes example-routes2
(PUT "/test-edn" r
(println r)
{:status 200
:headers {"Content-Type" "application/edn"}
:body (pr-str (str "request is: " r))}))
(defroutes ajax-example
(PUT "/ajax-example" r
(println r)
{:status 200
:headers {"Content-Type" "application/edn"}
:body (pr-str (str "request is: " r))}))
(def edn-routes
(-> example-routes2
edn/wrap-edn-params))
(def ajax-wrapped
(-> ajax-example
edn/wrap-edn-params))
;;combining all routes on this page into a single handler
(def example-routes (routes example-routes1 ajax-wrapped edn-routes))
Note that these two routes are essentially identical and are being wrapped identically. The one that fails to parse EDN into edn-params is the one listed second in the example-routes def!
curl -X PUT -H "Content-Type: application/edn" -d '{:name :barnabasss}' http://localhost:6542/test-edn
returns:
"request is: {:ssl-client-cert nil, :remote-addr \"0:0:0:0:0:0:0:1\", :params {}, :route-params {}, :headers {\"host\" \"localhost:6542\", \"content-length\" \"19\", \"content-type\" \"application/edn\", \"user-agent\" \"curl/7.37.1\", \"accept\" \"/\"}, :server-port 6542, :content-length 19, :content-type \"application/edn\", :character-encoding nil, :uri \"/test-edn\", :server-name \"localhost\", :query-string nil, :body #, :edn-params nil, :scheme :http, :request-method :put}"
curl -X PUT -H "Content-Type: application/edn" -d '{:name :barnabasss}' http://localhost:6542/ajax-example
returns:
"request is: {:ssl-client-cert nil, :remote-addr \"0:0:0:0:0:0:0:1\", :params {:name :barnabasss}, :route-params {}, :headers {\"host\" \"localhost:6542\", \"content-length\" \"19\", \"content-type\" \"application/edn\", \"user-agent\" \"curl/7.37.1\", \"accept\" \"/\"}, :server-port 6542, :content-length 19, :content-type \"application/edn\", :character-encoding nil, :uri \"/ajax-example\", :server-name \"localhost\", :query-string nil, :body #, :edn-params {:name :barnabasss}, :scheme :http, :request-method :put}"
Switching their order in the example-routes switches which one works. Can anyone explain?