how to fix issue with body-params values not being picked up by clojure muuntaja middleware - clojure

I'm trying to adapt the code here https://github.com/danownsthisspace/shorturl/blob/main/src/shorturl/core.clj with this:
(ns todo.core
(:require [clojure.pprint :as pprint]
[muuntaja.core :as m]
[reitit.ring :as ring]
[reitit.ring.middleware.muuntaja :as muuntaja]
[ring.adapter.jetty :as ring-jetty]
[ring.util.response :as r]
[todo.db :as db]))
(defn todo-items-save [req]
(clojure.pprint/pprint req)
(let [title (get-in req [:body-params :title])
content (get-in req [:body-params :content])]
(r/response (str "foooo" title))))
(def app
(ring/ring-handler
(ring/router
[["/"
["" {:handler (fn [req] {:body "hello" :status 200})}]]
["/api/todo" {:post {:handler todo-items-save}
:get (fn [req]
(let [todos db/get-todos]
(r/response todos)))}]
{:data {:muuntaja m/instance :middleware [muuntaja/format-middleware]}}])))
(defn start []
(ring-jetty/run-jetty #'app {:port 3002 :join? false}))
(def server (start))
(.stop server)
but I'm seeing that the body-params values are null. I was wondering why that is when I'm making a post request with the body {"content":"only a test", "title":"second"}. Thank you.

The problem was, that the muuntaja config was passed as part of the
routes instead of argument to ring/router
(def app
(ring/ring-handler
(ring/router
[["/"
["" {:handler (fn [req] {:body "hello" :status 200})}]]
["/api/todo" {:post {:handler todo-items-save}
:get (fn [req]
(let [todos db/get-todos]
(r/response todos)))}]]
; XXX this must be the second argument to `ring/router`
{:data {:muuntaja m/instance
:middleware [muuntaja/format-middleware]}})))

Related

Pedestal early termination is not working

http://pedestal.io/reference/servlet-interceptor Says this
Before invoking the :enter functions, the servlet interceptor sets up a "terminator" predicate on the context. It terminates the interceptor chain when the context map returned by an interceptor has a response map attached.
My server has this:
(ns wp-server.server
(:gen-class) ; for -main method in uberjar
(:require
[io.pedestal.http :as server]
[io.pedestal.http.route :as route]
[wp-server.service :as service]
[wp-server.datomic :as datomic]))
(defonce runnable-service (atom nil))
(defn -main
"The entry-point for 'lein run'"
[& args]
(println "/nConnecting to datomic...")
(datomic/connect!)
(println "\nCreating your server...")
(reset! runnable-service (server/create-servlet service/service))
(server/start runnable-service))
(defn create-runnable-dev-service
"The entry-point for 'lein run-dev'"
[& args]
(println "\nConnecting to [DEV] database...")
(datomic/connect-dev!)
(println "\nCreating your [DEV] server...")
(-> service/service
(merge {:env :dev
::server/join? false
::server/routes #(route/expand-routes (deref #'service/routes))
::server/allowed-origins {:creds true :allowed-origins (constantly true)}
::server/secure-headers {:content-security-policy-settings {:object-src "none"}}})
server/default-interceptors
server/dev-interceptors
server/create-servlet))
(defn start-dev []
(when-not #runnable-service
(reset! runnable-service (create-runnable-dev-service)))
(server/start #runnable-service))
(defn stop-dev []
(server/stop #runnable-service))
(defn restart-dev []
(stop-dev)
(start-dev))
My service looks like this:
(ns wp-server.service
(:require
[datomic.api :as d]
[io.pedestal.http :as http]
[io.pedestal.http.body-params :as body-params]
[io.pedestal.http.route :as route]
[ring.util.response :as ring-resp]
[wp-server.datomic :as datomic]
[wp-common.client :as wp-common-client]
[wp-common.core :as wp-common-core]
[clojure.spec.alpha :as s]
[ring.util.response :as ring-response]))
(defn about-page
[request]
(ring-resp/response (format "Clojure %s - served from %s"
(clojure-version)
(route/url-for ::about-page))))
(def home-page
{:name :home-page
:enter (fn [context]
(prn "TWO")
(assoc context :response {:status 200 :body "Hello, world!"}))})
(defn db-test-page
[{:keys [:database]}]
(ring-resp/response
(prn-str
(d/q '[:find ?text
:where
[?e :advisor/first-name ?text]]
database))))
(def common-interceptors [datomic/db-interceptor (body-params/body-params) http/html-body])
(defn create-spec-validator
[spec]
{:name :validate-spec
:enter
(fn [{{:keys [:edn-params]} :request :as context}]
(prn "ONE")
(if-let [explained (s/explain-data spec edn-params)]
(assoc context
:response {:status 400 :body explained})))})
(def routes #{
;["/" :get (conj common-interceptors `home-page)]
["/clients" :post (conj common-interceptors (create-spec-validator ::wp-common-client/schema) home-page)]
["/db-test" :get (conj common-interceptors `db-test-page)]
["/about" :get (conj common-interceptors `about-page)]})
(def service {:env :prod
::http/routes routes
::http/type :jetty
::http/port 5000
::http/container-options {:h2c? true
:h2? false
:ssl? false}})
When sending a request to localhost:5000/clients with a body that does not pass spec, the create-spec-validator interceptor adds a response to the context. I have confirmed this by logging the context in the home-page interceptor. I would expect the home-page interceptor to be skipped as per documentation. This does not happen. Instead the :enter function of the home-page interceptor is called and the response overwritten.
Why isn't the home-page interceptor being skipped when the create-spec-validator prior to it is returning context with a response?
If you track down the termination code, it invokes
(defn response?
"True if the supplied value is a valid response map."
{:added "1.1"}
[resp]
(and (map? resp)
(integer? (:status resp))
(map? (:headers resp))))
to test if there is a valid response-map in :response. Try setting an empty map in :headers in your response: it should terminate then.
Ideally, of course, you'll set Content-Type to be something meaningful.

How do I mock a PUT request using ring.mock.request

How do I make this test pass:
(ns imp-rest.parser-test-rest
(:require [clojure.test :refer :all])
(:require [ring.mock.request :as mock] )
(:require [imp-rest.web :as w]))
(deftest test-parser-rest
(testing "put settings"
(w/app
(mock/request :put "/settings/coordinateName" "FOO" ))
(let [response (w/app (mock/request :get "/settings"))]
(println response )
(is (= (get (:body response) :coordinateName) "FOO")))))
it fails with:
FAIL in (test-parser-rest) (parser_test_rest.clj:30)
put settings
expected: (= (get (:body response) :coordinateName) "FOO")
actual: (not (= nil "FOO"))
Here's my handler:
(ns imp-rest.web
(:use compojure.core)
(:use ring.middleware.json-params)
(:require [clj-json.core :as json])
(:require [ring.util.response :as response])
(:require [compojure.route :as route])
(:require [imp-rest.settings :as s]))
(defn json-response [data & [status]]
{:status (or status 200)
:headers {"Content-Type" "application/json"}
:body (json/generate-string data)})
(defroutes handler
(GET "/settings" []
(json-response (s/get-settings)))
(GET "/settings/:id" [id]
(json-response (s/get-setting id)))
(PUT "/settings" [id value]
(json-response (s/put-setting id value)))
(route/not-found "Page not found") )
(def app
(-> handler
wrap-json-params))
which exposes this map (of settings):
(ns imp-rest.settings)
(def settings
(atom
{:coordinateName nil
:burnin nil
:nslices nil
:mrsd nil
}))
(defn get-settings []
#settings)
(defn get-setting [id]
(#settings (keyword id)))
(defn put-setting [id value]
(swap! settings assoc (keyword id) value)
value)
and the entry point:
(ns imp-rest.core
(:use ring.adapter.jetty)
(:require [imp-rest.web :as web]))
(defn -main
"Entry point"
[& args]
(do
(run-jetty #'web/app {:port 8080})
);END;do
);END: main
Now when I 'lein run' I can make a (working) request like this:
curl -X PUT -H "Content-Type: application/json" \
-d '{"id" : "coordinateName", "value" : "FOO"}' \
http://localhost:8080/settings
which is what I try to mock with the test. Any help appreciated.
If you want to have :id in your PUT /settings/:id route accepting body in format {"value": "..."}, you need to change your routes definition:
(defroutes handler
(GET "/settings" []
(json-response (s/get-settings)))
(GET "/settings/:id" [id]
(json-response (s/get-setting id)))
(PUT "/settings/:id" [id value]
(json-response (s/put-setting id value)))
(route/not-found "Page not found"))
And change how you call your PUT endpoint in the test:
(w/app
(-> (mock/request
:put
"/settings/coordinateName"
(json/generate-string {:value "FOO"}))
(mock/content-type "application/json")))
What was changed?
:id in your PUT URL route definition (/settings -> /settings/:id)
Your PUT request didn't send a correct request and content type.
If you want to have a PUT /settings route expecting {"id": "...", "value": "..."} request body, then you need to change how you create a mock request:
(w/app
(-> (mock/request
:put
"/settings"
(json/generate-string {:id "coordinateName" :value "FOO"}))
(mock/content-type "application/json"))
Your curl request specifies the parameters as JSON in the body of the PUT request, but your mock request tries to use URL parameters.
There are two options to resolve this:
compojure can automatically translate parameters, but only when the relevant middleware is present -- you have wrap-json-params added to your handler, but you're missing wrap-params. The answer from Piotrek Bzdyl amounts to making these params explicit in the compojure routes.
Alternatively, you can add the ID/value pair as JSON in the body of the mock request using request.mock.body.

with-redefs doesn't redefine my function

I have a test:
(ns gui-proxy.handler-test
(:require [clojure.test :refer :all]
[ring.mock.request :as mock]
[gui-proxy.handler :as handler]))
(deftest test-app
(testing "not-found route"
(with-redefs-fn [handler/log-request (fn [type url] (str ""))]
(let [response (handler/app (mock/request :get "/invalid"))]
(is (= (:status response) 404))))))
and the code that are under test:
(ns gui-proxy.handler
(:require [compojure.core :refer :all]
[compojure.route :as route]
[ring.middleware.defaults :refer [wrap-defaults site-defaults]]
[clj-http.client :as client]
[gui-proxy.db :as db]))
(defn log-request [type url]
(db/insert-request-info type url))
(defn log-error []
(log-request ":fail" "fail"))
"gui-proxy - File not found")
(defroutes app-routes
(route/not-found (log-error)))
So, basically i'd like to stop the call to the database-namespace, but i end upp in a fat database exeception stacktrace...
What is wrong?
with-redefs-fn takes a map of bindings, not a vector. Note that the examples at clojuredocs use the #' reader macro to refer to the Var, so summing up you could try
(deftest test-app
(testing "not-found route"
(with-redefs-fn {#'handler/log-request (fn [type url] (str ""))}
(let [response (handler/app (mock/request :get "/invalid"))]
(is (= (:status response) 404))))))
Take a look at with-redefs it should match your usage.
http://clojuredocs.org/clojure.core/with-redefs

How to use http.kit server with drawbridge ring handler?

I have following server code:
(ns tweet-sentiment.server
(:require [clojure.java.io :as io]
[tweet-sentiment.dev :refer [is-dev? inject-devmode-html browser-repl start-figwheel start-less]]
[compojure.core :refer [GET POST defroutes]]
[compojure.route :refer [resources]]
[net.cgrand.enlive-html :refer [deftemplate]]
[net.cgrand.reload :refer [auto-reload]]
[ring.middleware.reload :as reload]
[ring.middleware.defaults :refer [wrap-defaults api-defaults]]
[tweet-sentiment.utils :refer [generate-response]]
[environ.core :refer [env]]
[org.httpkit.server :refer [run-server]]
[ring.middleware.edn :refer [wrap-edn-params]]
[tweet-sentiment.tweets :refer [tweets]]
[tweet-sentiment.dandelion :refer [dandelion-sentiment]]
[clojure.core.async :refer [>!! <!! put! take! pipe chan]]
[cemerick.drawbridge :as drawbridge]
[ring.middleware.basic-authentication :refer [wrap-basic-authentication]]
)
(:gen-class))
(defonce server (atom nil))
(deftemplate page (io/resource "index.html") []
[:body] (if is-dev? inject-devmode-html identity))
(defroutes routes
(resources "/")
(resources "/react" {:root "react"})
(GET "/*" req (page)))
(defn wrap-drawbridge [handler]
(fn [req]
(if (= "/repl" (:uri req))
(drawbridge/ring-handler req)
(handler req))))
(def http-handler
(if is-dev?
(-> #'routes
(wrap-defaults api-defaults)
reload/wrap-reload
wrap-edn-params
wrap-drawbridge)
(-> (wrap-defaults #'routes api-defaults)
wrap-edn-params
wrap-drawbridge)))
(defn run-web-server [& [port]]
(let [port (Integer. (or port (env :port) 10555))]
(println (format "Starting web server on port %d." port))
(reset! server (run-server http-handler {:port port :join? false}))))
(defn run-auto-reload [& [port]]
(auto-reload *ns*)
(start-figwheel)
(start-less))
(defn run [& [port]]
(when is-dev?
(run-auto-reload))
(run-web-server port))
(defn stop-server []
(when-not (nil? #server)
(#server :timeout 0)
(reset! server nil)))
(defn restart-server []
(stop-server)
(run-web-server))
(defn -main [& [port]]
(run port))
When I try to connect to drawbridge REPL as dev or production:
lein repl :connect http://localhost:10555/repl
I get following error:
ERROR - POST /repl
java.lang.IllegalArgumentException: No value supplied for key: {:remote-addr "127.0.0.1", :headers {"accept-encoding" "gzip, deflate", "connection" "close", "content-length" "48", "content-type" "application/x-www-form-urlencoded", "host" "localhost:10555", "user-agent" "Apache-HttpClient/4.3.3 (java 1.5)"}, :async-channel #<AsyncChannel /127.0.0.1:10555<->/127.0.0.1:60374>, :server-port 10555, :content-length 48, :websocket? false, :content-type "application/x-www-form-urlencoded", :character-encoding "utf8", :uri "/repl", :server-name "localhost", :query-string nil, :body #<BytesInputStream BytesInputStream[len=48]>, :scheme :http, :request-method :post}
at clojure.lang.PersistentHashMap.create(PersistentHashMap.java:77)
at cemerick.drawbridge$ring_handler.doInvoke(drawbridge.clj:49)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at tweet_sentiment.server$wrap_drawbridge$fn__21884.invoke(server.clj:36)
at org.httpkit.server.HttpHandler.run(RingHandler.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Problem is obviously in drawbridge/ring-handler function.
These are my dependencies:
[[org.clojure/clojure "1.6.0"]
[org.clojure/clojurescript "0.0-3058" :scope "provided"]
[ring "1.3.2"]
[ring/ring-defaults "0.1.4"]
[compojure "1.3.2"]
[enlive "1.1.6"]
[org.omcljs/om "0.8.8"]
[om-sync "0.1.1"]
[environ "1.0.0"]
[http-kit "2.1.19"]
[fogus/ring-edn "0.3.0"]
[prismatic/om-tools "0.3.11"]
[secretary "1.2.3"]
[sablono "0.3.4"]
[twitter-api "0.7.8"]
[racehub/om-bootstrap "0.5.3"]
[cheshire "5.2.0"]
[com.cemerick/drawbridge "0.0.7"]
[ring-basic-authentication "1.0.1"]]
Did anybody get working http.kit server with drawbrige? What am I missing?
I eventually figured it out, server file should look somewhat like this
(ns tweet-sentiment.server
(:require [clojure.java.io :as io]
[tweet-sentiment.dev :refer [is-dev? inject-devmode-html browser-repl start-figwheel start-less]]
[compojure.core :refer [GET POST defroutes]]
[compojure.route :refer [resources]]
[net.cgrand.enlive-html :refer [deftemplate]]
[net.cgrand.reload :refer [auto-reload]]
[ring.middleware.reload :as reload]
[ring.middleware.defaults :refer [wrap-defaults api-defaults]]
[tweet-sentiment.utils :refer [generate-response]]
[environ.core :refer [env]]
[org.httpkit.server :refer [run-server]]
[ring.middleware.edn :refer [wrap-edn-params]]
[clojure.core.async :refer [>!! <!! put! take! pipe chan]]
[cemerick.drawbridge :as drawbridge]
[ring.middleware.basic-authentication :refer [wrap-basic-authentication]]
[ring.middleware.params :as params]
[ring.middleware.keyword-params :as keyword-params]
[ring.middleware.nested-params :as nested-params]
[ring.middleware.session :as session]
)
(:gen-class))
(defonce server (atom nil))
(deftemplate page (io/resource "index.html") []
[:body] (if is-dev? inject-devmode-html identity))
(defroutes routes
(resources "/")
(resources "/react" {:root "react"})
(GET "/*" req (page)))
(defn authenticated? [name pass]
(= [name pass] [(System/getenv "AUTH_USER") (System/getenv "AUTH_PASS")]))
(def drawbridge-handler
(-> (drawbridge/ring-handler)
(keyword-params/wrap-keyword-params)
(nested-params/wrap-nested-params)
(params/wrap-params)
(session/wrap-session)))
(defn http-handler [handler]
(-> handler
(wrap-defaults api-defaults)
wrap-edn-params))
(defn wrap-http [handler]
(fn [req]
(let [handler (if (= "/repl" (:uri req))
(wrap-basic-authentication drawbridge-handler authenticated?)
(if is-dev?
(-> handler
http-handler
reload/wrap-reload)
(-> handler
http-handler)))]
(handler req))))
(defn run-web-server [& [port]]
(let [port (Integer. (or port (env :port) 10555))]
(println (format "Starting web server on port %d." port))
(reset! server
(run-server (wrap-http #'routes) {:port port :join? false})
)))
(defn run-auto-reload [& [port]]
(auto-reload *ns*)
(start-figwheel)
(start-less))
(defn run [& [port]]
(when is-dev?
(run-auto-reload))
(run-web-server port))
(defn stop-server []
(when-not (nil? #server)
(#server :timeout 0)
(reset! server nil)))
(defn restart-server []
(stop-server)
(run-web-server))
(defn -main [& [port]]
(run port))

Compojure app not playing well with with-redefs

I'm writing a Compojure application and am using clj-webdriver to graphically test it. I'm trying to use with-redefs to mock out the function that pulls out data from persistence to just return canned values, but it's ignoring my function overwrite. I know with-redefs works in terms of vars, but it's still not working:
project.clj relevant pieces:
(defproject run-hub "0.1.0-SNAPSHOT"
:main run-hub.handler/start-server)
handler.clj:
(ns run-hub.handler
(:require [compojure.core :refer :all]
[compojure.handler :as handler]
[compojure.route :as route]
[ring.adapter.jetty :refer :all]
[run-hub.controllers.log-controller :as log-controller]))
(defroutes app-routes
(GET "/MikeDrogalis/log" [] (log-controller/mikes-log))
(route/resources "/")
(route/not-found "Not Found"))
(def app (handler/site #'app-routes))
log-controller.clj:
(ns run-hub.controllers.log-controller
(:require [run-hub.views.log :as views]
[run-hub.persistence :as persistence]))
(defn mikes-log []
(views/mikes-log (persistence/mikes-log)))
persistence.clj
(ns run-hub.persistence
(require [clj-time.core :as time]
[run-hub.models.log :as log]))
(defn mikes-log [] [])
And finally, my graphical test - which tries to override mikes-log and fails:
(fact
"It has the first date of training as August 19, 2012"
(with-redefs [persistence/mikes-log (fn [] (one-week-snippet))]
(to (local "/MikeDrogalis/log"))
(.contains (text "#training-log") "August 19, 2012"))
=> true)
Where one-week-snippet is a function that returns some sample data.
(defn start-server []
(run-jetty (var app) {:port 3000 :join? false}))
I am able to use with-redefs in a clj-webdriver test doing the following:
(defn with-server
[f]
(let [server (run-jetty #'APP {:port 0 :join? false})
port (-> server .getConnectors first .getLocalPort)]
(binding [test-port port]
(try
(println "Started jetty on port " test-port)
(f)
(finally
(.stop server))))))
(use-fixtures :once with-server)
Then the whole bunch of tests gets its own jetty and this seems to run in
such a manner that with-redefs works.