Cloverage is now failing after I adding role based authentication to my compojure api. Whilst running lein cloverage I get this exception:
Exception in thread "main" java.lang.IllegalArgumentException: No method in multimethod 'restructure-param' for dispatch value: :auth-roles, compiling:(/private/var/folders/w_/yt926bqs21g44f257yz05ctsjbv948/T/form-init3368607975292148784.clj:1:125)
Here is my auth middleware:
(defmethod compojure.api.meta/restructure-param :auth-roles
[_ required-roles acc]
(update-in acc [:middleware] conj [require-roles required-roles]))
See github issue: https://github.com/cloverage/cloverage/issues/164#issuecomment-281673566
(defmethod compojure.api.meta/restructure-param :auth-roles
[_ required-roles acc]
(update-in acc [:middleware] conj [`require-roles required-roles]))
Related
I am learning clojure, and I've hit a snag while trying to refactor my web app to make it more functional and less reliant on global state.
The way you make a simple auto-reloading server with ring framework is like this:
(defn handler [req]
{:status 200
:body "<h1>Hello World</h1>"
:headers {}})
(defn -main []
(jetty/run-jetty (wrap-reload #'handler)
{:port 12345}))
(source: https://practicalli.github.io/clojure-webapps/middleware-in-ring/wrap-reload.html)
So handler is a global function. It is given as a var reference to wrap-reload. On each request, wrap-reload will reload the entire namespace, and then re-resolve handler reference to a potentially different function. Or at least this is my understanding.
This is all good in this simple example. The trouble starts when I replace this hello world handler with my actual handler, which has all sorts of state bound into it (eg. database connection). This state is initialized once inside -main, and then injected into the routing stack. Something like this:
(defn init-state [args dev]
(let
[db (nth args 1 "jdbc:postgresql://localhost/mydb")
routes (make-routes dev)
app (make-app-stack routes db)
{:port (Integer. (nth args 0 3000))
:route routes
:dev dev
:db db
:app app})
(defn start [state]
(println "Running in " (if (:dev state) "DEVELOPMENT" "PRODUCTION") " mode")
(model/create-tables (:db state))
(jetty/run-jetty (:app state) {:port (:port state)}))
(defn -main [& args]
(start (init-state args false)))
make-routes generates the router based on compojure library, and make-app-stack then wraps this router into a bunch of middlewares, which will inject global state (like DB string) for the use by handlers.
So how do I add wrap-reload to this setup? #'app macro won't work on local let "variable" (or whatever it's called). Do I need to expose my app as a global variable? Or can I re-generate the entire closure on each request.
My instincts tell me to avoid having global initialization code in the module body and keep all code in the main, but maybe I am overthinking. Should I just type my init-state code as globals and call it a day, or is there a better way?
I came up with a solution I can live with.
(ns webdev.core
(:require [ring.middleware.reload :as ring-reload]))
; Defeat private defn
(def reloader #'ring-reload/reloader)
; Other stuff...
(defn load-settings [args dev]
{:port (Integer. (nth args 0 3000))
:db (nth args 1 "jdbc:postgresql://localhost/webdev")
:dev dev})
(defn make-reloading-app [settings]
(let [reload! (reloader ["src"] true)]
(fn [request]
(reload!)
((make-app settings) request))))
(defn start [settings]
(let
[app
(if (:dev settings)
(make-reloading-app settings)
(make-app settings))]
(println "Running in " (if (:dev settings) "DEVELOPMENT" "PRODUCTION") " mode")
(model/create-tables (:db settings))
(jetty/run-jetty app {:port (:port settings)})))
(defn -main [& args]
(start (load-settings args false)))
Full code is available here: https://github.com/panta82/clojure-webdev/blob/master/src/webdev/core.clj
Instead of using wrap-reload directly, I use the underlying private reload function. I have to recreate the routing stack in every request, but it seems to work fine in dev. No global state needed :)
I keep getting "Invalid anti-forgery token" when wrapping specific routes created with metosin/reitit reitit.ring/ring-router. I've also tried reitit's middleware registry, but it didn't work too. Although I could just wrap the entire handler with wrap-session and wrap-anti-forgery, that defeats the reitit's advantage on allowing route-specific middleware.
(ns t.core
(:require [immutant.web :as web]
[reitit.ring :as ring]
[ring.middleware.anti-forgery :refer [wrap-anti-forgery]]
[ring.middleware.content-type :refer [wrap-content-type]]
[ring.middleware.params :refer [wrap-params]]
[ring.middleware.keyword-params :refer [wrap-keyword-params]]
[ring.middleware.session :refer [wrap-session]]
[ring.util.anti-forgery :refer [anti-forgery-field]]
[ring.util.response :as res]))
(defn render-index [_req]
(res/response (str "<form action='/sign-in' method='post'>"
(anti-forgery-field)
"<button>Sign In</button></form>")))
(defn sign-in [{:keys [params session]}]
(println "params: " params
"session:" session)
(res/redirect "/index.html"))
(defn wrap-af [handler]
(-> handler
wrap-anti-forgery
wrap-session
wrap-keyword-params
wrap-params))
(def app
(ring/ring-handler
(ring/router [["/index.html" {:get render-index
:middleware [[wrap-content-type]
[wrap-af]]}]
["/sign-in" {:post sign-in
:middleware [wrap-af]}]])))
(defn -main [& args]
(web/run app {:host "localhost" :port 7777}))
It turns out that metosin/reitit creates one session store for each route (refer to issue 205 for more information); in other words, ring-anti-forgery is not working because reitit does not use the same session store for each route.
As of the time of this answer, the maintainer suggests the following (copied from the issue for ease of reference within Stack Overflow):
mount the wrap-session outside of the router so there is only one instance of the mw for the whole app. There is :middleware option in ring-handler for this:
(require '[reitit.ring :as ring])
(require '[ring.middleware.session :as session])
(defn handler [{session :session}]
(let [counter (inc (:counter session 0))]
{:status 200
:body {:counter counter}
:session {:counter counter}}))
(def app
(ring/ring-handler
(ring/router
["/api"
["/ping" handler]
["/pong" handler]])
(ring/create-default-handler)
;; the middleware on ring-handler runs before routing
{:middleware [session/wrap-session]}))
create a single session store and use it within the routing table (all instances of the session middleware will share the single store).
(require '[ring.middleware.session.memory :as memory])
;; single instance
(def store (memory/memory-store))
;; inside, with shared store
(def app
(ring/ring-handler
(ring/router
["/api"
{:middleware [[session/wrap-session {:store store}]]}
["/ping" handler]
["/pong" handler]])))
Not shown in this answer is the third option that the maintainer calls for PR.
When I try and request a resource from a cljs app (running on http://localhost:3000) to my Pedestal server (running on http://localhost:8080) I get the below error. I would like to allow CORS from http://localhost:3000:
XMLHttpRequest cannot load http://localhost:8080/db/query. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access.
I am using cljs-http to send the request from the client. The request looks something like this:
(defn load-server-data
[]
(go
(let [q (<! (http/post "http://localhost:8080/db/query"
{:edn-params {:query '[:find ?rep ?last
:where
[?rep :sales-rep/first-name ?last]]}}))]
(println "q" q))))
The route for /db/query looks like this:
(defroutes routes
[[["/db"
{:post handlers/db-post}
["/query" {:post handlers/db-query}
^:interceptors [interceptors/edn-interceptor]]]]])
This is the handler for /db/query:
(defn db-query
[req]
(let [edn-params (:edn-params req)
q (:query edn-params)
args (:args edn-params)
q-result (apply d/q q (d/db conn) args)]
{:status 200
:body (pr-str q-result)}))
To run the server I execute this function in the REPL.
(defn run-dev
"The entry-point for 'lein run-dev'"
[& args]
(println "\nCreating your [DEV] server...")
(-> service/service
(merge {:env :dev
::server/join? false
::server/routes #(deref #'service/routes)
::server/allowed-origins {:creds true :allowed-origins (constantly true)}})
server/default-interceptors
server/dev-interceptors
server/create-server
server/start))
There does not seem to be much information around CORS for Pedestal. I have looked at the cors example but it seems to just work while mine does not. Is there another interceptor I need to add to my routes or some sort of configuration setting that I am missing here?
I have figured out the problem. It turns out that an error was being thrown, however, it was getting swallowed and hidden from my debugger. Simply adding a try catch around my handler function fixes the problem.
(defn db-query
[req]
(try
(let [edn-params (:edn-params req)
q (:query edn-params)
args (:args edn-params)
q-result (apply d/q q (d/db conn) args)]
{:status 200
:body (pr-str q-result)})
(catch Exception ex
{:status 400
:body "Not authorized"})))
My original response:
The purpose of CORS is to limit the origin of the requests. You have
to purposely tell it where requests can come from. This will fix it.
(def service {;other config stuff
io.pedestal.http/allowed-origins ["http://localhost:3000"]}
It appears this is a duplicate question. Apparently javascript ajax requests are by definition limited to single origin. That code would work in production only if the GET request is made by clj-http or http-kit on the ring server that spawn http://localhost:3000 and then a cljs-http ajax request is made to that same ring server on port 3000. I still don't know why your run-dev doesn't work, but if you're calling lein with run, this is definitely what's happening.
I have this code to get data from sumo logic and other services.
core.clj has this, which parses the arguments and routes it to the right function in route.clj
(def cli-options
[
["-a" "--app APPNAME" "set app. app can be:
sumologic or jira"]
["-?" "--help"]
])
(defn -main
[& args]
(let [{:keys [options summary errors arguments]} (parse-opts args cli-options)]
(cond
(:app options) (route/to (:app options) options arguments)
:else (print_usage summary))))
route.clj has this:
(defn to
[app options arguments]
(case app
"jira" (jira/respond options arguments)
"sumologic" (sumo/respond)))
And then sumo.clj has this. there are other functions, of course, but showing just the relevant parts.
(defn get-env-var
[var]
(let [result (System/getenv var)]
(if (nil? result)
(throw (Exception. (str "Environment variable: " var " not set. Aborting")))
result)))
(def access_key
(let [user (get-env-var "SUMO_ID")
pass (get-env-var "SUMO_KEY")]
[user pass]))
(defn respond
[]
(let [{:keys [status body error] :as response} (http/get endpoint rest-options)]
(if error
(println error)
(print-response body))))
When I run the program using leiningen as lein run -- -? or even just lein run, I get this error, even though I haven't explicitly called the sumologic function. What am I doing wrong and what are things that I can do differently?
Caused by: java.lang.Exception: Environment variable: SUMO_KEY not set. Aborting
at clarion.sumo$get_env_var.invoke(sumo.clj:14)
at clarion.sumo$fn__3765.invoke(sumo.clj:19)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3553)
You have def'd access_key so it is being evaluated when you load the application. You probably want to make it a function instead.
I'm trying to use netty via clojure. I'm able to startup the server, however, it fails to initialize an accepted socket. Below are the error message and code respectively. Does anyone know what is/or could be wrong? I believe the issue is with (Channels/pipeline (server-handler)) Thanks.
Error Message
#<NioServerSocketChannel [id: 0x01c888d9, /0.0.0.0:843]>
Jun 6, 2012 12:15:35 PM org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink
WARNING: Failed to initialize an accepted socket.
java.lang.IllegalArgumentException: No matching method found: pipeline
project.clj
(defproject protocol "1.0.0-SNAPSHOT"
:description "Upload Protocol Server"
:dependencies [
[org.clojure/clojure "1.2.1"]
[io.netty/netty "3.4.5.Final"]])
core.clj
(ns protocol.core
(:import (java.net InetSocketAddress)
(java.util.concurrent Executors)
(org.jboss.netty.bootstrap ServerBootstrap)
(org.jboss.netty.channel Channels ChannelPipelineFactory SimpleChannelHandler)
(org.jboss.netty.channel.socket.nio NioServerSocketChannelFactory)
(org.jboss.netty.buffer ChannelBuffers)))
(def policy
"<content>Test</content>")
(defn server-handler
"Returns netty handler."
[]
(proxy [SimpleChannelHandler] []
(messageReceived [ctx e]
(let [ch (.getChannel e)]
(.write ch policy)
(.close ch)))
(channelConnected [ctx e]
(let [ch (.getChannel e)]
(.write ch policy)
(.close ch)))
(exceptionCaught [ctx e]
(let [ex (.getCause e)]
(println "Exception" ex)
(-> e .getChannel .close)))))
(defn setup-pipeline
"Returns channel pipeline."
[]
(proxy [ChannelPipelineFactory] []
(getPipeline []
(Channels/pipeline (server-handler)))))
(defn startup
"Starts netty server."
[port]
(let [channel-factory (NioServerSocketChannelFactory. (Executors/newCachedThreadPool) (Executors/newCachedThreadPool))
bootstrap (ServerBootstrap. channel-factory)]
(.setPipelineFactory bootstrap (setup-pipeline))
(.setOption bootstrap "child.tcpNoDelay" true)
(.setOption bootstrap "child.keepAlive" true)
(.bind bootstrap (InetSocketAddress. port))))
There are three problems with your code
Java interop with vararg Channels.channel() method.
you can make a vector of channel handlers and wrap it with (into-array ChannelHandler ..)
You can not write String objects directly to a Netty Channel.
you have to write the string to a ChannelBuffer first and write that buffer or use a StringCodecHandler.
Writing to Netty channel is asynchronus, so you can not close it immediately.
you have to register a future listener and close the channel when its done.
Here is the working code.
(ns clj-netty.core
(:import (java.net InetSocketAddress)
(java.util.concurrent Executors)
(org.jboss.netty.bootstrap ServerBootstrap)
(org.jboss.netty.buffer ChannelBuffers)
(org.jboss.netty.channel Channels ChannelFutureListener ChannelHandler ChannelPipelineFactory SimpleChannelHandler)
(org.jboss.netty.channel.socket.nio NioServerSocketChannelFactory)
(org.jboss.netty.buffer ChannelBuffers)))
(def policy
(ChannelBuffers/copiedBuffer
(.getBytes "<content>Test</content>")))
(defn server-handler
"Returns netty handler."
[]
(proxy [SimpleChannelHandler] []
(messageReceived [ctx e]
(let [ch (.getChannel e)]
(.addListener
(.write ch policy)
(ChannelFutureListener/CLOSE))))
(channelConnected [ctx e]
(let [ch (.getChannel e)]
(.addListener
(.write ch policy)
(ChannelFutureListener/CLOSE))))
(exceptionCaught [ctx e]
(let [ex (.getCause e)]
(println "Exception" ex)
(-> e .getChannel .close)))))
(defn setup-pipeline
"Returns channel pipeline."
[]
(proxy [ChannelPipelineFactory] []
(getPipeline []
(let [handler (server-handler)]
(Channels/pipeline (into-array ChannelHandler [handler]))))))
(defn startup
"Starts netty server."
[port]
(let [channel-factory (NioServerSocketChannelFactory. (Executors/newCachedThreadPool) (Executors/newCachedThreadPool))
bootstrap (ServerBootstrap. channel-factory)]
(.setPipelineFactory bootstrap (setup-pipeline))
(.setOption bootstrap "child.tcpNoDelay" true)
(.setOption bootstrap "child.keepAlive" true)
(.bind bootstrap (InetSocketAddress. port))))
Have a look at Aleph (also uses Netty) which can used to build clients and servers in many different protocols with nice Clojure API.
In 2021, the easiest way to build Clojure services on top of Netty would be to adopt Donkey which provides good Clojure interop with Vert.x which uses Netty as the backend. Donkey is relatively new and doesn't yet have a lot of documentation so check out this blog which covers a relevant open source project (source available) in terms of architecture, design, coding, and testing under load.