I have the following Compojure routes:
(defroutes my-handler
(GET "/:my-model-id" [id] (render-my-model (parse-int id))))
It is unfortunate that, for every route that I define this way, I have to manually add a call to parse the incoming integer.
I have created Ring middleware that crawls through any form-params and request-params, and parses anything that looks like it might be an integer. However, this middleware does not apply to the custom-defined Compojure routes.
Does anybody know how I could get Compojure to automatically handle the integer-parsing? Can I somehow hook it up to my existing middleware?
Unfortunately compojure will directly invoke the function that is generated from your route definition after it has parsed the params.
AFAIC the only way to get in between is either to modify compojures codebase directly or to use Robert Hooke (by technomancy) on assoc-route-params in https://github.com/weavejester/compojure/blob/master/src/compojure/core.clj#L30
Related
Synopsis
Please note, this question isn't about how to serve a static file — that's working — it's about the special case when wrap-file delivers an index file by default and, for lack of a file extension in the URL, the wrong mime-type is being assigned to the Content-Type header.
How does one get the correct mime type on index files served by default?
Current answers don't address how to do this yet, and the workaround
I've come up with doesn't scale.
Working Code
Here's a simplified fragment from a Clojure application using Compojure and Ring middleware:
(def app
(-> handler
(wrap-file "public") ; If the route is a static file in public, serve it instead
(wrap-content-type))) ; Deduce and add the proper Content-Type header
The intent is to serve up any routes, but if there's a local file in the public directory serve it instead, finally add a meaningful Content-Type header with the corresponding mime type. All this works perfectly.
The Problem
When I browse to the base URL, it does serve index.html as expected, but it does not get a Content-Type of text/html, but rather application/octet-stream.
ring.middleware.file/wrap-file indicates that the index-files? option defaults to true, and this explains why a URL with no paths correctly serves the file. This appears to be the pedantic way of serving static resources.
ring.middleware.content-type/wrap-content-type indicates that the mime-type is deduced by the file extension in the URI, and without one defaults to application/octet-stream. As the URL contain no filename, this function is 'properly' doing what it states.
This begs the question, how to assign Content-Type by contents of the response's body?
However, it's ill-advised to have middleware reads the :body common problems, because it's a mutable InputStream that can only be read once. So that's obviously not the right way.
Is there a better way to serve the index.html by default?
An Ugly Workaround
The current ugly workaround is to have a special-case route that sets the Content-Type manually. <cringe/>
Worse, this solution doesn't scale, should an index file be served from
a subdirectory.
Consequently, I'm looking for a Middleware solution, not a routing hack.
Experiments
Exploring the Execution Order of the Middleware and Its Consequences:
Admittedly, although I understand the thread macro (->) in that (-> x A B) transforms into (B (A x) ), I still get a little jumbled in my head when working out the order that the execution flow resolved through a middle-ware chain to an eventual handler with routes. The reason for this stumbling is that code can mess with the request before calling the the handler it was passed, as well as fiddle with the response before returning. The order things need to be in doesn't feel "obvious" to know when I'm augmenting the request with details going in or twiddling with the response coming out, or the more complicated case of doing a different behavior based on some condition.
e.g., Does wrap-file happen "before" or "after" the handler has constructed a response, as the order matters in the threading? I feel this should be more intuitive to me, without having to run to the source code as much as I'm doing.
As it appears possible to have middleware applied only when a specific route matches, it may be that I'm making more of a distinction between Middleware and Handlers than perhaps I should.
Swapping the order (to test the threading-order assumptions) does not do what you think:
(def app ; THIS IS AN EXAMPLE OF BROKEN CODE - DON'T USE IT
(-> handler
(wrap-content-type))) ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
(wrap-file "public") ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
It "works," but for the wrong reason. The index.html file will get delivered and renders "properly," but only because there is no Content-Type added. The browser, for lack of a specified mime type, makes an educated guess and happens to guess correctly.
Since the goal is to have a Content-Type in the header, this suggests the threaded order was correct to start with.
What Middleware Should Be Used To Deliver Index Pages?
So with information in hand of what not to do, what is it I should be doing to deliver the default status index.html file when the URL doesn't specify it by name, since there's no extension to examine?
WORKS — http://localhost/index.html (serves page with correct mime type)
BROKEN — http://localhost/ (serves same page, but with the wrong content type, so the browser tries to download it)
Is there a better middleware stack, or even a recommended one, that someone could walk me through?
UPDATE 2020-05-24: Submitted Ring Issue 480; turns out this may be a design bug looking for a contributor.
This gives you a server which will serve index.html if present inside a resources/public/ folder.
(ns core
(:require [compojure.core :refer [routes GET]]
[ring.middleware.defaults :refer [wrap-defaults]]
[org.httpkit.server :as http-kit]))
(def handler
(routes
(GET "/foo" [] "Hello Foo")
(GET "/bar" [] "Hello Bar")))
(def app
(-> handler
(wrap-defaults {:static {:resources "public"
:files "resources/public"}})))
(def server (http-kit/run-server app {:port 8889}))
(comment
;; To stop the server
(server))
I'm using wrap-defaults as it provides a nice way to get a server up and running, while still providing a lot of flexibility to drop in customisations as required.
In this case I'm telling it to use public as a resources folder and also handing it resources/public to files so it can correctly wrap the files to be served.
ring.middleware.content-type defaults to application/octet-stream when it has insufficient information to guess the content type of the file it is serving.
If you specifically just want to serve files+provide routing, the answer I have given above is sufficient, if you want to explicitly return a Content-Type text/html for index.html, then you will need to wrap the content type using [ring.util.response :refer [content-type]].
So for example:
(GET "/" [] (content-type (io/resource "index.html") "text/html"))
I've normally done this by detecting the file extension in the request url and then returning the correct content-type, with a special case for things like index.html.
You need at least these deps, this is in deps.edn format, but just change it to [ring/ring-core "1.8.0"] for example if you need it in lein's project.clj form instead:
ring/ring-core {:mvn/version "1.8.0"}
ring/ring-defaults {:mvn/version "0.3.2"}
http-kit {:mvn/version "2.3.0"}
compojure {:mvn/version "1.6.1"}
Let me know if you have any issues!
I am new to Closure and trying out Ring and Compojure. I want to make an HTTP request when a user hits a route (to a third party API) and then use that response data in my HTML template
. I know this is likely a very easy thing to pull off - but being new to language and the syntax I am a bit lost.
(defroutes app
(GET "/" request
; try to GET "https://third-party-api" and do something with the response
)
)
What is the best practice and format for this - It's possible I am missing some key concepts in routing / response expectations here. Thanks very much!
I recommend the library clj-http for making http requests. You can find many examples on the linked page for how to use it.
Your usage of clj-http may look something like this:
(ns my-app.core
(:require [clj-http.client :as client]))
...
(defn get-api-data []
(:body (client/get "https://third-party-api" {:as :json})))
Note that clj-http.client/get returns a map that includes things like the response status code and headers.
If you use the {:as :json} option to coerce the response into json, make sure to include cheshire in your project.clj (assuming you're using leiningen)
:dependencies [...
[clj-http "3.9.0"]
[cheshire "5.8.0"]]
Documentation on ring requests and responses can be found here.
A large portion of the power in ring is its concept of middlewares. Most of the "nice" features that you'd want in an http server can be found as middlewares in ring itself or other libraries. For example, if you want all of your responses to be serialized as json by default, you might use ring-json
If you're trying to get something "that just works", up and running quickly with a few examples present, Luminus may be useful. It's a curated collection of libraries that prove useful for most webservers. (Disclaimer: I've only minimally experimented with Luminus, opting to understand more explicitly my dependencies).
I personally use compojure sweet at the start of most of my webservice projects, it includes some nicer routing features (including path params) and a swagger UI for testing your endpoints. Unfortunately, it uses its own form of destructuring and includes a bit more magic and "just needing to know" than I'd like, but I'm yet to find something that works better for me.
Context
I am writing an app using Clojure and following the functional programming paradigm. In this app I have two HTTP endpoints: /rank and /invite. In /rank the app ranks a list of costumers based on their scores. In /invite the app receives an invite from one costumer to another and this should yield changes on the scores of some costumers.
Problem
Data from costumers are kept in a single vector of maps called record.
Putting aside referential transparency for a moment, record should be a shared resource between the endpoints, one reads it and uses it in a ranking function to respond an HTTP request and the other reads it and updates scores in it.
Now, with functional programming in mind, record cannot be updated, so the /invite endpoint should read it and return a new record', problem is, /rank endpoint is setup to use record, but when a new record' is generated it should use it instead of the original one.
My ideas for solving this
I understand that in this context the whole app cannot be fully, functionally speaking, pure. It reads initial input from file and receive requests from the external environment, all of which renders functions that deal with these parts not referentially transparent. And almost every program will have these small portions of non-functional code, but in an attempt of trying not to add more of these non-functional functions to the app and this app being just to exercise some functional programming, I am not persisting record to a database or something, because if this were the case, problem would be solved as I could just call a function to update record on the database.
My best idea so far is: The endpoints are created with a routes function from Compojure, so in the /invite endpoint I should process the new record' vector, and recreate the /rank endpoint supplying it with record'. This part of recreating /rank is what I am struggling at, I am trying to call routes again and define again all the endpoints in the hope that it will override the original call of routes, but as expected, as I believe Compojure follows functional programming, once created, the routes are immutable and a new call of routes won't override anything, it will just create new routes that will be left in the void, not attached to the HTTP requests.
So, is it possible to do what I want with pure functional code? Or it is unavoidable to break referential transparency and I should persist record to a file or database to update it?
PS.: I don't know if this is relevant, but I am new to Clojure and to any sort of web interaction.
Clojure (in contrast to Haskell) is not pure and has its own constructs to manage changes to shared state. It doesn't isolate impureness using a type system (like IO monad in Haskell) but promotes using pure functions and managing the state with different types of references (atom, agent, ref) defining a clear semantics how and when the state is changed.
For your scenario Clojure's atom would be the simplest solution. It provides a clear contract on how its state is managed.
I would create a var holding your record within an atom:
(def record (atom [])) ;; initial record is empty
Then in your rank endpoint you could use its value using deref or with # as a syntactic sugar:
(GET "/rank" []
(calculate-rank #record))
Whereas your invite endpoint could update your record value atomically using swap!:
(POST "/invite/:id" [id]
(invite id)
(swap! record calculate-new-rank id)
(create-response))
Your calculate-new-rank function would look like that:
(defn calculate-new-rank [current-record id]
;; do some calculations
;; create a new record value and return it
(let [new-record ...]
new-record))
Your function will be called with the current version of the data stored in the atom and other optional parameters and the result of your function will be installed as the new value of your atom.
How can I list all the routes on a handler function? I'm looking for behavior similar to rails' rake routes. For example:
(defroutes foo-routes
(GET "/foo/:foo-id"
[foo-id]
"bar response")
(GET "/bar/:bar-id"
[bar-id]
"foo response"))
Is it then possible to extract a map from foo-bar-routes containing the following?
{:GET "/foo/:foo-id"
:GET "/bar/:bar-id"}
I don't think it is possible. defroutes is a macro that returns a ring handler. GET is a macro that returns a route. Route is again just a function that calls related handler only if method and path are matching. So in the end your foo-routes is just a clojure function that is composed of other functions defined by your routes and it doesn't maintain such map. If you need to get such map, maybe you can maintain it in your code yourself and generate routes out of this map.
I know this thread is quite old but I had the same question and could resolve it by myself, here's what I've got:
Assuming you defined your API this way:
(def my-api (compojure.api.api/api ...))
Then you can easily list the routes you defined that way:
(->> (.-get-routes my-api {})
(map (juxt second first)))
I am trying to write a middleware for converting all the string object ids in the request to ObjectId objects.
I am achieving this using the following:
(defn get-object-id
[id]
(when (and (string? id) (re-matches object-id-regex id))
(ObjectId. id)))
(defn maybe-obj->object-id [obj]
(or (get-object-id obj) obj))
(defn- convert-string->object-ids [obj]
(cwalk/postwalk
(partial pcommon/maybe-obj->object-id) obj))
(defn warp-params-string->objectid
"convert strings to object ids"
[handler]
(fn [request]
(handler (update-in request [:params] convert-string->object-ids))))
This is working for all the params coming for json, request params etc. But this is not applying to the route params, e.g. :fst for url "/:fst". I looked at the GET macro and the route params are being injected somewhere inside that macro. However since GET/POST etc are executed last, my middlewares do not have access to these. Any graceful way of achieving this.
Those /:foo/:bar-style parameters get bound as a result of pattern matching on URIs, with the patterns specified in the individual routes' definitions. Outer layers don't even know what the patterns look like. So, not really possible to lift processing of these to middleware.
Instead, you could write a macro, say with-preprocessed-params, to wrap your route handlers' bodies in. If it ends up being useful in many handlers, you can additionally provide your own versions of GET & Co., delegating to Compojure's macros with the body wrapped in your param-processing macro.
That's not really a good solution if you were hoping to use the results of this preprocessing in further layers of middleware. In that case, assuming you're happy to leave matching actual URI path segments to the core handler layer, you can perform your preprocessing of other parameter types in a piece of middleware, then use your GET & Co. variants to preprocess the route parameters only.