I am using liberator with ring/compojure and would like to handle authorization using the liberator defresource macros. I can easily get handle-ok to output html that is recognized by the browser, but handle-unauthorized will have the html output in pre tags.
I'm suspecting that my not being able to find out how to do this means that there is a good reason not to do it this way. All the examples I see online show handle-unauthorized returning text, but I was hoping to display a custom page.
Here is (one version of) the code I was using. I am using Hiccup:
(defresource deny-all
:authorized? (fn [_] false)
:available-media-types ["text/html"]
:handle-ok (fn [ctx] (html5 [:body [:h1 "Yes!"]])))
:handle-unauthorized (fn [ctx] (html5 [:body [:h1 "Noooo!"]])))
What I get back from the browser is a literal
<!DOCTYPE html>
<html><body><h1>Noooo!</h1></body></html>
</pre>
If I change authorized? to return true, then it outputs the html correctly.
I've tried returning ring-style responses, but those raise errors too. What am I missing?
In REST talk, html is one of many possible representations of a resource. Since the resource is unauthorized, no representation should be returned, html or otherwise. Instead, the client, instructed by the 401 status error, should take a different course of action, such as requesting a login page.
Most apps written in web frameworks will not return a 401, but instead redirect to the authorization page. This is possible in Liberator too, because nothing stops you from handling the authorization in the resource itself (handle-ok with logic).
This might be a breach of practice. I posted an issue on Liberator's github to ask for the opinion of people more knowledgeable in the recipes of RESTful cooking.
At any rate, the reason you see your html wrapped in a pre tag is the result of the following factors:
Content negotiation is not available for unauthorized resources in Liberator. The 401's body is always of type text/plain.
When you specify a html string as the response, it will render as is.
When using Chrome development tools, upon inspecting the source, you will see the html string wrapped in a pre tag.
I hope this helps.
Related
Synopsis
Please note, this question isn't about how to serve a static file — that's working — it's about the special case when wrap-file delivers an index file by default and, for lack of a file extension in the URL, the wrong mime-type is being assigned to the Content-Type header.
How does one get the correct mime type on index files served by default?
Current answers don't address how to do this yet, and the workaround
I've come up with doesn't scale.
Working Code
Here's a simplified fragment from a Clojure application using Compojure and Ring middleware:
(def app
(-> handler
(wrap-file "public") ; If the route is a static file in public, serve it instead
(wrap-content-type))) ; Deduce and add the proper Content-Type header
The intent is to serve up any routes, but if there's a local file in the public directory serve it instead, finally add a meaningful Content-Type header with the corresponding mime type. All this works perfectly.
The Problem
When I browse to the base URL, it does serve index.html as expected, but it does not get a Content-Type of text/html, but rather application/octet-stream.
ring.middleware.file/wrap-file indicates that the index-files? option defaults to true, and this explains why a URL with no paths correctly serves the file. This appears to be the pedantic way of serving static resources.
ring.middleware.content-type/wrap-content-type indicates that the mime-type is deduced by the file extension in the URI, and without one defaults to application/octet-stream. As the URL contain no filename, this function is 'properly' doing what it states.
This begs the question, how to assign Content-Type by contents of the response's body?
However, it's ill-advised to have middleware reads the :body common problems, because it's a mutable InputStream that can only be read once. So that's obviously not the right way.
Is there a better way to serve the index.html by default?
An Ugly Workaround
The current ugly workaround is to have a special-case route that sets the Content-Type manually. <cringe/>
Worse, this solution doesn't scale, should an index file be served from
a subdirectory.
Consequently, I'm looking for a Middleware solution, not a routing hack.
Experiments
Exploring the Execution Order of the Middleware and Its Consequences:
Admittedly, although I understand the thread macro (->) in that (-> x A B) transforms into (B (A x) ), I still get a little jumbled in my head when working out the order that the execution flow resolved through a middle-ware chain to an eventual handler with routes. The reason for this stumbling is that code can mess with the request before calling the the handler it was passed, as well as fiddle with the response before returning. The order things need to be in doesn't feel "obvious" to know when I'm augmenting the request with details going in or twiddling with the response coming out, or the more complicated case of doing a different behavior based on some condition.
e.g., Does wrap-file happen "before" or "after" the handler has constructed a response, as the order matters in the threading? I feel this should be more intuitive to me, without having to run to the source code as much as I'm doing.
As it appears possible to have middleware applied only when a specific route matches, it may be that I'm making more of a distinction between Middleware and Handlers than perhaps I should.
Swapping the order (to test the threading-order assumptions) does not do what you think:
(def app ; THIS IS AN EXAMPLE OF BROKEN CODE - DON'T USE IT
(-> handler
(wrap-content-type))) ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
(wrap-file "public") ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
It "works," but for the wrong reason. The index.html file will get delivered and renders "properly," but only because there is no Content-Type added. The browser, for lack of a specified mime type, makes an educated guess and happens to guess correctly.
Since the goal is to have a Content-Type in the header, this suggests the threaded order was correct to start with.
What Middleware Should Be Used To Deliver Index Pages?
So with information in hand of what not to do, what is it I should be doing to deliver the default status index.html file when the URL doesn't specify it by name, since there's no extension to examine?
WORKS — http://localhost/index.html (serves page with correct mime type)
BROKEN — http://localhost/ (serves same page, but with the wrong content type, so the browser tries to download it)
Is there a better middleware stack, or even a recommended one, that someone could walk me through?
UPDATE 2020-05-24: Submitted Ring Issue 480; turns out this may be a design bug looking for a contributor.
This gives you a server which will serve index.html if present inside a resources/public/ folder.
(ns core
(:require [compojure.core :refer [routes GET]]
[ring.middleware.defaults :refer [wrap-defaults]]
[org.httpkit.server :as http-kit]))
(def handler
(routes
(GET "/foo" [] "Hello Foo")
(GET "/bar" [] "Hello Bar")))
(def app
(-> handler
(wrap-defaults {:static {:resources "public"
:files "resources/public"}})))
(def server (http-kit/run-server app {:port 8889}))
(comment
;; To stop the server
(server))
I'm using wrap-defaults as it provides a nice way to get a server up and running, while still providing a lot of flexibility to drop in customisations as required.
In this case I'm telling it to use public as a resources folder and also handing it resources/public to files so it can correctly wrap the files to be served.
ring.middleware.content-type defaults to application/octet-stream when it has insufficient information to guess the content type of the file it is serving.
If you specifically just want to serve files+provide routing, the answer I have given above is sufficient, if you want to explicitly return a Content-Type text/html for index.html, then you will need to wrap the content type using [ring.util.response :refer [content-type]].
So for example:
(GET "/" [] (content-type (io/resource "index.html") "text/html"))
I've normally done this by detecting the file extension in the request url and then returning the correct content-type, with a special case for things like index.html.
You need at least these deps, this is in deps.edn format, but just change it to [ring/ring-core "1.8.0"] for example if you need it in lein's project.clj form instead:
ring/ring-core {:mvn/version "1.8.0"}
ring/ring-defaults {:mvn/version "0.3.2"}
http-kit {:mvn/version "2.3.0"}
compojure {:mvn/version "1.6.1"}
Let me know if you have any issues!
I am new to Closure and trying out Ring and Compojure. I want to make an HTTP request when a user hits a route (to a third party API) and then use that response data in my HTML template
. I know this is likely a very easy thing to pull off - but being new to language and the syntax I am a bit lost.
(defroutes app
(GET "/" request
; try to GET "https://third-party-api" and do something with the response
)
)
What is the best practice and format for this - It's possible I am missing some key concepts in routing / response expectations here. Thanks very much!
I recommend the library clj-http for making http requests. You can find many examples on the linked page for how to use it.
Your usage of clj-http may look something like this:
(ns my-app.core
(:require [clj-http.client :as client]))
...
(defn get-api-data []
(:body (client/get "https://third-party-api" {:as :json})))
Note that clj-http.client/get returns a map that includes things like the response status code and headers.
If you use the {:as :json} option to coerce the response into json, make sure to include cheshire in your project.clj (assuming you're using leiningen)
:dependencies [...
[clj-http "3.9.0"]
[cheshire "5.8.0"]]
Documentation on ring requests and responses can be found here.
A large portion of the power in ring is its concept of middlewares. Most of the "nice" features that you'd want in an http server can be found as middlewares in ring itself or other libraries. For example, if you want all of your responses to be serialized as json by default, you might use ring-json
If you're trying to get something "that just works", up and running quickly with a few examples present, Luminus may be useful. It's a curated collection of libraries that prove useful for most webservers. (Disclaimer: I've only minimally experimented with Luminus, opting to understand more explicitly my dependencies).
I personally use compojure sweet at the start of most of my webservice projects, it includes some nicer routing features (including path params) and a swagger UI for testing your endpoints. Unfortunately, it uses its own form of destructuring and includes a bit more magic and "just needing to know" than I'd like, but I'm yet to find something that works better for me.
Whats the best way to get a page from a website, and handling 404 redirects within clojure.
I've used enlive, but it automatically transforming the page, which I do not want as I want to store the HTML in a DB for future reference.
(defn fetch-page [url]
(html/html-resource (java.net.URL. url)))
I've came across slurp to get raw html content, but I don't know if this is the best method to retrieve things from external websites.
The second issue I have is handling 404, what is the best way to handle it, my clojure program ungracefully exists when it encounters a 404.
Code:
(println (slurp "http://www.google.com/doesnotexists.html"))
Output:
CompilerException java.io.FileNotFoundException: http://www.google.com/doesnotexists.html
404 isn't a redirect, it indicates "Not found". But regardless, you can handle the exception the same way you handle any exception in Clojure ... with try/catch:
(try
(slurp "http://www.google.com/doesnotexists.html")
(catch java.io.FileNotFoundException ex
<handle exception...>))
NOTE: I resolved my issue. However, it took a number of incremental changes. If you happen upon this page, feel free to checkout my github below to see how I made this application work.
I am using http-kit to post a request to btc-china. I want to use their trading api. I am able to do this just fine with python, but for some reason I keep getting 401s with clojure and http-kit. I've posted a snippit of code below which may show that I am not using http-kit correctly. In addition to that, here is a github for my full code if you wish to look at that: https://github.com/gilmaso/btc-trading
Here are the btc-china api docs: http://btcchina.org/api-trade-documentation-en
(def options {:timeout 2000 ; ms
:query-params (sorted-map :tonce tonce
:accesskey access-key
:requestmethod request-method
:id tonce
:method method
:params "")
:headers {"Authorization" auth-string
"Json-Rpc-Tonce" tonce}})
(client/post (str "https://" base-url) options
(fn [{:keys [status headers body error]}] ;; asynchronous handle response
(if error
(println "Failed, exception is " error)
(println "Async HTTP GET: " status))))
quoting from the example on the bttchina site:
# The order of params is critical for calculating a correct hash
clojure hash maps are unordered, and you cannot use a clojure hash map literal to provide the input if order is significant
I had very similar problem with bitstamp api. The solution was to replace :query-params with :form-params. Then the parameters are sent in the body. I noticed that in your api you are manually sending then in the body. It looks like using :form-params might help in your case as well.
I am trying to write Webdriver checks using Clojure. If I was using an object oriented language, I would use the Page Object Pattern. I think modeling a page as an object makes sense, I could create some java classes for the page objects and all would be well.
I want to know if there are any alternatives to the page object pattern using a functional style that maintain the same level of clarity.
A page (especially a RESTful one), can be thought of as a function from request to render (and, if you want to take the next step, the render exposes some set of new requests).
The translation from sending a request to page to applying a function to arguments is simple, and also quite comprehensive.
If you are providing a complex webapp, try taking a functional view of requests. GET can retrieve data, but should not modify server side state, use PUT to create a resource, use POST for mutation.
Once you write your controllers in this way, you can do quite a bit of testing without webdrivers. It should mostly suffice to provide a mockup of the request input to the controller, and verify some properties of the rendered result (for GET) or the storage state (for POST AND PUT).
I have even found it useful to break off request parsing and rendering into separate functions, in order to simplify the testing of the data processing that should happen in the middle, ie.:
(defn parse-home
[request]
(let [user-id (-> request :params :session :id)
account (get-user user-id)]
{:user-id user-id
:account account}))
(defn do-home
[user-id account]
(let [messages (get-messages account)
feed (generate-feed user-id)]
(update-user user-id :last-visited (java.util.Date.))
{:messages messages
:feed feed}))
(defn render-home
[request messages feed account]
(let [messages (mapv summarize (filter important messages))
feed (sort-by :priority feed)
template (-> request :page :template)]
(render template (assoc request :data {:messages messages :feed feed :account account}))))
(defn home
[request]
(let [{:keys [user-id account]} (parse-home request)
{:keys [messages feed]} (do-home user-id account)
body (render-home request messages feed account)]
{:status 200
:content-type "text/html"
:body body}))
Each element of the home page logic can be verified by providing one of these functions with the right input, and verifying some properties of the output. There is no need to mock up state or simulate page interaction unless you are also using clojurescript on the front end (and even in that case the logic you want to verify can be abstracted from the interface of the browser to avoid the need for replay testing).