I am new to Closure and trying out Ring and Compojure. I want to make an HTTP request when a user hits a route (to a third party API) and then use that response data in my HTML template
. I know this is likely a very easy thing to pull off - but being new to language and the syntax I am a bit lost.
(defroutes app
(GET "/" request
; try to GET "https://third-party-api" and do something with the response
)
)
What is the best practice and format for this - It's possible I am missing some key concepts in routing / response expectations here. Thanks very much!
I recommend the library clj-http for making http requests. You can find many examples on the linked page for how to use it.
Your usage of clj-http may look something like this:
(ns my-app.core
(:require [clj-http.client :as client]))
...
(defn get-api-data []
(:body (client/get "https://third-party-api" {:as :json})))
Note that clj-http.client/get returns a map that includes things like the response status code and headers.
If you use the {:as :json} option to coerce the response into json, make sure to include cheshire in your project.clj (assuming you're using leiningen)
:dependencies [...
[clj-http "3.9.0"]
[cheshire "5.8.0"]]
Documentation on ring requests and responses can be found here.
A large portion of the power in ring is its concept of middlewares. Most of the "nice" features that you'd want in an http server can be found as middlewares in ring itself or other libraries. For example, if you want all of your responses to be serialized as json by default, you might use ring-json
If you're trying to get something "that just works", up and running quickly with a few examples present, Luminus may be useful. It's a curated collection of libraries that prove useful for most webservers. (Disclaimer: I've only minimally experimented with Luminus, opting to understand more explicitly my dependencies).
I personally use compojure sweet at the start of most of my webservice projects, it includes some nicer routing features (including path params) and a swagger UI for testing your endpoints. Unfortunately, it uses its own form of destructuring and includes a bit more magic and "just needing to know" than I'd like, but I'm yet to find something that works better for me.
Related
Synopsis
Please note, this question isn't about how to serve a static file — that's working — it's about the special case when wrap-file delivers an index file by default and, for lack of a file extension in the URL, the wrong mime-type is being assigned to the Content-Type header.
How does one get the correct mime type on index files served by default?
Current answers don't address how to do this yet, and the workaround
I've come up with doesn't scale.
Working Code
Here's a simplified fragment from a Clojure application using Compojure and Ring middleware:
(def app
(-> handler
(wrap-file "public") ; If the route is a static file in public, serve it instead
(wrap-content-type))) ; Deduce and add the proper Content-Type header
The intent is to serve up any routes, but if there's a local file in the public directory serve it instead, finally add a meaningful Content-Type header with the corresponding mime type. All this works perfectly.
The Problem
When I browse to the base URL, it does serve index.html as expected, but it does not get a Content-Type of text/html, but rather application/octet-stream.
ring.middleware.file/wrap-file indicates that the index-files? option defaults to true, and this explains why a URL with no paths correctly serves the file. This appears to be the pedantic way of serving static resources.
ring.middleware.content-type/wrap-content-type indicates that the mime-type is deduced by the file extension in the URI, and without one defaults to application/octet-stream. As the URL contain no filename, this function is 'properly' doing what it states.
This begs the question, how to assign Content-Type by contents of the response's body?
However, it's ill-advised to have middleware reads the :body common problems, because it's a mutable InputStream that can only be read once. So that's obviously not the right way.
Is there a better way to serve the index.html by default?
An Ugly Workaround
The current ugly workaround is to have a special-case route that sets the Content-Type manually. <cringe/>
Worse, this solution doesn't scale, should an index file be served from
a subdirectory.
Consequently, I'm looking for a Middleware solution, not a routing hack.
Experiments
Exploring the Execution Order of the Middleware and Its Consequences:
Admittedly, although I understand the thread macro (->) in that (-> x A B) transforms into (B (A x) ), I still get a little jumbled in my head when working out the order that the execution flow resolved through a middle-ware chain to an eventual handler with routes. The reason for this stumbling is that code can mess with the request before calling the the handler it was passed, as well as fiddle with the response before returning. The order things need to be in doesn't feel "obvious" to know when I'm augmenting the request with details going in or twiddling with the response coming out, or the more complicated case of doing a different behavior based on some condition.
e.g., Does wrap-file happen "before" or "after" the handler has constructed a response, as the order matters in the threading? I feel this should be more intuitive to me, without having to run to the source code as much as I'm doing.
As it appears possible to have middleware applied only when a specific route matches, it may be that I'm making more of a distinction between Middleware and Handlers than perhaps I should.
Swapping the order (to test the threading-order assumptions) does not do what you think:
(def app ; THIS IS AN EXAMPLE OF BROKEN CODE - DON'T USE IT
(-> handler
(wrap-content-type))) ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
(wrap-file "public") ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
It "works," but for the wrong reason. The index.html file will get delivered and renders "properly," but only because there is no Content-Type added. The browser, for lack of a specified mime type, makes an educated guess and happens to guess correctly.
Since the goal is to have a Content-Type in the header, this suggests the threaded order was correct to start with.
What Middleware Should Be Used To Deliver Index Pages?
So with information in hand of what not to do, what is it I should be doing to deliver the default status index.html file when the URL doesn't specify it by name, since there's no extension to examine?
WORKS — http://localhost/index.html (serves page with correct mime type)
BROKEN — http://localhost/ (serves same page, but with the wrong content type, so the browser tries to download it)
Is there a better middleware stack, or even a recommended one, that someone could walk me through?
UPDATE 2020-05-24: Submitted Ring Issue 480; turns out this may be a design bug looking for a contributor.
This gives you a server which will serve index.html if present inside a resources/public/ folder.
(ns core
(:require [compojure.core :refer [routes GET]]
[ring.middleware.defaults :refer [wrap-defaults]]
[org.httpkit.server :as http-kit]))
(def handler
(routes
(GET "/foo" [] "Hello Foo")
(GET "/bar" [] "Hello Bar")))
(def app
(-> handler
(wrap-defaults {:static {:resources "public"
:files "resources/public"}})))
(def server (http-kit/run-server app {:port 8889}))
(comment
;; To stop the server
(server))
I'm using wrap-defaults as it provides a nice way to get a server up and running, while still providing a lot of flexibility to drop in customisations as required.
In this case I'm telling it to use public as a resources folder and also handing it resources/public to files so it can correctly wrap the files to be served.
ring.middleware.content-type defaults to application/octet-stream when it has insufficient information to guess the content type of the file it is serving.
If you specifically just want to serve files+provide routing, the answer I have given above is sufficient, if you want to explicitly return a Content-Type text/html for index.html, then you will need to wrap the content type using [ring.util.response :refer [content-type]].
So for example:
(GET "/" [] (content-type (io/resource "index.html") "text/html"))
I've normally done this by detecting the file extension in the request url and then returning the correct content-type, with a special case for things like index.html.
You need at least these deps, this is in deps.edn format, but just change it to [ring/ring-core "1.8.0"] for example if you need it in lein's project.clj form instead:
ring/ring-core {:mvn/version "1.8.0"}
ring/ring-defaults {:mvn/version "0.3.2"}
http-kit {:mvn/version "2.3.0"}
compojure {:mvn/version "1.6.1"}
Let me know if you have any issues!
I am trying to create a web app in clojure.
i have used clojurescript om and react.
there are two files core.cljs and db.clj.
core.cljs contains ui for login page and db.clj contains all database connections.
Now i am trying to call a db.clj method add-user[username password] in
core.cljs.
In db.clj
(defn add-user [username,password]
(sql/with-connection db
(sql/insert-values :users [:username :password]
[username password])))
In core.cljs
(dom/button #js {:ref "submit"
:onClick (fn[e](add-user usname passwrd))}"submit")
But i am not able to call that method in core.cljs.
It shows some error message like
clojure.lang.ExceptionInfo : failed compiling file:src\login_page\core.cljs
clojure.lang.ExceptionInfo : No such namespace: login_page.db, could not locate login_page/db.cljs, login_page/db.cljc, or Closure namespace "login_page.db"
Rename db.clj to either db.cljs or db.cljc. That should get you past the 'No such namespace' error message.
That's the basic gist of it. Of course your dependencies on clj libraries will have to be removed - that might be the reason for the negative comment below. Alter your code so that you use a simple atom as your database. That should get you developing.
And you can wait for a much better answer than this one that will show you how to get the Client and Server communication setup. But it may not come because, as pointed out in the comments, there is already documentation for this, and unfortunately quite a few choices that have to be made. Another unfortunate thing is that the way to do it now may not be the way to do it early next year. Watch the Om-Next space!
I've never had any problems compiling .cljs or .cljc files. You just have to set up your lein project.clj file properly. There will be plenty of examples if you Google around, or you can take a look at the following small Github project: https://github.com/chrismurrph/passing-time - no need to worry about the code, just look at its project.clj file.
I have the following Compojure routes:
(defroutes my-handler
(GET "/:my-model-id" [id] (render-my-model (parse-int id))))
It is unfortunate that, for every route that I define this way, I have to manually add a call to parse the incoming integer.
I have created Ring middleware that crawls through any form-params and request-params, and parses anything that looks like it might be an integer. However, this middleware does not apply to the custom-defined Compojure routes.
Does anybody know how I could get Compojure to automatically handle the integer-parsing? Can I somehow hook it up to my existing middleware?
Unfortunately compojure will directly invoke the function that is generated from your route definition after it has parsed the params.
AFAIC the only way to get in between is either to modify compojures codebase directly or to use Robert Hooke (by technomancy) on assoc-route-params in https://github.com/weavejester/compojure/blob/master/src/compojure/core.clj#L30
NOTE: I resolved my issue. However, it took a number of incremental changes. If you happen upon this page, feel free to checkout my github below to see how I made this application work.
I am using http-kit to post a request to btc-china. I want to use their trading api. I am able to do this just fine with python, but for some reason I keep getting 401s with clojure and http-kit. I've posted a snippit of code below which may show that I am not using http-kit correctly. In addition to that, here is a github for my full code if you wish to look at that: https://github.com/gilmaso/btc-trading
Here are the btc-china api docs: http://btcchina.org/api-trade-documentation-en
(def options {:timeout 2000 ; ms
:query-params (sorted-map :tonce tonce
:accesskey access-key
:requestmethod request-method
:id tonce
:method method
:params "")
:headers {"Authorization" auth-string
"Json-Rpc-Tonce" tonce}})
(client/post (str "https://" base-url) options
(fn [{:keys [status headers body error]}] ;; asynchronous handle response
(if error
(println "Failed, exception is " error)
(println "Async HTTP GET: " status))))
quoting from the example on the bttchina site:
# The order of params is critical for calculating a correct hash
clojure hash maps are unordered, and you cannot use a clojure hash map literal to provide the input if order is significant
I had very similar problem with bitstamp api. The solution was to replace :query-params with :form-params. Then the parameters are sent in the body. I noticed that in your api you are manually sending then in the body. It looks like using :form-params might help in your case as well.
I am using liberator with ring/compojure and would like to handle authorization using the liberator defresource macros. I can easily get handle-ok to output html that is recognized by the browser, but handle-unauthorized will have the html output in pre tags.
I'm suspecting that my not being able to find out how to do this means that there is a good reason not to do it this way. All the examples I see online show handle-unauthorized returning text, but I was hoping to display a custom page.
Here is (one version of) the code I was using. I am using Hiccup:
(defresource deny-all
:authorized? (fn [_] false)
:available-media-types ["text/html"]
:handle-ok (fn [ctx] (html5 [:body [:h1 "Yes!"]])))
:handle-unauthorized (fn [ctx] (html5 [:body [:h1 "Noooo!"]])))
What I get back from the browser is a literal
<!DOCTYPE html>
<html><body><h1>Noooo!</h1></body></html>
</pre>
If I change authorized? to return true, then it outputs the html correctly.
I've tried returning ring-style responses, but those raise errors too. What am I missing?
In REST talk, html is one of many possible representations of a resource. Since the resource is unauthorized, no representation should be returned, html or otherwise. Instead, the client, instructed by the 401 status error, should take a different course of action, such as requesting a login page.
Most apps written in web frameworks will not return a 401, but instead redirect to the authorization page. This is possible in Liberator too, because nothing stops you from handling the authorization in the resource itself (handle-ok with logic).
This might be a breach of practice. I posted an issue on Liberator's github to ask for the opinion of people more knowledgeable in the recipes of RESTful cooking.
At any rate, the reason you see your html wrapped in a pre tag is the result of the following factors:
Content negotiation is not available for unauthorized resources in Liberator. The 401's body is always of type text/plain.
When you specify a html string as the response, it will render as is.
When using Chrome development tools, upon inspecting the source, you will see the html string wrapped in a pre tag.
I hope this helps.