Can Secretary dispatch routes based on a submitted browser url, the way many javascript routing frameworks work? I can't seem to find any examples of wiring this up.
For example, when you enter http://myapp.com/#/my/route in the browser's url, I'd like that to dispatch as though I had programatically entered (secretary/dispatch! "/my/route") in the repl.
It can, but not by itself. The most common way of doing this is with Google Closure, as in the README.md example:
(let [h (History.)]
(goog.events/listen h EventType/NAVIGATE #(secretary/dispatch! (.-token %)))
(doto h (.setEnabled true)))
When you call setEnabled in the example above, the NAVIGATE event will automatically fire for the current location which will cause dispatch! to be invoked.
Related
Synopsis
Please note, this question isn't about how to serve a static file — that's working — it's about the special case when wrap-file delivers an index file by default and, for lack of a file extension in the URL, the wrong mime-type is being assigned to the Content-Type header.
How does one get the correct mime type on index files served by default?
Current answers don't address how to do this yet, and the workaround
I've come up with doesn't scale.
Working Code
Here's a simplified fragment from a Clojure application using Compojure and Ring middleware:
(def app
(-> handler
(wrap-file "public") ; If the route is a static file in public, serve it instead
(wrap-content-type))) ; Deduce and add the proper Content-Type header
The intent is to serve up any routes, but if there's a local file in the public directory serve it instead, finally add a meaningful Content-Type header with the corresponding mime type. All this works perfectly.
The Problem
When I browse to the base URL, it does serve index.html as expected, but it does not get a Content-Type of text/html, but rather application/octet-stream.
ring.middleware.file/wrap-file indicates that the index-files? option defaults to true, and this explains why a URL with no paths correctly serves the file. This appears to be the pedantic way of serving static resources.
ring.middleware.content-type/wrap-content-type indicates that the mime-type is deduced by the file extension in the URI, and without one defaults to application/octet-stream. As the URL contain no filename, this function is 'properly' doing what it states.
This begs the question, how to assign Content-Type by contents of the response's body?
However, it's ill-advised to have middleware reads the :body common problems, because it's a mutable InputStream that can only be read once. So that's obviously not the right way.
Is there a better way to serve the index.html by default?
An Ugly Workaround
The current ugly workaround is to have a special-case route that sets the Content-Type manually. <cringe/>
Worse, this solution doesn't scale, should an index file be served from
a subdirectory.
Consequently, I'm looking for a Middleware solution, not a routing hack.
Experiments
Exploring the Execution Order of the Middleware and Its Consequences:
Admittedly, although I understand the thread macro (->) in that (-> x A B) transforms into (B (A x) ), I still get a little jumbled in my head when working out the order that the execution flow resolved through a middle-ware chain to an eventual handler with routes. The reason for this stumbling is that code can mess with the request before calling the the handler it was passed, as well as fiddle with the response before returning. The order things need to be in doesn't feel "obvious" to know when I'm augmenting the request with details going in or twiddling with the response coming out, or the more complicated case of doing a different behavior based on some condition.
e.g., Does wrap-file happen "before" or "after" the handler has constructed a response, as the order matters in the threading? I feel this should be more intuitive to me, without having to run to the source code as much as I'm doing.
As it appears possible to have middleware applied only when a specific route matches, it may be that I'm making more of a distinction between Middleware and Handlers than perhaps I should.
Swapping the order (to test the threading-order assumptions) does not do what you think:
(def app ; THIS IS AN EXAMPLE OF BROKEN CODE - DON'T USE IT
(-> handler
(wrap-content-type))) ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
(wrap-file "public") ; WRONG ORDER - DON'T DO THIS (EXAMPLE ONLY)
It "works," but for the wrong reason. The index.html file will get delivered and renders "properly," but only because there is no Content-Type added. The browser, for lack of a specified mime type, makes an educated guess and happens to guess correctly.
Since the goal is to have a Content-Type in the header, this suggests the threaded order was correct to start with.
What Middleware Should Be Used To Deliver Index Pages?
So with information in hand of what not to do, what is it I should be doing to deliver the default status index.html file when the URL doesn't specify it by name, since there's no extension to examine?
WORKS — http://localhost/index.html (serves page with correct mime type)
BROKEN — http://localhost/ (serves same page, but with the wrong content type, so the browser tries to download it)
Is there a better middleware stack, or even a recommended one, that someone could walk me through?
UPDATE 2020-05-24: Submitted Ring Issue 480; turns out this may be a design bug looking for a contributor.
This gives you a server which will serve index.html if present inside a resources/public/ folder.
(ns core
(:require [compojure.core :refer [routes GET]]
[ring.middleware.defaults :refer [wrap-defaults]]
[org.httpkit.server :as http-kit]))
(def handler
(routes
(GET "/foo" [] "Hello Foo")
(GET "/bar" [] "Hello Bar")))
(def app
(-> handler
(wrap-defaults {:static {:resources "public"
:files "resources/public"}})))
(def server (http-kit/run-server app {:port 8889}))
(comment
;; To stop the server
(server))
I'm using wrap-defaults as it provides a nice way to get a server up and running, while still providing a lot of flexibility to drop in customisations as required.
In this case I'm telling it to use public as a resources folder and also handing it resources/public to files so it can correctly wrap the files to be served.
ring.middleware.content-type defaults to application/octet-stream when it has insufficient information to guess the content type of the file it is serving.
If you specifically just want to serve files+provide routing, the answer I have given above is sufficient, if you want to explicitly return a Content-Type text/html for index.html, then you will need to wrap the content type using [ring.util.response :refer [content-type]].
So for example:
(GET "/" [] (content-type (io/resource "index.html") "text/html"))
I've normally done this by detecting the file extension in the request url and then returning the correct content-type, with a special case for things like index.html.
You need at least these deps, this is in deps.edn format, but just change it to [ring/ring-core "1.8.0"] for example if you need it in lein's project.clj form instead:
ring/ring-core {:mvn/version "1.8.0"}
ring/ring-defaults {:mvn/version "0.3.2"}
http-kit {:mvn/version "2.3.0"}
compojure {:mvn/version "1.6.1"}
Let me know if you have any issues!
I am new to Closure and trying out Ring and Compojure. I want to make an HTTP request when a user hits a route (to a third party API) and then use that response data in my HTML template
. I know this is likely a very easy thing to pull off - but being new to language and the syntax I am a bit lost.
(defroutes app
(GET "/" request
; try to GET "https://third-party-api" and do something with the response
)
)
What is the best practice and format for this - It's possible I am missing some key concepts in routing / response expectations here. Thanks very much!
I recommend the library clj-http for making http requests. You can find many examples on the linked page for how to use it.
Your usage of clj-http may look something like this:
(ns my-app.core
(:require [clj-http.client :as client]))
...
(defn get-api-data []
(:body (client/get "https://third-party-api" {:as :json})))
Note that clj-http.client/get returns a map that includes things like the response status code and headers.
If you use the {:as :json} option to coerce the response into json, make sure to include cheshire in your project.clj (assuming you're using leiningen)
:dependencies [...
[clj-http "3.9.0"]
[cheshire "5.8.0"]]
Documentation on ring requests and responses can be found here.
A large portion of the power in ring is its concept of middlewares. Most of the "nice" features that you'd want in an http server can be found as middlewares in ring itself or other libraries. For example, if you want all of your responses to be serialized as json by default, you might use ring-json
If you're trying to get something "that just works", up and running quickly with a few examples present, Luminus may be useful. It's a curated collection of libraries that prove useful for most webservers. (Disclaimer: I've only minimally experimented with Luminus, opting to understand more explicitly my dependencies).
I personally use compojure sweet at the start of most of my webservice projects, it includes some nicer routing features (including path params) and a swagger UI for testing your endpoints. Unfortunately, it uses its own form of destructuring and includes a bit more magic and "just needing to know" than I'd like, but I'm yet to find something that works better for me.
I'm trying to re-write some Common Lisp web-scraping code in Racket.
In Common Lisp, I'm POSTing a login request, and storing the cookie-jar for subsequent GETs:
(defun login (username password)
"Logs in to www.example.com. Returns a cookie-jar containing authentication details."
(let ((cookie-jar (make-instance 'drakma:cookie-jar)))
(drakma:http-request "http://www.example.com/login"
:method :post
:parameters `(("username" . ,username) ("password" . ,password))
:cookie-jar cookie-jar)
cookie-jar))
; snip
(defun get-page (page-num cookie-jar)
"Downloads a potentially invalid HTML page containing data to scrape. Returns a string containing the HTML."
(let ((url (concatenate 'string "http://www.example.com/data/" (write-to-string page-num))))
(let ((body (drakma:http-request url :cookie-jar cookie-jar)))
(if (search "No data found." body)
nil
body))))
However, I can't find an equivalent in Racket. The latest HTTP library makes no mention of cookies at all, and AFAICT the cookie library seems more about correctly serializing and deserializing them.
Can anyone suggest a way of re-writing the above CL in Racket without having to implement a bunch of header-parsing stuff?
There is no such functionality in the Racket libraries (at the time of writing, 30 Jan 2014).
There has been some discussion on the Racket users list about this, and there is a third-party library called client-cookies that addresses the issue. At the time of writing that library is not yet production-ready, but is under active development.
NOTE: I resolved my issue. However, it took a number of incremental changes. If you happen upon this page, feel free to checkout my github below to see how I made this application work.
I am using http-kit to post a request to btc-china. I want to use their trading api. I am able to do this just fine with python, but for some reason I keep getting 401s with clojure and http-kit. I've posted a snippit of code below which may show that I am not using http-kit correctly. In addition to that, here is a github for my full code if you wish to look at that: https://github.com/gilmaso/btc-trading
Here are the btc-china api docs: http://btcchina.org/api-trade-documentation-en
(def options {:timeout 2000 ; ms
:query-params (sorted-map :tonce tonce
:accesskey access-key
:requestmethod request-method
:id tonce
:method method
:params "")
:headers {"Authorization" auth-string
"Json-Rpc-Tonce" tonce}})
(client/post (str "https://" base-url) options
(fn [{:keys [status headers body error]}] ;; asynchronous handle response
(if error
(println "Failed, exception is " error)
(println "Async HTTP GET: " status))))
quoting from the example on the bttchina site:
# The order of params is critical for calculating a correct hash
clojure hash maps are unordered, and you cannot use a clojure hash map literal to provide the input if order is significant
I had very similar problem with bitstamp api. The solution was to replace :query-params with :form-params. Then the parameters are sent in the body. I noticed that in your api you are manually sending then in the body. It looks like using :form-params might help in your case as well.
I am trying to write Webdriver checks using Clojure. If I was using an object oriented language, I would use the Page Object Pattern. I think modeling a page as an object makes sense, I could create some java classes for the page objects and all would be well.
I want to know if there are any alternatives to the page object pattern using a functional style that maintain the same level of clarity.
A page (especially a RESTful one), can be thought of as a function from request to render (and, if you want to take the next step, the render exposes some set of new requests).
The translation from sending a request to page to applying a function to arguments is simple, and also quite comprehensive.
If you are providing a complex webapp, try taking a functional view of requests. GET can retrieve data, but should not modify server side state, use PUT to create a resource, use POST for mutation.
Once you write your controllers in this way, you can do quite a bit of testing without webdrivers. It should mostly suffice to provide a mockup of the request input to the controller, and verify some properties of the rendered result (for GET) or the storage state (for POST AND PUT).
I have even found it useful to break off request parsing and rendering into separate functions, in order to simplify the testing of the data processing that should happen in the middle, ie.:
(defn parse-home
[request]
(let [user-id (-> request :params :session :id)
account (get-user user-id)]
{:user-id user-id
:account account}))
(defn do-home
[user-id account]
(let [messages (get-messages account)
feed (generate-feed user-id)]
(update-user user-id :last-visited (java.util.Date.))
{:messages messages
:feed feed}))
(defn render-home
[request messages feed account]
(let [messages (mapv summarize (filter important messages))
feed (sort-by :priority feed)
template (-> request :page :template)]
(render template (assoc request :data {:messages messages :feed feed :account account}))))
(defn home
[request]
(let [{:keys [user-id account]} (parse-home request)
{:keys [messages feed]} (do-home user-id account)
body (render-home request messages feed account)]
{:status 200
:content-type "text/html"
:body body}))
Each element of the home page logic can be verified by providing one of these functions with the right input, and verifying some properties of the output. There is no need to mock up state or simulate page interaction unless you are also using clojurescript on the front end (and even in that case the logic you want to verify can be abstracted from the interface of the browser to avoid the need for replay testing).