I have set up a WebSocket server using http-kit that should accept web socket connections. It is the basic example shown in the http-kit documentation.
The question is:
How do you create a Clojure client that connects to it?
The client can be any Clojure http library, I don't really mind. I already know about Javascript clients, and I believe that Java has an API that I could use from Clojure. But what I am looking for is a Clojure library that supports websockets both for client and server. I saw nothing in http-kit code that would make it easy.
aleph has support for websockets on both server and client. It can take some time to get used to the asynchronous style and aleph's core abstractions, but it's a very good tool once you get the hang of it.
Gniazdo is a WebSocket client for Clojure. It wraps the Jetty's implementation of the protocol.
http-kit's client does not support WebSocket yet(I can't think of a good API for it). Aleph is a good option for this kind of use case. Another option is http.async.client, I've used it in http-kit's server's websocket unit test: here
For those joining us in 2015: being new to this, I just spent a while trying out all the different options available, and it was pretty difficult to find a library that provides an easy way to set up a simple Clojure WebSocket client. (To be fair, it seems like it's not very common for a WebSocket client to be running in a non-browser/JavaScript context, which is why there seems to be so much emphasis on ClojureScript WebSocket clients.)
Although it is not well-documented, http.async.client ended up being the path of least resistance for me. I was able to successfully read streaming data from a WebSocket server and print it to the console by doing this:
(ns example.core
(:require [http.async.client :as async]))
(def url "ws://localhost:1337")
(defn on-open [ws]
(println "Connected to WebSocket."))
(defn on-close [ws code reason]
(println "Connection to WebSocket closed.\n"
(format "(Code %s, reason: %s)" code reason)))
(defn on-error [ws e]
(println "ERROR:" e))
(defn handle-message [ws msg]
(prn "got message:" msg))
(defn -main []
(println "Connecting...")
(-> (async/create-client)
(async/websocket url
:open on-open
:close on-close
:error on-error
:text handle-message
:byte handle-message))
;; Avoid exiting until the process is interrupted.
(while true))
The infinite loop at the end is just to keep the process from ending. Until I press Ctrl-C, messages received from the socket are printed to STDOUT.
According to this announcement, http-kit has support for web sockets. If you're not bound to the asynchronous facilities that http-kit client offer, you could also use clj-http. They have a very similar interface, it seems (I have use but clj-http yet).
(ns playground.experiments.ws
(:use aleph.http lamina.core))
(defn ws-client [] (websocket-client {:url "ws://echo.websocket.org:80"}))
(defn echo [message]
(let [channel (wait-for-result (ws-client) 500)]
(enqueue channel message)
(let [echo (wait-for-result (read-channel channel) 500)]
(close channel)
echo)))
(echo "Echo me!")
I built a basic websocket client and server, it uses java sockets and wraps websocket frames. It's unique in that the server can accept both regular socket connections as well as websockets, simultaneously.
http://github.com/viperscape/gulfstream
Example client code:
(def clienthandler
(with-conn server
(send! server "i'm here!")
(with-data server data (prn "client received" data))
(prn "client is now disconnected")))
(def ws-conn-details {:host "ws://echo.websocket.org/chat",:handler clienthandler})
(def client-conn (start-client ws-conn-details))
Related
Usually Ring middleware is associated with the use on the server side.
In the post I'll discuss how the concept of ring middleware can be applied to http clients.
A very typical server side example might look like this:
(def server
(-> server-handler
wrap-transit-params
wrap-transit-response))
Desugared:
(def server (wrap-transit-response (wrap-transit-params handler)))
server is a function now, which accepts a request hash-map. Middleware can operate on this data before its send to the handler. It can also operate on the response hash-map that the handler returns. Or on both. It can even manipulate the execution of the handler.
Server
The above middleware could look like this in a very simplified way:
(1.) This operates on data before it gets to the actual handler (request, incoming data), parsing the body and providing the result as value to the :params key. It's called pre-wrap.
(defn wrap-transit-params [handler]
(fn [req]
(handler (assoc req :params (from-transit-str (req :body))))))
(2.) This one manipulates the outgoing data, the response of the server, outgoing data. - It's a post-wrap.
(defn wrap-tranist-response [handler]
(fn [req]
(let [resp (handler req)]
(update resp :body to-transit-str))))
With this a server can receive and respond data as transit.
Client
The same behavior could be desirable for an http-client.
(def client
(-> client-handler
wrap-transit-params
wrap-transit-response))
It turns out that the above middleware cannot easily be reused as client middleware, even though there is some symmetry.
For the client-side they should be implemented like this:
(defn wrap-transit-params [handler]
(fn [req]
(handler (assoc req :body (to-transit-str (req :params))))))
(defn wrap-transit-response [handler]
(fn [req]
(let [resp (handler req)]
(update resp :body from-transit-str))))
Now it could be used like this:
(client {:url "http://..."
:params {:one #{1 2 3}}})
Since in reality there would be much more things invloved, so I think having reusable middleware for both server and client side is utopian.
Though it remains up to discussion if the concept generally makes sense for clients. I could not find any client side middleware on the net.
The server side middleware is usually under the namespace ring.middleware... My concrete question here is if a library providing client side middleware should use this namespace or not.
Unless you believe that code that uses your new library could be written to be completely portable between client and server, I feel that using a different namespace will lead to less confusion. Find a fun name! :)
I would like to write tests for a Pedestal web-service.
If I have :
(defn pong
[request]
(ring-resp/response "pong"))
(defroutes routes[[["/" {:get pong}]]])
How would I write a test for that ?
(deftest alive-system
(testing "ping-pong route"
;; How do I test my route ?
;; If possible :
;; - I would like to have direct access to it
;; ie. no need to bind pedestal to a port would be nice
;; - The ability to omit some interceptors would be nice also,
;; as it would allow me to receive plain Clojure data structures
;; instead of, for instance, JSON which I would have to parse.
...)
Edit:
Here is what I tried :
(deftest alive-system
(testing "ping-pong route"
(let [response (my-other.ns/routes (mock/request :get "/ping"))]
(is (= (:status response) 200))
(is (= (:body response) "pong")))))
But I get an exception :
ERROR in (alive-system) (service_test.clj:13)
Uncaught exception, not in assertion.
expected: nil
actual: java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to clojure.lang.IFn
So after asking on the issue I linked ohpaulez replied :
#nha - Thanks for using Pedestal! Sorry your question didn't get an
answer on StackOverflow - I don't think anyone monitors SO for
Pedestal questions. The best place to ask those kinds of questions is
on the mailing list.
Pedestal ships with its own utility for making requests directly to
the servlet (similar to ring/mock, although I've never used mock
myself) called response-for. The Pedestal Service template produces a
test automatically for you. Check out one of the samples for an
example.
Also note that said response-for doesn't yet support asynchronous responses (so my routes that uses asynchronous interceptors with core.async failed - I had to make them synchronous).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I'm struggling to get my head around how to use Stuart Sierra's component library within a Clojure app. From watching his Youtube video, I think I've got an OK grasp of the problems that led to him creating the library; however I'm struggling to work out how to actually use it on a new, reasonably complex project.
I realise this sounds very vague, but it feels like there's some key concept that I'm missing, and once I understand it, I'll have a good grasp on how to use components. To put it another way, Stuart's docs and video go into the WHAT and WHY of components in considerable detail, but I'm missing the HOW.
Is there any sort of detailed tutorial/walkthrough out there that goes into:
why you'd use components at all for a non-trivial Clojure app
a methodology for how you'd break down the functionality in a non-trivial Clojure app, such that components can be implemented in a reasonably optimal fashion. It's reasonably simple when all you've got is e.g. a database, an app server and a web server tier, but I'm struggling to grasp how you'd use it for a system that has many different layers that all need to work together coherently
ways to approach development/testing/failover/etc. in a non-trivial Clojure app that's been built using components
Thanks in advance
In short, Component is a specialized DI framework. It can set up an injected system given two maps: the system map and the dependency map.
Let's look at a made-up web app (disclaimer, I typed this in a form without actually running it):
(ns myapp.system
(:require [com.stuartsierra.component :as component]
;; we'll talk about myapp.components later
[myapp.components :as app-components]))
(defn system-map [config] ;; it's conventional to have a config map, but it's optional
(component/system-map
;; construct all components + static config
{:db (app-components/map->Db (:db config))
:handler (app-components/map->AppHandler (:handler config))
:server (app-components/map->Server (:web-server config))}))
(defn dependency-map
;; list inter-dependencies in either:
;; {:key [:dependency1 :dependency2]} form or
;; {:key {:name-arg1 :dependency1
;; :name-arg2 :dependency2}} form
{:handler [:db]
:server {:app :handler})
;; calling this creates our system
(def create-system [& [config]]
(component/system-using
(system-map (or config {})
(dependency-map)))
This allows us to call (create-system) to create a new instance of our entire application when we need one.
Using (component/start created-system), we can run a system's services it provides. In this case, it's the webserver that's listening on a port and an open db connection.
Finally, we can stop it with (component/stop created-system) to stop the system from running (eg - stop the web server, disconnect from db).
Now let's look at our components.clj for our app:
(ns myapp.components
(:require [com.stuartsierra.component :as component]
;; lots of app requires would go here
;; I'm generalizing app-specific code to
;; this namespace
[myapp.stuff :as app]))
(defrecord Db [host port]
component/Lifecycle
(start [c]
(let [conn (app/db-connect host port)]
(app/db-migrate conn)
(assoc c :connection conn)))
(stop [c]
(when-let [conn (:connection c)]
(app/db-disconnect conn))
(dissoc c :connection)))
(defrecord AppHandler [db cookie-config]
component/Lifecycle
(start [c]
(assoc c :handler (app/create-handler cookie-config db)))
(stop [c] c))
;; you should probably use the jetty-component instead
;; https://github.com/weavejester/ring-jetty-component
(defrecord Server [app host port]
component/Lifecycle
(start [c]
(assoc c :server (app/create-and-start-jetty-server
{:app (:handler app)
:host host
:port port})))
(stop [c]
(when-let [server (:server c)]
(app/stop-jetty-server server)
(dissoc c :server)))
So what did we just do? We got ourselves a reloadable system. I think some clojurescript developers using figwheel start seeing similarities.
This means we can easily restart our system after we reload code. On to the user.clj!
(ns user
(:require [myapp.system :as system]
[com.stuartsierra.component :as component]
[clojure.tools.namespace.repl :refer (refresh refresh-all)]
;; dev-system.clj only contains: (def the-system)
[dev-system :refer [the-system]])
(def system-config
{:web-server {:port 3000
:host "localhost"}
:db {:host 3456
:host "localhost"}
:handler {cookie-config {}}}
(def the-system nil)
(defn init []
(alter-var-root #'the-system
(constantly system/create-system system-config)))
(defn start []
(alter-var-root #'the-system component/start))
(defn stop []
(alter-var-root #'the-system
#(when % (component/stop %))))
(defn go []
(init)
(start))
(defn reset []
(stop)
(refresh :after 'user/go))
To run a system, we can type this in our repl:
(user)> (reset)
Which will reload our code, and restart the entire system. It will shutdown the exiting system that is running if its up.
We get other benefits:
End to end testing is easy, just edit the config or replace a component to point to in-process services (I've used it to point to an in-process kafka server for tests).
You can theoretically spawn your application multiple times for the same JVM (not really as practical as the first point).
You don't need to restart the REPL when you make code changes and have to restart your server
Unlike ring reload, we get a uniform way to restart our application regardless of its purpose: a background worker, microservice, or machine learning system can all be architected in the same way.
It's worth noting that, since everything is in-process, Component does not handle anything related to fail-over, distributed systems, or faulty code ;)
There are plenty of "resources" (aka stateful objects) that Component can help you manage within a server:
Connections to services (queues, dbs, etc.)
Passage of Time (scheduler, cron, etc.)
Logging (app logging, exception logging, metrics, etc.)
File IO (blob store, local file system, etc.)
Incoming client connections (web, sockets, etc.)
OS Resources (devices, thread pools, etc.)
Component can seem like overkill if you only have a web server + db. But few web apps are just that these days.
Side Note: Moving the-system into another namespace reduces the likelihood of refreshing the the-system var when developing (eg - calling refresh instead of reset).
I'm writing an application server in Clojure that will use ClojureScript on the client.
I'd like to find an efficient, idiomatic way to push data from the server to the client as realtime events, ideally using a some combination of:
http-kit
core.async
Ring
(But I'm open to other possibilities)
Can anyone provide a good example / approach to doing this?
I prefer to use aleph, here is the wiki, you can simply use wrap-ring-handler function to wrap existed handlers.
For the 'push' function, most useful part is aleph's async handler. It builds on top of netty, not a one connection one thread model, so the server side do not need worry about the tcp connection count.
Some implement details:
Server side use aysnc handler, hold all the client connections (channels)
In 60(for example) seconds, if there is no 'new data', send an empty response
if server side has a response ,send it.
The client side can simply send a normal http request to the server
when the client get the response, handle the response body, and then, re-send a http request again
please check the client and all the proxy servers to set the correct timeout value
There are more ways here : http://en.wikipedia.org/wiki/Push_technology
I've been trying out the library Chord recently, and I really like it.
It provides a small core.async wrapper around the Websocket support in http-kit.
From the github page:
On the Server
(:require [chord.http-kit :refer [with-channel]]
[clojure.core.async :refer [<! >! put! close! go]])
(defn your-handler [req]
(with-channel req ws-ch
(go
(let [{:keys [message]} (<! ws-ch)]
(println "Message received:" message)
(>! ws-ch "Hello client from server!")
(close! ws-ch)))))
On the Client
(:require [chord.client :refer [ws-ch]]
[cljs.core.async :refer [<! >! put! close!]])
(:require-macros [cljs.core.async.macros :refer [go]])
(go
(let [ws (<! (ws-ch "ws://localhost:3000/ws"))]
(>! ws "Hello server from client!")))
I think it's still in early stages though - it doesn't handle disconnections yet.
I'm developed one project now, where I had the exact same requirement. I used the pedestal service in combination with core.async to implement SSE and it's working really well.
Unfortunately, I can't open-source this work now, but basically, I did something like the snippets below, only more complicated because of authentication. (It's not particularly easy to do authentication in SSE from browser, because you can't pass any custom headers in your new EventSource(SOME_URI); call.
So the snippets:
(ns chat-service.service
(:require [clojure.set :as set]
[clojure.core.async :as async :refer [<!! >!! <! >!]]
[cheshire.core :as json]
[io.pedestal.service.http :as bootstrap]
[io.pedestal.service.log :as log]
[io.pedestal.service.http.route :as route]
[io.pedestal.service.http.sse :as sse]
[io.pedestal.service.http.route.definition :refer [defroutes]]))
(def ^{:private true :doc "Formatting opts"} json-opts {:date-format "MMM dd, yyyy HH:mm:ss Z"})
(def ^{:private true :doc "Users to notification channels"} subscribers->notifications (atom {}))
;; private helper functions
(def ^:private generate-id #(.toString (java.util.UUID/randomUUID)))
(defn- sse-msg [event msg-data]
{:event event :msg msg-data})
;; service functions
(defn- remove-subscriber
"Removes transport channel from atom subscribers->notifications and tears down
SSE connection."
[transport-channel context]
(let [subscriber (get (set/map-invert #subscribers->notifications) transport-channel)]
(log/info :msg (str "Removing SSE connection for subscriber with ID : " subscriber))
(swap! subscribers->notifications dissoc subscriber)
(sse/end-event-stream context)))
(defn send-event
"Sends updates via SSE connection, takes also transport channel to close it
in case of the exception."
[transport-channel context {:keys [event msg]}]
(try
(log/info :msg "calling event sending fn")
(sse/send-event context event (json/generate-string msg json-opts))
(catch java.io.IOException ioe
(async/close! transport-channel))))
(defn create-transport-channel
"Creates transport channel with receiving end pushing updates to SSE connection.
Associates this transport channel in atom subscribers->notifications under random
generated UUID."
[context]
(let [temporary-id (generate-id)
channel (async/chan)]
(swap! subscribers->notifications assoc temporary-id channel)
(async/go-loop []
(when-let [payload (<! channel)]
(send-event channel context payload)
(recur))
(remove-subscriber channel context))
(async/put! channel (sse-msg "eventsourceVerification"
{:handshakeToken temporary-id}))))
(defn subscribe
"Subscribes anonymous user to SSE connection. Transport channel with timeout set up
will be created for pushing any new data to this connection."
[context]
(create-transport-channel context))
(defroutes routes
[[["/notifications/chat"
{:get [::subscribe (sse/start-event-stream subscribe)]}]]])
(def service {:env :prod
::bootstrap/routes routes
::bootstrap/resource-path "/public"
::bootstrap/type :jetty
::bootstrap/port 8081})
One "problem" i encountered is the default way how pedestal handles dropped SSE connections.
Because of the scheduled heartbeat job, it logs exception whenever connection is dropped and you didn't call end-event-stream context.
I wish there was a way to disable/tweak this behavior, or at least provide my own tear-down function which will be called whenever heartbeat job fails with EofException.
I put together a simple socket server (see below). Currently, it is not capable of handling multiple/concurrent requests. How can I make the socket server more efficient -- i.e. capable of handling concurrent requests? Are there any clojure constructs I can leverage? Thus far, I've thought of using either java's NIO (instead of IO) or netty (as pointed out here).
(ns server.policy
(:import
(java.net ServerSocket SocketException)
java.io.PrintWriter))
(defn create-socket
"Creates a socket on given port."
[port]
(ServerSocket. port))
(defn get-writer
"Create a socket file writer."
[client]
(PrintWriter. (.getOutputStream client)))
(defn listen-and-respond
"Accepts connection and responds."
[server-socket service]
(let [client (.accept server-socket)
socket-writer (get-writer client)]
(service socket-writer)))
(defn policy-provider
"Returns domain policy content."
[socket-writer]
(.print socket-writer "<content>This is a test</content>")
(.flush socket-writer)
(.close socket-writer))
(defn run-server
[port]
(let [server-socket (create-socket port)]
(while (not (.isClosed server-socket))
(listen-and-respond server-socket policy-provider))))
I have had success using Netty directly. However, if you want something that feels a little more like idiomatic Clojure code, take a look at the aleph library. It uses Netty internally, but results in much more simple code:
(use 'lamina.core 'aleph.tcp)
(defn echo-handler [channel client-info]
(siphon channel channel))
(start-tcp-server echo-handler {:port 1234})
Also, keep in mind that sometimes you need to reference the lamina documentation in addition to the aleph documentation.