I am working on an application that has to download some external resources and make them accessible through a public/static directory in Ring.
But.. I have a problem saving the resources into a static directory in my application, when developing I use the ring-jetty-adapter; the test & production servers are running Tomcat.
I added :web-content "public" to my leiningen project and added the public directory in the root of the project, then I have a download function using http-agent and duck-streams:
(defn download
[file-name url]
(h/http-agent url
:handler (fn [agnt]
(let [fname file-name]
(with-open [w (d/writer fname)]
(d/copy (h/stream agnt) w))))))
If I am booting Jetty from the REPL and use savepath: "public/my.file", the downloaded file is placed correctly in the public directory.
But when I deploy it using a .war file to Tomcat, it looks for a public directory in the Tomcat root directory, and not under the application context path.
I tried to add a middleware wrapper to determine the context path and from there build the correct save path, but I cannot find any way to access the HttpServlet or a way to determine if the application is running in the adapter or if it is deployed under a specific context.
Here the wrapper:
(defn wrap-context-info [handler]
(fn [req]
(let [resp (handler req)]
(assoc resp :servlet (:servlet req) :req (:servlet-request req)))))
both :servlet and :req are nil.
Looking at the ring-servlet source, it appears that the ring servlet adapter associates the HttpServlet, HttpServletRequest, and HttpServletResponse objects with the ring request map under :servlet, :servlet-request, and :servlet-response keys, accordingly.
It also adds a :servlet-context entry to the request map with the value of (.getServletContext servlet) for convenience.
Inside your handler, you might want to check for the presence of these keys in the request map, and then pull further information that you need from the associated objects.
Related
I deployed my Clojure app to AWS and I also have Datomic Transactor(EC2) with DynamoDB when my users started to use my app something happened after a while and Datomic Transactor rebooted(for some reason), my app became unresponsive so I re-started my application server(to re-establish transactor connection) then there were no users data, no data at all!?
I was using same URI("datomic:ddb://us-east-1/my-table/my-db"), also I don't call delete-db or some function like that.
I only call d/create-database and d/connect for connecting to Datomic.
It happened me on development but I did not care that time because I thought I did something wrong or something like that.
How can I restore that old data?
P.S: Also I can see that DynamoDB table has some storage size like 900KB I assume some how my data in there? There were just couple of users tho.
My Datomic Version: "0.9.5561"
Here is my code:
(defn establish-conn
[]
(try
;conf/get returns DB URI like: "datomic:ddb://us-east-1/my-table/my-db"
(d/create-database (conf/get :db-uri))
(reset! conn (d/connect (conf/get :db-uri)))
(catch Throwable t
(log/error "Could not establish db conn." t))))
Also I have this code for recovering connection from Transactor failure/unavailable etc.
(defn fix-if-conn-ex
;;getting exception's error message
[err-msg]
(when (any? (str/includes? err-msg %) [":db.error/connection-released"
":db.error/transactor-unavailable"])
(db/establish-conn)))
I could not reproduce the issue with the latest Datomic version "0.9.5561.56" on my staging environment, I assume it's resolved with that one.
While running my tests with lein doo phantom, I receive a -1 status response and an empty string as the body. However, when I run the test in the repl, I am able to retrieve the request data with a 200 status response and the appropriate data in the body. Is this because a manytomany channel is being returned first as mentioned below, thus giving me the inappropriate response? If so, how could I account for this?
https://github.com/r0man/cljs-http#async-response-handling
I also thought maybe I need to use a timeout to wait for the request to complete. If so, how would I apply that appropriately with my existing code? It looks like cljs-http has :timeout as a parameter but I haven't been ably to get it to work appropriately (assuming this is the cause of the issue).
(deftest test-async
(async done
(go (let [response (<! (http/get "http://localhost:3000/api/user/1"
{:with-credentials? false
:query-params {"id" 1}}))]
(is (= {:status 200}
(select-keys response [:status]))))
(done))))
Since you are running your test under phantomjs. Phantomjs default disable cross domain XHR access and your tests js are running on localhost,all external ajax calls are denied.
you can set the --web-security=false to allow your test to do cross domain ajax.
In your project.clj add this
:doo {:paths {:phantom "phantomjs --web-security=false"}}
more info about phantomjs
http://phantomjs.org/api/command-line.html
We have a Clojure web application that is used by multiple projects (>20) that have multiple users logging in simultaneously. All projects have their own MySQL database. We have tried to figure out a way to use one application instance to serve requests from users that are delivered from their project's database.
The following script shows the principles of our multiple connections and should be executable in REPL (with correct database setup).
(ns testmultiple.core
(:require
[clojure.java.jdbc :as jdbc]
[compojure.core :refer [defroutes GET ANY routes context]]
[conman.core :as conman]
[mount.core :refer [defstate]]))
(def database-urls {:DB1 "jdbc:mysql://localhost:3306/DB1?user=DB1_user&password=DB1_password"
:DB2 "jdbc:mysql://localhost:3306/DB2?user=DB2_user&password=DB2_password"})
;; Connects to all databases in pool-specs
(defn connect!
[pool-specs]
(reduce merge (map (fn [pool-spec]
{(keyword (key pool-spec)) (conman/connect! {:jdbc-url (val pool-spec)})}) pool-specs)))
;; Disconnect from all databases in db-connections
(defn disconnect!
[db-connections]
(map (fn [db] (conman/disconnect! (val db))) db-connections))
;; Establish connections to all databases
;; and store connections in *dbs*
(defstate ^:dynamic *dbs*
:start (connect!
database-urls)
:stop (disconnect! *dbs*))
;; Bind queries to *db* dynamic variable which is bound
;; to each clients database before executing queries
;; The queries file defines the query get-user which
;; returns user by user id
(def ^:dynamic *db* nil)
(conman/bind-connection *db* "sql/queries.sql")
(mount.core/start)
; Define function that executes in current *db* binding
(defn getuser [id] (get-user {:id id}))
; Works, the user with Id 670 is returned from DB1
(with-bindings {#'*db* (:DB1 *dbs*)} (getuser 670))
; Works, the user with Id 670 is returned from DB2
(with-bindings {#'*db* (:DB2 *dbs*)} (getuser 670))
More specifically, the project is inferred from the URL request in the router. The following code shows the principle for the router. Accessing www.example.com/DB1/page1 and www.example.com/DB2/page2 will show page1 with data from DB1 and page2 with data from DB2, respectively.
(defn serve-page1 [] (str "page1" (getuser 670)))
(defn serve-page2 [] (str "page2" (getuser 670)))
(def home-routes
(context "/:project" [project]
(if (contains? *dbs* (keyword project))
(routes
(GET "/page1" []
(with-bindings {#'*db* ((keyword project) *dbs*)}
(serve-page1)))
(GET "/page2" []
(with-bindings {#'*db* ((keyword project) *dbs*)}
(serve-page2))))
(ANY "*" [] (str "Project not found")))))
This will be an application with considerable traffic. Notably, we are still in development phase and have thus not been able to test this solution with more than a couple of databases running on localhost. Our questions are
Is establishing multiple connections like this reasonable, stable and scalable?
Are there other better methods for the routing and dynamic binding of the project's database?
Is establishing multiple connections like this reasonable, stable and scalable?
Yes, this is a very reasonable approach. Very few database systems are limited by the number of outgoing connections. Both JDBC and Korma will handle this just fine in clojure. You do need to be aware of which requests are dependent on which DB when building the monitoring and ops related components of course. So you can tell which DB is causing problems.
Are there other better methods for the routing and dynamic binding of the project's database?
My only suggestion would be to explicitly pass the DB to each function rather than using a binding, though this is a personal style opinion and your approach will clearly work.
I am trying to use immutant to manage transactions across HornetQ and mysql. As I understand the docs, to do this I must use XA transactions, because I am running a standalone app and not inside an app server.
However when I try and set :xa? for the context of my application, I get exceptions when tying to setup a listener.
(ns example
(:require [immutant.messaging :as msg]))
(def capture (atom nil))
(let [ctx (msg/context :host "localhost" :xa? true)
queue (msg/queue "example" :context ctx)]
(reset! capture nil)
(msg/listen queue (fn [m] (reset! capture m)))
(msg/publish queue {:my :msg}))
This throws "java.lang.IllegalStateException: You can't create a child context from an XA context." from the (msg/listen) invocation. What am I doing wrong?
I think you've discovered a bug, but in your case, I think there's a workaround: you only need that :xa? true option if your queue is remote. You can still create an XA transaction binding your HornetQ actions to MySQL in your listener handler using the immutant.transactions/transaction macro. See the docs for an example.
Currently I have an instance of ActiveMQ running, that I am attempting to connect to using immutant. Currently the code for this connection looks like so;
(defn make-ctx
[]
(log/debug "making context")
(let [ctx (m/context :host (:host immutant-host) :port (:port immutant-host))]
(log/debug "context created")
ctx))
(defn make-listener
[ctx]
(let [listener (m/listen topic #(log/debug %) :context ctx)]
(log/debug "listener created")
listener))
(defn immutant-test
[]
(log/debug "testing immutant messaging with ActiveMQ")
(let [ctx (make-ctx)
listener (make-listener ctx)]
(Thread/sleep 15000)
(.close listener)))
Though my code does not make it passed the make-ctx function. When it attempts to create the context I get the error
Exception in thread "main" java.lang.RuntimeException: javax.jms.JMSException: Failed to create session factory
at org.projectodd.wunderboss.messaging.jms.DestinationUtil.mightThrow(DestinationUtil.java:47)
at org.projectodd.wunderboss.messaging.jms.JMSMessagingSkeleton.createContext(JMSMessagingSkeleton.java:64)
at org.projectodd.wunderboss.messaging.jms.JMSMessagingSkeleton.createContext(JMSMessagingSkeleton.java:181)
at immutant.messaging$context.doInvoke(messaging.clj:84)
at clojure.lang.RestFn.invoke(RestFn.java:457)
at jms_test.core$make_ctx.invoke(core.clj:24)
at jms_test.core$immutant_test.invoke(core.clj:37)
at jms_test.core$_main.invoke(core.clj:158)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at jms_test.core.main(Unknown Source)
Caused by: javax.jms.JMSException: Failed to create session factory
at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:673)
at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:112)
at org.hornetq.jms.client.HornetQConnectionFactory.createConnection(HornetQConnectionFactory.java:107)
at org.projectodd.wunderboss.messaging.jms.JMSMessagingSkeleton$1.call(JMSMessagingSkeleton.java:73)
at org.projectodd.wunderboss.messaging.jms.DestinationUtil.mightThrow(DestinationUtil.java:45)
... 10 more
Caused by: HornetQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=HQ119013: Timed out waiting to receive cluster topology. Group:null]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:946)
at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:669)
... 14 more
The immutant-host is defined as
(def immutant-host {:host "127.0.0.1" :port 61616})
I've been able to connect to my broker with the clamq libray ,and am able to send and receive messages with that. Though because the rest of the application is built with immutant messaging I'd like to stick with that library if possible to keep from having to support several messaging libraries.
Immutant is built on top of HornetQ, so can only connect to HornetQ servers by default. This is because the JMS spec doesn't provide a wire protocol, so each implementation has its own. However, if the remote ActiveMQ is actually Artemis, you can use wunderboss-artemis to enable using it from Immutant (note that the article states you have to use an incremental build of Immutant - that is no longer true, you can use Immutant 2.1.0).
If it's not Artemis, it wouldn't be too difficult to implement a wunderboss-activemq adapter using the artemis version as a guide.