I'm trying to retrieve a URL. At the time of writing: https://upvote.pub/
Using the java.net.URL, I can retrieve this fine:
=> (count (slurp (.openStream (java.net.URL. "https://upvote.pub/"))))
44353
Ditto clj-http:
=> (count (:body (clj-http.client/get "https://upvote.pub/")))
44353
But when I use HTTPKit I run into problems:
=> #(http/get "https://upvote.pub/")
{:opts {:method :get, :url "https://upvote.pub/"}, :error #error {
:cause "No status"
:via
[{:type org.httpkit.ProtocolException
:message "No status"
:at [org.httpkit.client.RespListener onCompleted "RespListener.java" 126]}]
:trace
[[org.httpkit.client.RespListener onCompleted "RespListener.java" 126]
[org.httpkit.client.Request finish "Request.java" 51]
[org.httpkit.client.HttpClient doRead "HttpClient.java" 156]
[org.httpkit.client.HttpClient run "HttpClient.java" 426]
[java.lang.Thread run "Thread.java" 745]]}}
This site uses Let's Encrypt. However, other Let's Encrypt sites seem to work:
=> (-> "https://letsencrypt.org/" http/get deref :body count)
21490
The certificate has different properties, but it's from the same authority.
Furthermore, if I include :insecure? true, the error still happens.
I thought this might be related to this HTTPS + proxy bug, as the symptoms are similar and Google App Engine seems to be load-balancing the site. However, this fix was rolled into release 2.2.0.
Is this a bug with HTTP-Kit? Can I work round it?
openjdk version "1.8.0_92-internal"
Clojure version 1.8.0
http-kit version "2.2.0"
It looks like this was caused by SNI (Server Name Identification). I used HTTP Kit version “2.3.0-alpha5” and followed this blog post. It now works.
Related
Twilio docs for Java lib
MVN for this lib
I'm trying to use Twilio from Clojure. Pretty new to Clojure dev, so I'm trying to get to grips with importing libs, in general.
My project.clj looks like this:
(defproject magical-new-project "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
:url "https://www.eclipse.org/legal/epl-2.0/"}
:dependencies [[org.clojure/clojure "1.10.0"]
[com.twilio.sdk/twilio "7.47.5"]]
:repl-options {:init-ns magical-new-project.core})
I think I'm correctly importing the Twilio Java lib, but I'm not sure.
My core.clj looks like:
(ns magical-new-project.core
(:import [com.twilio Twilio]
;[com.twilio.http Request Response]
[com.twilio.rest.api.v2010.account Message]
[com.twilio.rest.api.v2010.account Call]
[com.twilio.type PhoneNumber]))
(defn foo
"I don't do a whole lot."
[x]
(println x "Peacefully disengage, World!"))
As far as I can tell, this should be correctly importing the Twilio class shown here.
So then I try (from REPL) to initialize the Twilio object that I hope I have successfully imported, but it fails.
$ lein repl
nREPL server started on port 62356 on host 127.0.0.1 - nrepl://127.0.0.1:62356
REPL-y 0.4.3, nREPL 0.6.0
Clojure 1.10.0
OpenJDK 64-Bit Server VM 12.0.1+12
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
magical-new-project.core=> (Twilio. "My API or Account SID goes here" "My AUTH_TOKEN or API secret goes here")
Syntax error (IllegalArgumentException) compiling new at (form-init14687807219308370487.clj:1:1).
No matching ctor found for class com.twilio.Twilio
As far as I can tell, the No matching ctor thing means there is no constructor function that accepts the arguments I'm presenting, but line 39 of the Twilio.java file seems to take two strings, and then if you look here, you'll see that I'm sending the right arguments (the ACCOUNT_SID and the AUTH_TOKEN).
At the moment, I'm not sure whether I'm (1) correctly importing the Twilio class, (2) adding the dependencies to the project correctly, (2) correctly using the REPL, or (4) correctly using the Twilio SDK. Maybe I'm doing all of these incorrectly.
I would really appreciate any help or advice that you could afford me.
the function you linked to is not a constructor but a static function on the class. You call a static function on a class in clojure like (Twilio/init "foo" "bar"). Your syntax is correct if it were actually a constructor.
Thanks #juraj. I wasn't sure if this actually qualified as an answer :-)
I am using swagger to provide an API for a db access program. During development I normally have 2 versions running, the dev version and the prod version that I automatically start on login. I want to have a different title visible on the swagger front page so I don't accidentally trash my live database. So far I have been hand editing the title field in the swagger setup, but this is error prone, I often forget to change it before I run lein uberjar to build the prod version.
The env setup seems like an ideal way to do this. The luminus lein template already uses an env map that is built from dev and prod config files which works fine, allowing me to automatically specify different ports for the 2 builds. I added an entry to these that gives me a title that is different in prod and dev versions. I can see it from the repl, but including it in the swagger specification just gives null.
This is the :swagger definition from the start of my photo-api.routes.services.clj file:
(ns photo-api.routes.services
(:require [cheshire.core :as json]
[compojure.api.sweet :refer :all]
[image-lib.images :as ilim]
[image-lib.preferences :as ilpf]
[image-lib.projects :as ilpr]
[image-lib.write :as ilwr]
[photo-api.db.core :as db]
[photo-api.config :refer [env]]
[photo-api.routes.helpers.build :as build]
[photo-api.routes.helpers.keywords :as keywords]
[photo-api.routes.helpers.open :as open]
[photo-api.routes.helpers.photos :as photos]
[photo-api.routes.helpers.projects :as projects]
[ring.util.codec :refer [url-decode]]
[ring.util.http-response :refer [ok]]
[schema.core :as s]
[clojure.string :as str]))
(defapi service-routes
{:swagger {:ui "/swagger-ui"
:spec "/swagger.json"
:data
{:info
{:version "1.0.1"
;; Switch to correct title before lein uberjar
;; TODO Automate this so swagger page always shows dev or prod version
;;:title "Photo API"
:title (:title env)
:description "Access a mongo database containing details of photos"}}}}
The commented out :title specification works fine, but the (:title env) call doesn't although it is the exact same call I can successfully use from the repl. I believe the env map is built as part of photo-api.config and from the startup messages when I start the server it looks like this is being successfully started before the http-server:
{:started
["#'photo-api.config/env"
"#'photo-api.db.core/db*"
"#'photo-api.db.core/db"
"#'photo-api.handler/init-app"
"#'photo-api.handler/app"
"#'photo-api.core/http-server"]}
user>
This is photo-api.config, unchanged from the luminus default:
(ns photo-api.config
(:require [cprop.core :refer [load-config]]
[cprop.source :as source]
[mount.core :refer [args defstate]]))
(defstate env :start (load-config
:merge
[(args)
(source/from-system-props)
(source/from-env)]))
and the dev config.edn file:
{:title "**** Photos Development API ****"
:dev true
:port 31999
;; when :nrepl-port is set the application starts the nREPL server on load
:nrepl-port 57251}
Am I missing something obvious here? Is there another step necessary to make the env map visible to the swagger setup?
EDIT:
Changing the call from (:title env) to (env :title) causes cider-jack-in to fail with a long error message/stack trace that includes:
Caused by: java.lang.ClassCastException: mount.core.DerefableState cannot be cast to clojure.lang.IFn
Changing it again to (#env :title) then gives a similarly long error message/stack trace that contains:
Caused by: java.lang.ClassCastException: mount.core.NotStartedState cannot be cast to clojure.lang.IFn, compiling:(services.clj:29:23)
So it looks like env is not being started till after the call to it from the swagger setup. I still have no idea why as the when cider-jack-in did work it clearly showed the config.env state starting before the http-server. (see above)
It looks like (defstate env) is an atom that needs to be derefed. Mount's README points to the tests to for some examples .
You might try (:title #env) in service-routes
(defapi service-routes
{:swagger {:ui "/swagger-ui"
:spec "/swagger.json"
:data
{:info
{:version "1.0.1"
;; Switch to correct title before lein uberjar
;; TODO Automate this so swagger page always shows dev or prod version
;;:title "Photo API"
:title (:title #env) ;;;--------> UPDATED
:description "Access a mongo database containing details of photos"}}}}
EDIT --
Not an atom. See this issue for the same deref error, which, presumably means that no deref is needed.
I would like to write tests for a Pedestal web-service.
If I have :
(defn pong
[request]
(ring-resp/response "pong"))
(defroutes routes[[["/" {:get pong}]]])
How would I write a test for that ?
(deftest alive-system
(testing "ping-pong route"
;; How do I test my route ?
;; If possible :
;; - I would like to have direct access to it
;; ie. no need to bind pedestal to a port would be nice
;; - The ability to omit some interceptors would be nice also,
;; as it would allow me to receive plain Clojure data structures
;; instead of, for instance, JSON which I would have to parse.
...)
Edit:
Here is what I tried :
(deftest alive-system
(testing "ping-pong route"
(let [response (my-other.ns/routes (mock/request :get "/ping"))]
(is (= (:status response) 200))
(is (= (:body response) "pong")))))
But I get an exception :
ERROR in (alive-system) (service_test.clj:13)
Uncaught exception, not in assertion.
expected: nil
actual: java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to clojure.lang.IFn
So after asking on the issue I linked ohpaulez replied :
#nha - Thanks for using Pedestal! Sorry your question didn't get an
answer on StackOverflow - I don't think anyone monitors SO for
Pedestal questions. The best place to ask those kinds of questions is
on the mailing list.
Pedestal ships with its own utility for making requests directly to
the servlet (similar to ring/mock, although I've never used mock
myself) called response-for. The Pedestal Service template produces a
test automatically for you. Check out one of the samples for an
example.
Also note that said response-for doesn't yet support asynchronous responses (so my routes that uses asynchronous interceptors with core.async failed - I had to make them synchronous).
I'm a Ruby dev moving over to Clojure and I'm having trouble understanding how to translate the following Java call into Clojure based on the conventions used in the Clojure library Amazonica.
AmazonS3 client = new AmazonS3Client(credentials);
client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));
The code I have at the moment is:
(ns spurious-aws-sdk-helper.core
(:use [amazonica.aws.s3]])
(:require [amazonica.core :refer [ex->map]]))
(def credentials {:access-key "development_access"
:secret-key "development_secret"
:endpoint "s3.spurious.localhost:49154"
:client-config {:protocol "http"}})
(try
(amazonica.aws.s3/set-s3client-options {:path-style-access true})
(create-bucket credentials "testing")
(catch Exception e
(clojure.pprint/write (ex->map e))))
But I'm getting the following error:
com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Unable to execute HTTP request: testing.s3.spurious.localhost
java.net.UnknownHostException: testing.s3.spurious.localhost
This doesn't look to be correct as it's prefixing the bucket name (testing) onto the hostname. Where as I need the SDK to talk to our local (fake) S3 service (s3.spurious.localhost:49154) using path style.
e.g. like http://s3.spurious.localhost:49154/testing
I think it's because I'm not translating the Java code correctly...
(amazonica.aws.s3/set-s3client-options {:path-style-access true})
...this is passing a map to set-s3client-options rather than what it should be, which is passing the result of calling withPathStyleAccess(true) on a new instance of S3ClientOptions. But I don't know how to do that here?
Any help would be greatly appreciated.
UPDATE
Here is the latest version of the code (which still doesn't work)...
(ns spurious-aws-sdk-helper.core
(:use [amazonica.aws.s3])
(:require [amazonica.core :refer [ex->map]]))
(def credentials {:access-key "development_access"
:secret-key "development_secret"
:endpoint "s3.spurious.localhost:49154"
:client-config {:protocol "http"}})
(try
(amazonica.aws.s3/set-s3client-options
(. (com.amazonaws.services.s3.S3ClientOptions.) setPathStyleAccess true))
(create-bucket credentials "testing")
(catch Exception e
(clojure.pprint/write (ex->map e))))
Your first call to set up the S3ClientOptions is the correct one.
I think you're confusing what that setting does.
It changes the pre-signed URL generation from using the Virtual Hosted Style https://my_bucket_name.s3.amazonaws.com/my_key to the Path Style https://s3.amazonaws.com/my_bucket_name/my_key
This is necessary if your bucket name contains dots, as the virtual hosted style breaks Amazon's SSL certificate identification (i.e. it only supports the wildcard *.s3.amazonaws.com)
So if you're generating pre-signed URLs on a bucket name containing dots in Clojure e.g.
(s3/generate-presigned-url bucket key (-> 6 hours from-now)))
You will first need to set the path style using
(amazonica.aws.s3/set-s3client-options {:path-style-access true})
Thanks to #stukennedy for responding to the open question. I should have come back long ago and updated this space with the actual solution which I managed to figure out and implement ~8months ago (Feb 2nd 2015):
(ns spurious-aws-sdk-helper.s3
(:use [amazonica.aws.s3])
(:require [spurious-aws-sdk-helper.utils :refer [endpoint cred]]))
(defn resource [type]
(endpoint type :spurious-s3))
(defn setup
([type]
(set-s3client-options (cred (resource type)) :path-style-access true))
([type name]
(try
(let [credentials (cred (resource type))]
(set-s3client-options credentials :path-style-access true)
(create-bucket credentials name))
(catch Exception e
(prn "S3 Error: chances are you're creating a bucket that already exists")))))
You can find the full details here: https://github.com/Integralist/spurious-clojure-aws-sdk-helper if you need more context
If I remember correctly, I needed to pass credentials through to set-s3client-options (without them it wouldn't work)
I've written a small Swing App before in Clojure and now I'd like to create an Ajax-style Web-App. Compojure looks like the best choice right now, so that's what I'm going to try out.
I'd like to have a real tiny edit/try feedback-loop, so I'd prefer not to restart the web server after each small change I do.
What's the best way to accomplish this? By default my Compojure setup (the standard stuff with ant deps/ant with Jetty) doesn't seem to reload any changes I do. I'll have to restart with run-server to see the changes. Because of the Java-heritage and the way the system is started etc. This is probably perfectly normal and the way it should be when I start the system from command-line.
Still, there must be a way to reload stuff dynamically while the server is running. Should I use Compojure from REPL to accomplish my goal? If I should, how do I reload my stuff there?
This is quite an old question, and there have been some recent changes that make this much easier.
There are two main things that you want:
Control should return to the REPL so you can keep interacting with your server. This is accomplished by adding {:join? false} to options when starting the Jetty server.
You'd like to automatically pick up changes in certain namespaces when the files change. This can be done with Ring's "wrap-reload" middleware.
A toy application would look like this:
(ns demo.core
(:use webui.nav
[clojure.java.io]
[compojure core response]
[ring.adapter.jetty :only [run-jetty]]
[ring.util.response]
[ring.middleware file file-info stacktrace reload])
(:require [compojure.route :as route] view)
(:gen-class))
; Some stuff using Fleet omitted.
(defroutes main-routes
(GET "/" [] (view/layout {:body (index-page)})
(route/not-found (file "public/404.html"))
)
(defn app
[]
(-> main-routes
(wrap-reload '(demo.core view))
(wrap-file "public")
(wrap-file-info)
(wrap-stacktrace)))
(defn start-server
[]
(run-jetty (app) {:port 8080 :join? false}))
(defn -main [& args]
(start-server))
The wrap-reload function decorates your app routes with a function that detects changes in the listed namespaces. When processing a request, if those namespaces have changed on disk, they are reloaded before further request processing. (My "view" namespace is dynamically created by Fleet, so this auto-reloads my templates whenever they change, too.)
I added a few other pieces of middleware that I've found consistently useful. wrap-file handles static assets. wrap-file-info sets the MIME type on those static assets. wrap-stacktrace helps in debugging.
From the REPL, you could start this app by using the namespace and calling start-server directly. The :gen-class keyword and -main function mean that the app can also be packaged as an uberjar for startup from outside the REPL, too. (There's a world outside the REPL? Well, some people have asked for it anyway...)
Here's an answer I got from James Reeves in the Compojure Google Group (the answer's here with his permission):
You can reload a namespace in Clojure using the :reload key on the use
or require commands. For example, let's say you have a file "demo.clj" that contains your routes:
(ns demo
(:use compojure))
(defroutes demo-routes
(GET "/"
"Hello World")
(ANY "*"
[404 "Page not found"]))
At the REPL, you can use this file and start a server:
user=> (use 'demo)
nil
user=> (use 'compojure)
nil
user=> (run-server {:port 8080} "/*" (servlet demo-routes))
...
You could also put the run-server command in another clojure file.
However, you don't want to put it in the same file as the stuff you want to reload.
Now make some changes to demo.clj. At the REPL type:
user=> (use 'demo :reload)
nil
And your changes should now show up on http://localhost:8080
I wanted to add an answer, since things have changed a bit since the newest answer and I had spent a bit of time looking for this myself.
Install leiningen (just follow the instructions there)
Create project
lein new compojure compojure-test
Edit the ring section of project.clj
:ring {:handler compojure-test.handler/app
:auto-reload? true
:auto-refresh? true}
Start the server on whatever port you want
lein ring server-headless 8080
Check that the server is running in your browser, the default base route should just say "Hello world". Next, go modify your handler (it's in src/project_name). Change the hello world text, save the file and reload the page in your browser. It should reflect the new text.
Following up on Timothy's link to Jim Downing's setup, I recently posted on a critical addition to that baseline that I found was necessary to enable automatic redeployment of compojure apps during development.
I have a shell script that looks like this:
#!/bin/sh
CLASSPATH=/home/me/install/compojure/compojure.jar
CLASSPATH=$CLASSPATH:/home/me/clojure/clojure.jar
CLASSPATH=$CLASSPATH:/home/me/clojure-contrib/clojure-contrib.jar
CLASSPATH=$CLASSPATH:/home/me/elisp/clojure/swank-clojure
for f in /home/me/install/compojure/deps/*.jar; do
CLASSPATH=$CLASSPATH:$f
done
java -server -cp $CLASSPATH clojure.lang.Repl /home/me/code/web/web.clj
web.clj looks like this
(use '[swank.swank])
(swank.swank/ignore-protocol-version "2009-03-09")
(start-server ".slime-socket" :port 4005 :encoding "utf-8")
Whenever I want to update the server I create an ssh tunnel from my local machine to the remote machine.
Enclojure and Emacs (running SLIME+swank-clojure) can connect to the remote REPL.
This is highly configuration dependent but works for me and I think you can adapt it:
Put compojure.jar and the jars under the compojure/deps directory are in your classpath. I use clojure-contrib/launchers/bash/clj-env-dir to do this, all you need to do is set the directory in CLOJURE_EXT and it will find the jars.
CLOJURE_EXT Colon-delimited list of paths to directories whose top-level
contents are (either directly or as symbolic links) jar
files and/or directories whose paths will be in Clojure's
classpath.
Launch clojure REPL
Paste in hello.clj example from compojure root directory
Check localhost:8080
Re-define the greeter
(defroutes greeter
(GET "/"
(html [:h1 "Goodbye World"])))
Check localhost:8080
There are also methods for attaching a REPL to an existing process, or you could keep a socket REPL embedded in your server or you could even define a POST call that will eval on the fly to allow you to redefine functions from the browser itself! There are lots of ways to approach this.
I'd like to follow up on mtnygard's answer and post the full project.clj file and core.clj file that got the given functionality working. A few modifications were made, and it's more barebones
pre-setup commands
lein new app test-web
cd test-web
mkdir resources
project.clj
(defproject test-web "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.5.1"]
[compojure "1.1.6"]
[ring "1.2.1"]]
:main ^:skip-aot test-web.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})
core.clj
(ns test-web.core
(:use
[clojure.java.io]
[compojure core response]
[ring.adapter.jetty :only [run-jetty]]
[ring.util.response]
[ring.middleware file file-info stacktrace reload])
(:require [compojure.route :as route])
(:gen-class))
(defroutes main-routes
(GET "/" [] "Hello World!!")
(GET "/hello" [] (hello))
(route/not-found "NOT FOUND"))
(def app
(-> main-routes
(wrap-reload '(test-web.core))
(wrap-file "resources")
(wrap-file-info)
(wrap-stacktrace)))
(defn hello []
(str "Hello World!"))
(defn start-server
[]
(run-jetty #'app {:port 8081 :join? false}))
(defn -main [& args]
(start-server))
Pay Attention to the change from (defn app ...) to (def app ...)
This was crucial to getting the jetty server to work correctly
Compojure uses ring internally (by the same author), the ring web server options allow automatic realoading. So two alternatives would be :
lein ring server
lein ring server-headless
lein ring server 4000
lein ring server-headless 4000
Note that :
You need to have a line in your project.clj file that looks like:
:ring {:handler your.app/handler}