Datomic Ions: Connecting to on-prem database - clojure

I am trying to connect my Datomic Ions app to an on-prem Datomic DB running on DynamoDB. When I run (datomic.api/connect "datomic:ddb://us-west-2/***/***") I am getting the following error: Could not initialize class datomic.Peer
deps.edn is as follows
:deps {com.datomic/ion {:mvn/version "0.9.28"}
org.clojure/data.json {:mvn/version "0.2.6"}
org.clojure/clojure {:mvn/version "1.9.0"}
com.datomic/client-cloud {:mvn/version "0.8.71"}
com.datomic/datomic-pro {:mvn/version "0.9.5697"}
com.amazonaws/aws-java-sdk-dynamodb {:mvn/version "1.11.33"}
com.datomic/client-impl-pro {:mvn/version "0.8.7"}}
:mvn/repos {"datomic-cloud" {:url "s3://datomic-releases-1fc2183a/maven/releases"}}
:aliases
{:dev {:extra-deps {com.datomic/client-cloud {:mvn/version "0.8.71"}
com.datomic/ion-dev {:mvn/version "0.9.186"}}}}}

Related

Adding unit-tests to Clojure CLI tools and deps.edn

I'm used to using Clojure with Leiningen.
But I have a new project that I booted into existence with LightMod. And it uses CLI tools and deps.
This works but I now want to add some unit testing to it.
So here's (a slightly simplified version of) my deps.edn file :
{:paths ["src" "resources"],
:aliases
{
:dev
{:extra-deps
{orchestra #:mvn{:version "2018.12.06-2"},
expound #:mvn{:version "0.7.2"},
nightlight #:mvn{:version "RELEASE"},
com.bhauman/figwheel-main #:mvn{:version "0.2.0"}},
:main-opts ["dev.clj"]},
:test
{:extra-paths ["test"]
:extra-deps {com.cognitect/test-runner {:git/url "https://github.com/cognitect-labs/test-runner.git" :sha "209b64504cb3bd3b99ecfec7937b358a879f55c1"}}
:main-opts ["-m" "cognitect.test-runner"]}
:prod
{:extra-deps
{leiningen #:mvn{:version "2.9.0"},
org.clojure/clojurescript #:mvn{:version "1.10.439"}},
:main-opts ["prod.clj"]},
:app
{:extra-deps
{
markdown-clj {:mvn/version "1.10.1"}
instaparse {
:mvn/version"1.4.10"
}
... MORE ...
}}}}
Now this file has worked OK with both :dev and :prod compilation.
But now I've just added the :test alias.
And put my tests into test/myns/core_tests.clj
However, when I try to run tests, I get this
Could not locate markdown/core__init.class, markdown/core.clj or markdown/core.cljc on classpath
So my interpretation. While the :dev and the :prod branches are successfully pulling in all the dependencies in the :app branch. The :test branch is not pulling in markdown. (And probably none of the other dependencies declared in :app
So why should this be? And how can I explicitly tell my :test clause that it needs to use all the same dependencies that are used in :dev and :prod?
You can use the Kaocha test runner for this. A working template project can be found here. It even includes the ability to compile Java source files as well as Clojure.
To get started, set up deps.edn like so
{:paths ["src/clj"]
:deps {
cambium/cambium.codec-simple {:mvn/version "0.9.3"}
cambium/cambium.core {:mvn/version "0.9.3"}
cambium/cambium.logback.core {:mvn/version "0.4.3"}
org.clojure/clojure {:mvn/version "1.10.1"}
prismatic/schema {:mvn/version "1.1.12"}
tupelo {:mvn/version "0.9.200"}
}
:aliases {:test {
:extra-deps {lambdaisland/kaocha {:mvn/version "1.0-612"}}
}}
}
and add another file tests.edn to configure Kaocha:
; ***** Be sure to use tagged reader literal '#kaocha/v1' *****
#kaocha/v1 {:tests
[{:id :unit
:test-paths ["test/clj"]
:ns-patterns ["^tst..*"]
}]
; :reporter kaocha.report.progress/report
; :plugins [:kaocha.plugin/profiling :kaocha.plugin/notifier]
}
then create a short shell script to launch Koacha in the subdir ./bin. Name it bin/kaocha with contents:
#!/bin/bash
clojure -A:test -m kaocha.runner "$#"
Now you are ready to go! A test file:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test) )
(dotest
(is= 5 (+ 2 3))
(isnt= 9 (+ 2 3))
(throws? (/ 1 0)) ; verify that an illegal operation throws an exception
(is= 3 (add2 1 2))
(throws? (add2 1 "two"))) ; Prismatic Schema will throw since "two" is not a number
; NOTE: Clojure Deps/CLI can't handle Java source code at present
and we are off to the races:
~/io.tupelo/clj-template > bin/kaocha
[(.....)]
1 tests, 5 assertions, 0 failures.

Unable to build standalone jar using boot

EDIT: Added more information at the end. Also, got war to run with jetty-runner.jar (not with tomcat however)
I am using boot as the build tool for my clojure application.
This is the first application I have written with cljs, cljc and clj code (along with sass) all in one project.
My build.boot has the following task-options
(def db-opts (:db (clojure.edn/read-string (slurp "resources/config.edn"))))
(task-options!
pom {:project 'gcr
:version "0.1"}
cljs {:optimizations :simple}
uber {:as-jars true}
war {:file "gcr.war"}
ragtime {:database (str "jdbc:"
(:dbtype db-opts) "://"
(:user db-opts) ":"
(:password db-opts) "#"
(:host db-opts) ":"
(:port db-opts) "/"
(:dbname db-opts))}
test {:include #"gcr.test"
:junit-output-to "junit-out"})
my build task is defined as follows:
(deftask build
"Builds an uberjar of this project that can be run with java -jar"
[]
(comp
(aot :all true ;; :namespace #{'gcr.server}
)
(pom)
(cljs :optimizations :simple)
(sass)
(uber)
(jar :file "gcr.jar" :main 'gcr.server)
(sift :include #{#"gcr.jar"})
(target)))
When i run boot build the project builds fine without any errors.
But when I run java -jar target/gcr.jar
I get the following:
$ java -jar target/gcr.jar
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=lcd -Dswing.aatext=true
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/lang/Var
at gcr.server.<clinit>(Unknown Source)
Caused by: java.lang.ClassNotFoundException: clojure.lang.Var
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
I have been looking at this for about 24 hours now and still unable to figure out how to get this working. (I finally was able to build a war that works with jetty-runner but not on tomcat)
----------
Some more info:
My boot.properties is
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_CLOJURE_VERSION=1.10.0
BOOT_VERSION=2.8.2
and I am using openjdk version "1.8.0_202"
I checked if the jar file built has the clojure jar using jar tf target/gcr.jar and the following line is listed:
7c506484bc48541ffa5dcb4d9366a8a2-clojure-1.10.0.jar
My IFDE works fine, I have the following dev task defined as follows:
(deftask dev
"Launch Immediate Feedback Development Environment"
[]
(comp
(serve :handler 'gcr.core/app ;; ring handler
:resource-root "public" ;; root classpath
:reload true) ;; reload ns
(watch)
(reload)
(cljs-repl) ;; before cljs task
(cljs)
(sass)
(target :dir #{"public"})))
set-env! part is as follows:
(set-env!
:source-paths #{"src/scss" "src/clj" "src/cljs" "src/cljc"}
:resource-paths #{"html" "resources"}
:dependencies '[[org.clojure/clojure "1.10.0"]
[org.clojure/clojurescript "1.10.520"]
;; server
[compojure "1.6.1"]
[ring/ring-core "1.7.1"]
[ring/ring-jetty-adapter "1.7.1"]
[ring/ring-json "0.4.0"]
[ring-middleware-format "0.7.4"]
;; [javax.servlet/servlet-api "2.5" :scope "test"]
;; [yogthos/config "1.1.1"]
;; java 10 upgrade
[javax.xml.bind/jaxb-api "2.4.0-b180830.0359"]
;; common
[org.clojure/core.async "0.4.490"]
[org.clojars.akiel/async-error "0.3"]
[hickory "0.7.1"]
[instaparse "1.4.10"]
;; clj
[clj-time "0.15.1"]
[clj-http "3.9.1"]
[clout "2.2.1"]
[org.clojure/data.json "0.2.6"]
[org.clojure/data.codec "0.1.1"] ;; Base64
;; cljs
[com.andrewmcveigh/cljs-time "0.5.2"]
[prismatic/dommy "1.1.0"]
[hipo "0.5.2"]
[hodgepodge "0.1.3"]
[cljs-http "0.1.46"]
[com.cemerick/url "0.1.1"]
[reagent "0.8.1"]
;; db
[ragtime "0.8.0"] ;; Migrations
[hikari-cp "2.7.1"] ;; Connection Pooling
[org.clojure/java.jdbc "0.7.9"]
[honeysql "0.9.4"]
[org.postgresql/postgresql "42.2.5"]
;; Authentication framework with JWT
[buddy/buddy-auth "2.1.0"]
[buddy/buddy-core "1.5.0"]
[buddy/buddy-sign "3.0.0"]
;; argon2 for password hashing
[de.mkammerer/argon2-jvm "2.5"]
;; logging
[com.taoensso/timbre "4.10.0"] ;; logging
[com.fzakaria/slf4j-timbre "0.3.12"]
[org.slf4j/log4j-over-slf4j "1.7.26"]
[org.slf4j/jul-to-slf4j "1.7.26"]
[org.slf4j/jcl-over-slf4j "1.7.26"]
;; spec , test & gen
[org.clojure/test.check "0.9.0"]
;; Dependencies for build process
[adzerk/boot-cljs "2.1.5"]
[adzerk/boot-test "1.2.0" :scope "test"]
[pandeiro/boot-http "0.8.3"]
[adzerk/boot-reload "0.6.0"]
[adzerk/boot-cljs-repl "0.4.0"]
[cider/piggieback "0.3.9" :scope "test"]
[nrepl "0.4.5" :scope "test"]
;; [com.cemerick/piggieback "0.2.2" :scope "test"]
[weasel "0.7.0" :scope "test"]
[org.clojure/tools.nrepl "0.2.13" :scope "test"]
[mbuczko/boot-ragtime "0.3.1"]
[deraen/boot-sass "0.3.1"]
[tolitius/boot-check "0.1.12"]])
After a lot of playing around with tasks, I was able to build a war file that works with jetty-runner with the following task:
(deftask build-war
"Build the target/app.war file"
[]
(comp
(aot :all true)
(web :serve 'gcr.core/app)
(cljs :optimizations :advanced)
(sass)
(sift :move {#"(.*)js$" "public/$1js"
#"(.*)html$" "public/$1html"
#"css/(.*)" "public/css/$1"})
(uber)
(war :file "gcr.war")
(sift :include [#"gcr.war"])
(target)))
Using the sift task to move the static resources to the public folder was key, since I had my compojure resources defined as follows in my routes.clj (not including API and other routes).
(defroutes gen-routes
(GET "/login" [] auth/auth-handler)
(files "/" {:root "public"}) ;; to serve static resources
(resources "/" {:root "public"}) ;; to serve anything else
(not-found "404 Page Not Found")) ;; page not found
This however still does not produce a war file that works on tomcat. That is, on deploying to tomcat, all the API routes works as expected, but the static html files come up as HTTP/404.

Clojure lein uberjar: java.lang.ClassNotFoundException

I have a Clojure library that has two gen-class directives. When I run lein run, there are no issues. However, when I run lein uberjar, I get errors:
$ lein uberjar
Compiling 6 source files to /Users/frank/src/user/target/uberjar/classes
Compiling user.common
Compiling user.core
java.lang.ClassNotFoundException: user.server.UserAuthenticationServer, compiling:(user/core.clj:15:30)
Exception in thread "main" java.lang.ClassNotFoundException: user.server.UserAuthenticationServer, compiling:(user/core.clj:15:30)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6926)
.....
at clojure.lang.Compiler.analyze(Compiler.java:6701)
Caused by: java.lang.ClassNotFoundException: user.server.UserAuthenticationServer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
...
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6919)
... 86 more
In addition to the generated java files, there is the project.clj, server.clj, and core.clj.
project.clj
(defproject user "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.9.0-alpha14"]
[io.grpc/grpc-core "1.7.0"]
[io.grpc/grpc-netty "1.7.0"
:exclusions [io.grpc/grpc-core]]
[io.grpc/grpc-protobuf "1.7.0"]
[io.grpc/grpc-stub "1.7.0"]]
:main ^:skip-aot user.core
:aot [user.server]
:target-path "target/%s"
:source-paths ["src/clj"]
:java-source-paths ["src/generated/proto"
"src/generated/grpc"]
:profiles {:uberjar {:aot :all}})
core.clj
(ns user.core
(:import [io.grpc Server ServerBuilder])
(:gen-class))
(defonce start-server-atom (atom nil))
(def port 8080)
(defn start-server []
(when-not #start-server-atom
(reset! start-server-atom
(-> (ServerBuilder/forPort port)
(.addService (new user.server.UserAuthenticationServer))
.build
.start
.awaitTermination))))
(defn -main
[& args]
(start-server))
server.clj
(ns user.server
(:gen-class
:main false
:name user.server.UserAuthenticationServer
:extends xyz.skroo.user.UserAuthenticationGrpc$UserAuthenticationImplBase))
(defn -startUserAuthentication [this req res]
(.onNext res req)
(.onCompleted res))
It's weird because this was working and I think the compile-time order changed and now I cannot generate a standalone jar.
:profiles {:uberjar {:aot :all}} means that when you run uberjar it will try to compile all Namespaces. When you do lein run it only compiles the namespace in the :aot key.
Try to update the uberjar profile to only aot the server namespace and see if that works.

Ragtime Migrate with Environment Variables throwing Error (Heroku Deployment)

I'm trying to run lein ragtime migrate on a heroku dyno. Normally, I would set the database path in my project.clj like so:
(defproject my-project "0.1.0-SNAPSHOT"
:min-lein-version "2.0.0"
:dependencies [[org.clojure/clojure "1.6.0"]
[org.clojure/java.jdbc "0.3.7"]
[postgresql "9.3-1102.jdbc41"]
[ragtime "0.3.9"]
[ring "1.4.0-RC1"]
[ring/ring-defaults "0.1.2"]]
:plugins [[lein-ring "0.8.13"]
[ragtime/ragtime.lein "0.3.9"]]
...
:ragtime {:migrations ragtime.sql.files/migrations
:database (System/getenv "DATABASE_URL")}
...
:profiles
{:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
[ring-mock "0.1.5"]]
:test {:ragtime {:database (System/getenv "DATABASE_URL")}}})
When I run the command, I get the following error both locally and depolying over Heroku
java.lang.IllegalArgumentException: No method in multimethod 'connection' for dispatch value: postgres
Any pointers in the right direction would be very appreciated.
Ragtime 0.3.9 uses the scheme from the connection url as the dispatch value for the connection multimethod. The code is here and here. But the DATABASE_ENV from heroku doesn't have a "jdbc" but a "postgres" scheme (which makes sense, it has to be generic).
A workaround could be to add the "jdbc://" prefix:
:ragtime {:migrations ragtime.sql.files/migrations
:database ~(str "jdbc://" (System/getenv "DATABASE_URL"))}
You can also upgrade to [ragtime "0.4.0"] which doesn't use the scheme to find out how to create the connection. See the wiki for info about the upgrade path from 0.3.x

leiningen with multiple main classes

I'd like to have two main classes (or more) with leiningen, and then be able to choose which one at the java command line. For example I have:
(ns abc (:gen-class))
(defn -main [] (println "abc"))
(ns def (:gen-class))
(defn -main [] (println "def"))
With a project.clj having:
(defproject my-jar "0.0.1"
:description "test"
:dependencies [
]
:main abc)
Then I build with lein uberjar, and run:
java -cp my-jar-0.0.1-standalone.jar abc
java -cp my-jar-0.0.1-standalone.jar def
I get it that when I specified :main abc in the project.clj it was calling that out as the main-class in the manifest, but I couldn't get it to run without putting something. But either way when I try to run the 'def' main, I get a class not found:
Exception in thread "main" java.lang.NoClassDefFoundError: def
This works at least with leiningen 2.0+
(defproject my-jar "0.0.1"
:description "test"
:dependencies [
]
:profiles {:main-a {:main abc}
{:main-b {:main def}}
:aliases {"main-a" ["with-profile" "main-a" "run"]
"main-b" ["with-profile" "main-b" "run"]})
Then you can run each main like so:
lein main-a
lein main-b
Which expands to this:
lein with-profile main-a run
lein with-profile main-b run
I'm using this in one of my projects and it works perfectly.
I added :aot [abc def] to the project.clj to generate compiled code and it worked.
What worked for me in both lein 2.7.0's run task as well as from the resulting uberjar is as follows...
project.clj:
(defproject many-mains "0.1.0-SNAPSHOT"
:description "Project containing multiple main methods"
:dependencies [[org.clojure/clojure "1.8.0"]]
:main nil
:target-path "target/%s"
:profiles {:main-abc {:main many-mains.abc}
:main-def {:main many-mains.def}
:main-ghi {:main org.rekdev.mm.ghi}
:core {:main many-mains.core}
:uberjar {:aot :all}})
For source like...
$ cat src/many_mains/abc.clj
(ns many-mains.abc
(:gen-class))
(defn -main
""
[& args]
(println "Hello, from many-mains.abc!"))
This lets lein run work like...
$ lein with-profile main-abc run
Hello, from many-mains.abc!
From the command line the '-' in many-mains needs to become a '_' which makes it a legal Java classname.
$ java -cp target/uberjar/many-mains-0.1.0-SNAPSHOT-standalone.jar many_mains.abc
Hello, from many-mains.abc!
There seems to have been some behavior changes between Lein 2.7.0 and prior around the effect of :main nil on the MANIFEST.MF. What I've got here works like a champ in Lein 2.7.0. The full source is at https://github.com/robertkuhar/many-mains