Ragtime Migrate with Environment Variables throwing Error (Heroku Deployment) - clojure

I'm trying to run lein ragtime migrate on a heroku dyno. Normally, I would set the database path in my project.clj like so:
(defproject my-project "0.1.0-SNAPSHOT"
:min-lein-version "2.0.0"
:dependencies [[org.clojure/clojure "1.6.0"]
[org.clojure/java.jdbc "0.3.7"]
[postgresql "9.3-1102.jdbc41"]
[ragtime "0.3.9"]
[ring "1.4.0-RC1"]
[ring/ring-defaults "0.1.2"]]
:plugins [[lein-ring "0.8.13"]
[ragtime/ragtime.lein "0.3.9"]]
...
:ragtime {:migrations ragtime.sql.files/migrations
:database (System/getenv "DATABASE_URL")}
...
:profiles
{:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
[ring-mock "0.1.5"]]
:test {:ragtime {:database (System/getenv "DATABASE_URL")}}})
When I run the command, I get the following error both locally and depolying over Heroku
java.lang.IllegalArgumentException: No method in multimethod 'connection' for dispatch value: postgres
Any pointers in the right direction would be very appreciated.

Ragtime 0.3.9 uses the scheme from the connection url as the dispatch value for the connection multimethod. The code is here and here. But the DATABASE_ENV from heroku doesn't have a "jdbc" but a "postgres" scheme (which makes sense, it has to be generic).
A workaround could be to add the "jdbc://" prefix:
:ragtime {:migrations ragtime.sql.files/migrations
:database ~(str "jdbc://" (System/getenv "DATABASE_URL"))}
You can also upgrade to [ragtime "0.4.0"] which doesn't use the scheme to find out how to create the connection. See the wiki for info about the upgrade path from 0.3.x

Related

Problems creating an executeable for a luminus clojure project

I am building a server program to provide an API to a mongo database containing details of photos. All works well running it from the REPL, or if I start it using lein run.
The problems happen when I try and run the uberjar. Building the uberjar works ok:
(master) photo-api: lein uberjar
Compiling photo-api.env
Compiling photo-api.config
Compiling photo-api.core
WARNING: find-keyword already refers to: #'clojure.core/find-keyword in namespace: image-lib.core, being replaced by: #'image-lib.core/find-keyword
Compiling photo-api.db.core
Compiling photo-api.handler
Compiling photo-api.layout
Compiling photo-api.middleware
Compiling photo-api.routes.home
Compiling photo-api.routes.services
Created /Users/iain/Code/Clojure/Luminus/photo-api/target/uberjar/photo-api-0.1.0-SNAPSHOT.jar
Created /Users/iain/Code/Clojure/Luminus/photo-api/target/uberjar/photo-api.jar
(master) photo-api:
but running the jar gives an error:
(master) photo-api: java -jar target/uberjar/photo-api.jar
WARNING: find-keyword already refers to: #'clojure.core/find-keyword in namespace: image-lib.core, being replaced by: #'image-lib.core/find-keyword
read config from resource: "config.edn"
Exception in thread "main" java.lang.RuntimeException: could not start [#'photo-api.db.core/db*] due to
at mount.core$up$fn__420.invoke(core.cljc:92)
at mount.core$up.invokeStatic(core.cljc:92)
at mount.core$up.invoke(core.cljc:90)
at mount.core$bring.invokeStatic(core.cljc:210)
at mount.core$bring.invoke(core.cljc:202)
at mount.core$start.invokeStatic(core.cljc:252)
at mount.core$start.doInvoke(core.cljc:244)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at mount.core$start_with_args.invokeStatic(core.cljc:350)
at mount.core$start_with_args.doInvoke(core.cljc:346)
at clojure.lang.RestFn.invoke(RestFn.java:410)
at photo_api.core$start_app.invokeStatic(core.clj:42)
at photo_api.core$start_app.invoke(core.clj:41)
at photo_api.core$_main.invokeStatic(core.clj:50)
at photo_api.core$_main.doInvoke(core.clj:49)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at photo_api.core.main(Unknown Source)
Caused by: java.lang.NullPointerException
at com.mongodb.ConnectionString.<init>(ConnectionString.java:222)
at com.mongodb.MongoClientURI.<init>(MongoClientURI.java:189)
at com.mongodb.MongoClientURI.<init>(MongoClientURI.java:171)
at monger.core$connect_via_uri.invokeStatic(core.clj:230)
at monger.core$connect_via_uri.invoke(core.clj:230)
at photo_api.db.core$fn__12620.invokeStatic(core.clj:29)
at photo_api.db.core$fn__12620.invoke(core.clj:28)
at mount.core$record_BANG_.invokeStatic(core.cljc:86)
at mount.core$record_BANG_.invoke(core.cljc:85)
at mount.core$up$fn__420.invoke(core.cljc:93)
... 18 more
Or the SNAPSHOT jar.
(master) photo-api: java -jar target/uberjar/photo-api-0.1.0-SNAPSHOT.jar
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/lang/Var
at photo_api.core.<clinit>(Unknown Source)
Caused by: java.lang.ClassNotFoundException: clojure.lang.Var
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
(master) photo-api:
What am I missing here? I'm guessing it's something to do with dev build and production build differences, but I can't spot anything obvious and the error messages are not a lot of help to me.
This is the project file which as far as I remember is mostly straight from the original luminus template:
(defproject photo-api "0.1.0-SNAPSHOT"
:description "API for geting data from photos db"
:url "http://soulflyer.com"
:dependencies [[cider/cider-nrepl "0.14.0"]
[clj-time "0.13.0"]
[com.google.guava/guava "20.0"]
[com.novemberain/monger "3.1.0" :exclusions [com.google.guava/guava]]
[compojure "1.6.0"]
[cprop "0.1.10"]
[funcool/struct "1.0.0"]
[luminus-immutant "0.2.3"]
[luminus-nrepl "0.1.4"]
[luminus/ring-ttl-session "0.3.2"]
[markdown-clj "0.9.99"]
[metosin/compojure-api "1.1.10"]
[metosin/muuntaja "0.3.1"]
[metosin/ring-http-response "0.9.0"]
[mount "0.1.11"]
[org.clojure/clojure "1.8.0"]
[org.clojure/tools.cli "0.3.5"]
[org.clojure/tools.logging "0.4.0"]
[org.clojure/data.json "0.2.6"]
[org.webjars.bower/tether "1.4.0"]
[org.webjars/bootstrap "4.0.0-alpha.5"]
[org.webjars/font-awesome "4.7.0"]
[org.webjars/jquery "3.1.1"]
[ring-webjars "0.2.0"]
[ring/ring-core "1.6.1"]
[ring/ring-defaults "0.3.0"]
[selmer "1.10.7"]
[image-lib "0.2.1-SNAPSHOT"]]
:min-lein-version "2.0.0"
:jvm-opts ["-server" "-Dconf=.lein-env"]
:source-paths ["src/clj"]
:test-paths ["test/clj"]
:resource-paths ["resources"]
:target-path "target/%s/"
:main ^:skip-aot photo-api.core
:plugins [[lein-cprop "1.0.3"]
[org.clojars.punkisdead/lein-cucumber "1.0.5"]
[lein-immutant "2.1.0"]]
:cucumber-feature-paths ["test/clj/features"]
:profiles
{:uberjar {:omit-source true
:aot :all
:uberjar-name "photo-api.jar"
:source-paths ["env/prod/clj"]
:resource-paths ["env/prod/resources"]}
:dev [:project/dev :profiles/dev]
:test [:project/dev :project/test :profiles/test]
:project/dev {:dependencies [[prone "1.1.4"]
[ring/ring-mock "0.3.0"]
[ring/ring-devel "1.6.1"]
[pjstadig/humane-test-output "0.8.2"]
[clj-webdriver/clj-webdriver "0.7.2"]
[org.apache.httpcomponents/httpcore "4.4"]
[org.clojure/core.cache "0.6.3"]
[org.seleniumhq.selenium/selenium-server "2.48.2"]]
:plugins [[com.jakemccrary/lein-test-refresh "0.19.0"]
[refactor-nrepl "2.3.0-SNAPSHOT"]]
:source-paths ["env/dev/clj"]
:resource-paths ["env/dev/resources"]
:repl-options {:init-ns user}
:injections [(require 'pjstadig.humane-test-output)
(pjstadig.humane-test-output/activate!)]}
:project/test {:resource-paths ["env/test/resources"]}
:profiles/dev {}
:profiles/test {}})
I think you're just missing required configuration - at least Mongo connection string.
When running in dev mode, the luminus (and cprops configuration library) uses the config from your profiles.clj.
However, this is the leiningen thing and it's not available when running from uberjar.
You can specify all your "production" configuration in "config.edn" file and run uberjar like this:
java -Dconf=config.edn -jar target/uberjar/photo-api.jar
You can also use java system properties to set configuration values one by one.
See http://www.luminusweb.net/docs/environment.md for more details.

Correct usage of Environ library in profiles

I have a question regarding of the usage of profiles and environment variables. Is it possible to export variables with environ depending on the which profile is set? This is what I have tried (partial project.clj)
:env {:time-to-wait-for-response "72" ; in hours
:crm-address "https://app.onepagecrm.com/api/v3/"}
:profiles
{:uberjar {:omit-source true
:aot :all}
:uberwar {:omit-source true
:aot :all}
:prod {:ring {:open-browser? false
:stacktraces? false
:auto-reload? false}
{:env {:prod? true
:db-user "mailer"
:db-password "" }}
:dev {:env {:db-user "mailer"
:db-password ""}
:dependencies [[ring-mock "0.1.5"]
[ring/ring-devel "1.3.1"]
[midje "1.6.3"]]
:ring {:open-browser? false}
:plugins [[lein-midje "3.2.1"]]}})
However if I build the WAR file with lein ring uberwar(even with with-profile dev) and deploy it to Tomcat I will get db-user as nil (called with (env :db-user). However lein ring server correctly uses dev profile so it works. Is there something wrong I am doing? Or environ is not supposed to be used this way?
This reason it is not working as you expect is because you're only setting those variables when the profile is active. Once you have built a WAR file and deployed it, lein is no longer in the picture. You'll need to load these variables in a different way. If you want to stick with environ, the two options seem to be as Java system properties or environment variables.

How to create an uberjar correctly with "lein uberjar"?

if I use lein run the project is no problem.
But when I use jave -jar blog.jar after lein uberjar It happen exceptions.
16-Jul-20 11:28:05 DESKTOP-C3SC9AR INFO [slf4j-timbre.adapter] - >> starting.. *db*
Exception in thread "main" java.lang.RuntimeException: could not start [*db*] due to
.....
Caused by: java.lang.Exception: :jdbc-url, :datasource, or :datasource-
AND my project.clj file
(defproject blog "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:dependencies ....
:min-lein-version "2.0.0"
:uberjar-name "blog.jar"
:jvm-opts ["-server"]
:main blog.core
:migratus {:store :database}
:plugins [[lein-environ "1.0.1"]
[migratus-lein "0.2.0"]]
:profiles
{:uberjar {:omit-source true
:env {:production true}
:aot :all
:source-paths ["env/prod/clj"]}
:dev [:project/dev :profiles/dev]
:test [:project/test :profiles/test]
:project/dev ...
:project/test ...
:profiles/dev {:env {:database-url "jdbc:postgresql://localhost/blog?user=postgres&password=root"}}
:profiles/test {}})
Maybe it can't find the key of "database-url"?
Data provided via environment settings in project.clj are not provided when you use java directly. It's your responsibility to make sure the environment settings visible to the app provide any credentials or connection info needed.
It could look something like:
DATABASE_URL="jdbc...." java -jar my-project-standalone.jar

project.clj understanding some parts

I am create a web application with duct lein template.
Now I am looking at project.clj to see how its works.
But I don't know if I undersant correcntly this particular part:
{:dev [:project/dev :profiles/dev]
:test [:project/test :profiles/test]
:uberjar {:aot :all}
:profiles/dev {}
:profiles/test {}
:project/dev {:dependencies [[duct/generate "0.6.1"]
[reloaded.repl "0.2.1"]
[org.clojure/tools.namespace "0.2.11"]
[org.clojure/tools.nrepl "0.2.12"]
[eftest "0.1.1"]
[kerodon "0.7.0"]]
:source-paths ["dev"]
:repl-options {:init-ns user}
:env {:port "3000"}}
For example:
Whats this [:project/dev :profiles/dev] mean? and why I have a :dev key and a :project/dev key.
I know it is probably about profiles.clj file. But I did don't really understand it right.
Maybe my question seems a little bit silly, but I hope I have been myself clear.
This is described in composite profiles:
Sometimes it is useful to define a profile as a combination of other
profiles. To do this, just use a vector instead of a map as the
profile value. This can be used to avoid duplication:
{:shared {:port 9229, :protocol "https"}
:qa [:shared {:servers ["qa.mycorp.com"]}]
:stage [:shared {:servers ["stage.mycorp.com"]}]
:production [:shared {:servers ["prod1.mycorp.com", "prod1.mycorp.com"]}]}

Leiningen 2-steps build: how to combine lein-less and lein-asset-minifier

I'm making a Clojure/ClojureScript website that I deploy on Heroku. I'm using Leiningen for project management.
I want to write my stylesheets in LESS. I need the build process to compile the LESS files into CSS, then minify these CSS files; obviously, I do not want the generated CSS files to be under version control.
LESS files --lein-less--> CSS files --lein-asset-minifier--> minified CSS files
I have tried to implement this with the lein-less and lein-asset-minifier Leiningen plugins. My attempt consisted of declaring leiningen.less and minify-assets.plugin/hooks as hooks of the :uberjar task, in proper order (see code below). But running lein uberjar fails with the following error:
Uberjar aborting because jar failed: resources/public/css/site.css (No such file or directory)
So it appears the order of build steps is not enforced.
Is it possible to implement this multi-step build with these Leiningen plugins? If not, how are people doing it?
Code
Here is the relevant part of my project.clj :
(defproject sncf-cljs "0.1.0-SNAPSHOT"
;; ...
:min-lein-version "2.5.0"
:source-paths ["src/clj" "src/cljs"]
:dependencies [
;; ...
]
:plugins [
[lein-cljsbuild "1.0.4"]
[lein-environ "1.0.0"]
[lein-ring "0.9.1"]
[lein-asset-minifier "0.2.2"]
[lein-less "1.7.2"]]
:less {:source-paths ["src/less"]
:target-path "resources/public/css"}
:uberjar-name "sncf-cljs.jar"
:minify-assets {:assets
{"resources/public/css/site.min.css" "resources/public/css/site.css"}}
:cljsbuild {
;; ...
}
:profiles {
;; ...
:uberjar {:hooks [leiningen.less
leiningen.cljsbuild
minify-assets.plugin/hooks]
:env {:production true}
:aot :all
:omit-source true
:cljsbuild {:jar true
:builds {:app
{:source-paths ["env/prod/cljs"]
:compiler
{:optimizations :advanced
:pretty-print false}}}}}
:production {:ring {:open-browser? false
:stacktraces? false
:auto-reload? false}
:cljsbuild {:builds {:app {:compiler {:main "sncf-cljs.prod"}}}}
}})
I did some research because it is an interesting question. So I looked around, and could not find informations how to configure this. So I continued to search and I found a plugin for this task.
https://github.com/kumarshantanu/lein-cascade
The readme on the github page should be sufficient to solve this task.