I have the following aliases
:aliases {
"start-server" ["do" ["ring" "server-headless"]]
"build-site" ["run" "-m" "cjohansen-no.web/export"]
"build-html" ["run" "-m" "cjohansen-no.web/export-pages"]
"build-prod" ["do" ["build-site"] ["cljsbuild" "once" "prod"]]
"build-js" ["do" ["cljsbuild" "auto" "dev" ]]
"watch-stuff" [ "do" ["start-server"] ["auto" "build-html"]]
"build-dev" ["do" ["build-site"] ["build-js"] "watch-stuff" ]
}
For some reason I can't run "build-dev" without it stopping at "build-js" How can I run these task parallel or without be suspending in a previous task?
lein do runs things in sequence, to run in parallel you can use lein pdo instead.
Related
I am doing some Clojure pet project. I have some profiles like the following
{:test {:env {:database-name "library_test",
:host-name "192.168.33.10"
:username "library_admin"
:password ""
:dbtype "postgres"
:driver-class-name "org.postgresql.Driver"}},
:dev {:env {:database-name "library",
:host-name "192.168.33.10"
:username "library_admin"
:password ""
:dbtype "postgres"
:driver-class-name "org.postgresql.Driver"}},
:travis {:env {:database-name "test_library_test",
:host-name "localhost"
:username "test_user"
:password "password"
:dbtype "postgres"
:driver-class-name "org.postgresql.Driver"}}}
Now I am trying to setup Travis-CI for the project. I want to override the value of test profile CI while running test, for that I am using the following command
lein with-profile travis test
Here lein is activating the travis profile but it's picking up the environment variables value from test profile instead of travis profile.
Did anyone face such issues?
Why: Lein merges test profile by default. You can see the effective project map with lein with-profile travis,test pprint
Solution: I assume you're using environ or something like that. If so, you can export values with env in UPCASE_WITH_UNDERSCORE (e.g. DATABASE_NAME=test_library_test) and they will override values in profiles
The https://github.com/yogthos/config approach let's you lay out per-profile env variables in separate files, like the below , in a project.clj .
Per the below, one can use lein with-profile prod uberjar or lein with-profile dev repl and the like.
But my issue is I have been unable to figure out how to place some common values into a shared area, accessible by dev, stage, prod profiles.
Basic example
(defproject edn-config-test "0.1.0-SNAPSHOT"
...
:profiles {:shared {:resource-paths ["config/shared"]}
:dev {:resource-paths ["config/dev"]}
:stage {:resource-paths ["config/stage"]}
:prod {:resource-paths ["config/prod"]}}
...
(with files)
config/shared/config.edn
config/dev/config.edn
config/stage/config.edn
config/prod/config.edn
I tried this without luck
lein with-profile shared,prod lein , borrowing from the composite approach in
https://github.com/technomancy/leiningen/blob/stable/doc/PROFILES.md#composite-profiles
When I do that, I only get variables in prod profile, for example.
I think it is a limitation of config. I tried this (more explicit):
:profiles {:dev {:resource-paths ["config/shared" "config/dev"]}
:prod {:resource-paths [ "config/prod" "config/shared"]}}
However, the last file wins and the first is ignored. So for :dev the shared stuff is ignored, and for :prod the prod stuff is ignored (like it doesn't exist):
config/dev/config.edn => {:special-val :dev-val}
config/prod/config.edn => {:special-val :prod-val}
cat config/shared/config.edn => {:shared-val 42}
and results:
> lein with-profile prod run
(:shared-val env) => 42
(:special-val env) => nil
> lein with-profile dev run
(:shared-val env) => nil
(:special-val env) => :dev-val
Perhaps you'd like to submit an enhancement PR to the project?
Here is the problem. It uses io/resource to read config.edn, which implicitly expects there to be only one file config.edn anywhere on the classpath:
(defn- read-config-file [f]
(try
(when-let [url (io/resource f)]
(with-open [r (-> url io/reader PushbackReader.)]
(edn/read r))) ...
(read-config-file "config.edn")
So you'd have to get away from the hard-coded filename config.edn, and make something like config-dev.edn, config-prod.edn, and config-shared.edn. At least then they could all live in a single ./resources dir.
There's something fundamental I'm not getting here. I expected the following test to pass. But the 2nd test case "staging"/"staging" fails. Its as if with-redefs-fn is failing to advance through the test-case instances. But the logging says everything is fine. This is confusing.
(deftest test-bad-derive-s3-environment
(testing "variants of props environments"
(doseq [test-case [{:env "qa1" :expect "qa1"}
{:env "dev" :expect "qa1"}
{:env "staging" :expect "staging"}]]
(log/infof "test-case %s" test-case)
(with-redefs-fn {#'config/environment (fn [] (:env test-case))}
(let [actual (fs/derive-s3-environment (config/environment))
_ (log/infof "within redefs :env %s :expect %s" (:env test-case) (:expect test-case))]
#(is (= actual (:expect test-case))))))))
...
lein test com.climate.test.mapbook.filestore
2016-05-03 16:16:29,353 INFO filestore:288 - test-case {:env "qa1", :expect "qa1"}
2016-05-03 16:16:29,355 INFO EnvConfig:98 - Loading config properties from /export/disk0/wb/etc/env.properties
2016-05-03 16:16:29,357 INFO EnvConfig:98 - Loading config properties from /export/disk0/wb/etc/local.properties
2016-05-03 16:16:29,358 INFO filestore:288 - within redefs :env qa1 :expect qa1
2016-05-03 16:16:29,359 INFO filestore:288 - test-case {:env "staging", :expect "staging"}
2016-05-03 16:16:29,359 INFO filestore:288 - within redefs :env staging :expect staging
lein test :only com.climate.test.mapbook.filestore/test-bad-derive-s3-environment
FAIL in (test-bad-derive-s3-environment) (filestore.clj:29)
variants of props environments
expected: (= actual (:expect test-case))
actual: (not (= "qa1" "staging"))
2016-05-03 16:16:29,364 INFO filestore:288 - test-case {:env "dev", :expect "qa1"}
2016-05-03 16:16:29,364 INFO filestore:288 - within redefs :env dev :expect qa1
Why does my with-redefs-fn fail to redefine the config/environment function in terms of the current test-case?
First of all, notice that your final test instance has an :expect of "qa1" – the same as the first test instance – so it should actually fail if the code worked as you intended it to; its passing is a symptom of the same problem as the second instance's failing.
Now for the fix – there are two options:
Just use with-redefs instead of with-redefs-fn:
(with-redefs [config/environment (fn [] (:env test-case))]
…)
Most of the time this is what you want to do and you can consider with-redefs-fn to be an implementation detail behind with-redefs – although strictly speaking it does have some utility of its own in that it can redefine dynamically constructed collections of Vars.
Use with-redefs-fn, but move the inner let form inside the anonymous function:
(with-redefs-fn {…}
#(let […]
(is …)))
Finally, the reason these work and the version from the question text does not:
with-redefs-fn is a function, so at runtime its arguments will be evaluated before it is actually invoked with their runtime values passed in. In particular, the let expression that you pass in as the second argument will be evaluated before the redefinition takes place, and so the local called actual will get the result of evaluating (config/environment) before the redefinition as its value, and that value will be installed in the anonymous closure created in the let's body. That closure, however, will then be called with the redefinition in place, and so it will take its notion of the "actual" value from before the redefinition and compare it with the expectation set after the redefinition, resulting in the observed behaviour.
Moving the let inside the closure, as in the second approach above, fixes this mismatch problem – the let local's value is computed with the redefinition in place and all is well. The first approach using with-redefs expands to the second approach.
The log printouts are fine, because they are only concerned with the doseq local and never examine any Vars. If they did, they would only see the pre-redefinition values.
I have a ring project with the following configuration
:ring {:port 3000
:handler myservice.core/standalone-app
:init myservice.core/init!
:destroy myservice.core/destroy!}
These functions are simple, they just log. They may do more someday.
(defn init! [] (log/info "init!"))
(defn destroy! [] (log/info "destroy!"))
I build this kid with the uberwar thing. lein ring uberwar myservice.war
The jetty log shows the init! logging on startup, but the destroy! logging is nowhere to be seen. Is destroy even being called? How can I tell?
The full project is at https://github.com/robertkuhar/myservice
It turns out I needed to configure the logging framework. It seems strange in the absence of logging configuration allows init! to log to stdout, but destroy! goes nowhere.
I added logback to my project.clj as a dependency and gin'ed up a skeleton logback.xml config file and, viola!
2015-12-23 11:42:53.192:INFO:oejs.Server:jetty-8.1.16.v20140903
2015-12-23 11:42:53.208:INFO:oejdp.ScanningAppProvider:Deployment monitor /Users/robert.kuhar/work/jetty-distribution-8.1.16.v20140903/webapps at interval 1
...
2015-12-23 11:42:53.598:INFO:oejd.DeploymentManager:Deployable added: /Users/robert.kuhar/work/jetty-distribution-8.1.16.v20140903/webapps/myservice.war
2015-12-23 11:42:53.600:INFO:oejw.WebInfConfiguration:Extract jar:file:/Users/robert.kuhar/work/jetty-distribution-8.1.16.v20140903/webapps/myservice.war!/ to /private/var/folders/1g/fnytl2x93sx6hp2f1rsf4h1r5xtqv_/T/jetty-0.0.0.0-8080-myservice.war-_myservice-any-/webapp
2015-12-23T11:42:58.851 INFO m.core - app
2015-12-23T11:42:58.856 INFO m.core - standalone-app
println init!
2015-12-23T11:42:58.857 INFO m.core - init!
...
2015-12-23 11:42:59.146:INFO:oejs.AbstractConnector:Started SelectChannelConnector#0.0.0.0:8080
...
2015-12-23 11:45:48.432:INFO:oejs.Server:Graceful shutdown SelectChannelConnector#0.0.0.0:8080
...
2015-12-23 11:45:48.434:INFO:oejs.Server:Graceful shutdown o.e.j.w.WebAppContext{/myservice,file:/private/var/folders/1g/fnytl2x93sx6hp2f1rsf4h1r5xtqv_/T/jetty-0.0.0.0-8080-myservice.war-_myservice-any-/webapp/},/Users/robert.kuhar/work/jetty-distribution-8.1.16.v20140903/webapps/myservice.war
println destroy!
2015-12-23T11:45:49.553 INFO m.core - destroy!
...
2015-12-23 11:45:49.553:INFO:oejsh.ContextHandler:stopped o.e.j.w.WebAppContext{/myservice,file:/private/var/folders/1g/fnytl2x93sx6hp2f1rsf4h1r5xtqv_/T/jetty-0.0.0.0-8080-myservice.war-_myservice-any-/webapp/},/Users/robert.kuhar/work/jetty-distribution-8.1.16.v20140903/webapps/myservice.war
...
Newbie Clojure and leiningen question:
Given the code snippet in my project below, this works from the lein repl :
==> (-main "something")
produces the expected "Command: something ... running ... done"
but doesn't work from the command line:
me pallet1]lein run "something"
produces "Command: something ... error: not resolved as a command"
Why? / how do I fix it?
To reproduce:
lein new eg
Then edit the generated project file, adding :main eg.core to define the main function, and edit the generated src/eg/core.clj file, and paste this in:
core.clj
(ns eg.core)
(defn something [] (println "Something!"))
(defn run-command-if-any [^String commandname]
(printf "Command: %s ..." commandname)
(if-let [cmd (ns-resolve *ns* (symbol commandname))]
(
(println "running ...") (cmd) (println "done.")
)
(println "error: not resolved as a command.")
))
(defn -main [ commandname ] (run-command-if-any commandname))
Then
lein repl
eg.core=> (-main "something")
works (ie prints "Something!) , but
lein run something
doesn't (ie prints the "error: not resolved" message)
The problem is that when you run it from lein your default namespace is "user" namespace:
(defn -main [ commandname ] (println *ns*))
Prints #<Namespace user>. So it doesn't contain something function because it is from another namespace. You have several choices:
Pass fully qualified function name: your-namespace/something instead of something.
Use your-namespace instead of *ns*: (ns-resolve 'your-namespace (symbol commandname))
Change namespace to your-namespace in -main.
Example of method 3:
(defn -main [ commandname ]
(in-ns 'your-namespace)
(run-command-if-any commandname))
Also you if you want to call several functions one by one you should use do:
(do (println "Hello")
(println "World"))
Not just braces like ( (println "hello") (println "World"))
the lein exec plugin is very useful for scripting such things in the context of a project. I have used this extensively for writing Jenkins jobs in clojure and other scripting situations
lein exec -pe '(something ...) (something-else) (save-results)'