I'm used to using Clojure with Leiningen.
But I have a new project that I booted into existence with LightMod. And it uses CLI tools and deps.
This works but I now want to add some unit testing to it.
So here's (a slightly simplified version of) my deps.edn file :
{:paths ["src" "resources"],
:aliases
{
:dev
{:extra-deps
{orchestra #:mvn{:version "2018.12.06-2"},
expound #:mvn{:version "0.7.2"},
nightlight #:mvn{:version "RELEASE"},
com.bhauman/figwheel-main #:mvn{:version "0.2.0"}},
:main-opts ["dev.clj"]},
:test
{:extra-paths ["test"]
:extra-deps {com.cognitect/test-runner {:git/url "https://github.com/cognitect-labs/test-runner.git" :sha "209b64504cb3bd3b99ecfec7937b358a879f55c1"}}
:main-opts ["-m" "cognitect.test-runner"]}
:prod
{:extra-deps
{leiningen #:mvn{:version "2.9.0"},
org.clojure/clojurescript #:mvn{:version "1.10.439"}},
:main-opts ["prod.clj"]},
:app
{:extra-deps
{
markdown-clj {:mvn/version "1.10.1"}
instaparse {
:mvn/version"1.4.10"
}
... MORE ...
}}}}
Now this file has worked OK with both :dev and :prod compilation.
But now I've just added the :test alias.
And put my tests into test/myns/core_tests.clj
However, when I try to run tests, I get this
Could not locate markdown/core__init.class, markdown/core.clj or markdown/core.cljc on classpath
So my interpretation. While the :dev and the :prod branches are successfully pulling in all the dependencies in the :app branch. The :test branch is not pulling in markdown. (And probably none of the other dependencies declared in :app
So why should this be? And how can I explicitly tell my :test clause that it needs to use all the same dependencies that are used in :dev and :prod?
You can use the Kaocha test runner for this. A working template project can be found here. It even includes the ability to compile Java source files as well as Clojure.
To get started, set up deps.edn like so
{:paths ["src/clj"]
:deps {
cambium/cambium.codec-simple {:mvn/version "0.9.3"}
cambium/cambium.core {:mvn/version "0.9.3"}
cambium/cambium.logback.core {:mvn/version "0.4.3"}
org.clojure/clojure {:mvn/version "1.10.1"}
prismatic/schema {:mvn/version "1.1.12"}
tupelo {:mvn/version "0.9.200"}
}
:aliases {:test {
:extra-deps {lambdaisland/kaocha {:mvn/version "1.0-612"}}
}}
}
and add another file tests.edn to configure Kaocha:
; ***** Be sure to use tagged reader literal '#kaocha/v1' *****
#kaocha/v1 {:tests
[{:id :unit
:test-paths ["test/clj"]
:ns-patterns ["^tst..*"]
}]
; :reporter kaocha.report.progress/report
; :plugins [:kaocha.plugin/profiling :kaocha.plugin/notifier]
}
then create a short shell script to launch Koacha in the subdir ./bin. Name it bin/kaocha with contents:
#!/bin/bash
clojure -A:test -m kaocha.runner "$#"
Now you are ready to go! A test file:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test) )
(dotest
(is= 5 (+ 2 3))
(isnt= 9 (+ 2 3))
(throws? (/ 1 0)) ; verify that an illegal operation throws an exception
(is= 3 (add2 1 2))
(throws? (add2 1 "two"))) ; Prismatic Schema will throw since "two" is not a number
; NOTE: Clojure Deps/CLI can't handle Java source code at present
and we are off to the races:
~/io.tupelo/clj-template > bin/kaocha
[(.....)]
1 tests, 5 assertions, 0 failures.
Related
I'm trying to use a function as the value for the :profiles key in a defproject form. Starting from a fresh project (lein new app test) this works fine:
:profiles {}
(as you might hope!). But if I change it to:
:profiles (merge {})
then when I run lein repl it explodes:
Caused by: java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to java.util.Map$Entry
I'm confused by this since if I set :profiles back to the empty map and ask the repl these things are equal:
test.core=> (= {} (merge {}))
true
Where is my misunderstanding? Have I missed something basic? Is this an unfortunate artifact of the defproject macro? Something else?
(clojure 1.8.0, leiningen 2.7.1, java 1.8.0_102)
Edit - working solution with Scott's answer:
(def project-name 'myproj)
(def mains ["foo" "bar"])
...
(defn- lein-alias [main]
{ main ["with-profile" main] })
(defn- lein-profile [main]
(let [jar (str main ".jar")
entry `~(str project-name "." main)]
{(keyword main) {:main entry
:bin {:name main}
:jar-name jar
:uberjar-name jar}}))
(defproject project-name "0.1.0"
...
:profiles ~(apply merge (concat (map lein-profile mains) {:uberjar {:aot :all}}))
:aliases ~(apply merge (map lein-alias mains))
...
So now I can lein foo bin and lein bar bin to my heart's content.
If you unquote your form Leiningen will execute the form before the project map is evaluated. So ~(merge {}) should work.
There is a function called unquote-project in Leiningen
src/leiningen/core/project.clj#L176
"Inside defproject forms, unquoting (~) allows for arbitrary evaluation."
It looks like it looks items to unquote to allow them to be execute. Based on the comment it looks like this might go away in 3.0 and suggests use read-eval syntax
Note: If you are just trying to merge values of different profiles you should look at the Profiles documentation About Merging and Composite Profiles
Composite Profiles
Sometimes it is useful to define a profile as a combination of other profiles. To do this, just use a vector instead of a map as the profile value. This can be used to avoid duplication:
{:shared {:port 9229, :protocol "https"}
:qa [:shared {:servers ["qa.mycorp.com"]}]
:stage [:shared {:servers ["stage.mycorp.com"]}]
:production [:shared {:servers ["prod1.mycorp.com", "prod1.mycorp.com"]}]}
You do not merge the profiles manually. They are merged together by lein either automatically (normal case) or when you use the with-profile keyword (manual control).
For example, consider this project.clj:
(defproject xyz "0.1.0-SNAPSHOT"
:dependencies [ [org.clojure/clojure "1.8.0"]
[tupelo "0.9.19"] ]
:profiles {:dev {:dependencies [ [org.clojure/test.check "0.9.0"]
[criterium "0.4.4"] ] }
:sample {:dependencies [medley "0.8.2"] }
...
)
This project.clj says that the project always requires both org.clojure/clojure and tupelo. During development, the map for :dev will be merged into the root-level, so :dependencies will be updated to include both test.check and criterium. The :dev profile values are not included when a uberjar is created, however, so these libs won't be included in the code delivered to users.
Since :sample is not one of the default profiles, it will only be included if you use a command like:
> lein with-profile sample test
Notice that the leading colon is not included on the command line, although we use a keyword :sample with the colon in the project.clj file.
Full details are here: https://github.com/technomancy/leiningen/blob/master/doc/PROFILES.md#default-profiles
and here: https://github.com/technomancy/leiningen/blob/master/sample.project.clj
Having said all this, I normally don't need to use :profiles. Unless you have something more complex than usual, you should usually just put all of your dependencies at the root level (i.e. the :dependencies keyword at the first level under (defproject xyz ...) in the project that lein new app xyz gives you.
Another suggestion: the word test is used in many places in a lein project (directory name, file name suffix, and others), so it can be very confusion to name the project itself test! You'll save yourself (and any other readers) some grief if you choose a unique name like xyz, joe, or anything else.
In my clojure project I built several java classes using gen-class command. They are [extractor.yaml YAMLExtractor YAMLExtractorFactory]. I wanted now build unit test against those classes but I have error: java.lang.ClassNotFoundException: extractor.yaml.YAMLExtractor when I run test.
File which cause the error is:
yaml_extrator_factory.clj
(ns extractor.yaml-extractor-factory
(:gen-class :name extractor.yaml.YAMLExtractorFactory
:extends org.apache.any23.extractor.SimpleExtractorFactory
:implements [org.apache.any23.extractor.ExtractorFactory]
:init init
:constructors {[] [String org.apache.any23.rdf.Prefixes java.util.Collection String]}
:main false)
(:require [extractor.yaml-extractor])
(:import [extractor.yaml YAMLExtractor]
[org.apache.any23.rdf Prefixes]
[org.apache.any23.extractor SimpleExtractorFactory ExtractorFactory Extractor ExtractionContext ]))
The error occurs only during testing. Whole project is AOT-compilable with no error and it is fine when I build a jar file as well.
The test simple.clj contains head:
(ns extractor.simple
(:use [clojure.tools.logging :as log]
[clojure.java.io :as jio]
[clojure.test :as test])
(:require [extractor.yaml-extractor-factory])
(:import [java.util Arrays]
[extractor.yaml.YAMLExtractorFactory]))
And test which prints CLASSPATH. yaml-extractor-factory was not used.
Test is run with command:
boot aot -a update-classpath run-test -t extractor.simple
where task update-classpath adds (get-env :directories) into classpath
and run-test runs a test. run-test works fine with normal clojure code.
run-test is my task with following content:
(deftask run-test "Run unit tests"
[t test-name NAME str "Test to execute. Run all tests if not given."]
(require '[extractor.simple])
(with-pass-thru [_]
(if (nil? test-name)
(do
(util/info "Run all tests")
(test/run-all-tests))
(do
(util/info (format "Run test: %s" (:test-name *opts*)))
(test/run-tests (symbol (:test-name *opts*)))
))))
I've got a two profiles defined in project.clj, one locally, one for testing on travis:
:profiles {:dev {:dependencies [[midje "1.6.0"]
[mysql/mysql-connector-java "5.1.25"]]
:plugins [[lein-midje "3.1.3"]]
:user "root" :pass "root"}
:travis {:user "travis" :pass ""}}
I'm hoping to be able to get access to the :user and :pass values in my projects. How can this be done?
Update:
I also want to be able to use the lein with-profile command... so my tests would have:
lein with-profile dev test
-> would use "root", "root" credentials
lein with-profile dev,travis test
-> would use "travis", "" credentials
If you don't need the values defined in project.clj for anything else (IE, you're free to choose the representation) consider Environ.
You can then define the following in your project.clj
:profiles {:dev {:env {:user "root" :pass "root"}}}
and read the values:
(use 'environ.core)
(def creds
{:user (env :user)
:pass (env :pass)})
This has the advantage that you can also specify the values using environment variables and system properties.
Leiningen's build file is Clojure code so you can just read it in:
(->> "project.clj" slurp read-string (drop 3) (partition 2) (map vec) (into {})
:profiles :dev)
; => {:dependencies [[midje "1.5.1"] [ring-server "0.2.8"]], :plugins [[lein-midje "3.1.0"]]}
If you need heavier functionalities (such as access to the final project map) then something like configleaf might be better suited.
Another way to manage this (which I've utilized quite often) is to have a separate config file for profile specific data:
example/profiles/travis/example/config.clj:
(ns example.config)
(def user "travis")
(def pass "")
example/dev-resources/example/config.clj:
(ns example.config)
(def user "root")
(def pass "root")
example/src/example/core.clj:
(ns example.core
(:require [example.config :as config]))
(println config/user)
And you need to add the profile specific resource path to your project.clj:
:profiles {:travis {:resource-paths ["profiles/travis/"]}}
Is it possible to grab the project information within the clojure repl?
For example, if there was a project defined:
(defproject blahproject "0.1.2" ....)
When running a repl in the project directory, is there a function like this?
> (project-version)
;=> 0.1.2
While you can parse project.clj yourself, this may be annoying. It's also a lot of work. Instead, you can just do:
(System/getProperty "projectname.version")
Leiningen project files are just Clojure data :)
(-> "/path/to/project.clj" slurp read-string (nth 2))
I use environ (https://github.com/weavejester/environ) which sucks in settings from a number of sources, including system properties. The project version appears as :<project-name>-version:
foobar.repl=> (require '[environ.core :refer [env]])
nil
foobar.repl=> (:foobar-version env)
"0.1.0-SNAPSHOT"
Add the below code to the end of project.clj:
(def project (assoc-in project [:repl-options :init]
`(~'def ~'project-version ~(project :version))))
Now you will have a var called project-version in the initial namespace for the repl.
As described in this discussion.
(ns myproject.example
(:require [clojure.java.io :as io])
(:import java.util.Properties))
(defn get-version [dep]
(let [path (str "META-INF/maven/" (or (namespace dep) (name dep))
"/" (name dep) "/pom.properties")
props (io/resource path)]
(when props
(with-open [stream (io/input-stream props)]
(let [props (doto (Properties.) (.load stream))]
(.getProperty props "version"))))))
(get-version 'myproject) ; => 0.1.0
(get-version 'org.clojure/clojure) ; => 1.3.0
As vemv said, Leiningen project files are just Clojure data. So, it's easy to access your project as an ordinary hash-map:
(->> "project.clj"
slurp
read-string
(drop 2)
(cons :version)
(apply hash-map)
(def project))
If you need this variable only in your repl, you can add it to repl-options to your project.clj:
(defproject yourproject "0.1.0"
:description ""
:url ""
:dependencies [ [org.clojure/clojure "1.4.0"]]
:repl-options { :init (->> "project.clj"
slurp
read-string
(drop 2)
(cons :version)
(apply hash-map)
(def project))})
Now, you have project variable in your repl. So, to access the version of your project you can simply type (:version project).
Of course, you can simply use native Leiningen code to parse you project file:
(defproject yourproject "0.1.0"
:description ""
:url ""
:dependencies [ [org.clojure/clojure "1.4.0"]
[leiningen-core "2.1.3"]]
:repl-options { :init (do (require 'leiningen.core.project)
(def project
(leiningen.core.project/read)))})
But, if you need only the version of your project and nothing more, then it's best to use Ankur's solution.
For a more fully-featured approach, you might want to take a look at the configleaf plugin for Leiningen (https://github.com/davidsantiago/configleaf). It will make the project map, with active profiles merged in, available to project code in a namespace of your choosing.
In case you need to do this from clojurescript you could create a macro (from another clj file) and call it from the cljs code :
;;ex: macro.clj
(defmacro get-project-version []
(System/getProperty "penelope.version"))
;;my_logic_code.cljs
(ns my-logic-code
(:require-macros [macros :as m]))
(def project-version (m/get-project-version))
One of the ways to get an organization to accept an alternate JVM language is to first use it for unit testing Java code -- "Boss, I am just going to write some unit tests in XXX. It'll never go out into production."
Are there any tutorials for doing this in Clojure?
I have just started using Scala to do this to test a Java REST server. Writing the tests in Scala allows me to embed expected XML output, mock the database calls with literal List objects, etc., not to mention that traits make it very easy to abstract out common code for the tests.
Basically what you need is clojure.test (or one of the many other clojure test libs) and standard Clojure Java interop.
Example:
(ns example.test-java-util
(:use
[clojure.test])
(:import [java.util HashSet]))
(defn new-empty-set []
(HashSet.))
(deftest test-empty-set
(is (= 0 (.size (new-empty-set))))
(is (= true (.isEmpty (new-empty-set))))
(is (= (new-empty-set) (new-empty-set))))
(deftest test-add-remove
(is (= (new-empty-set)
(doto (new-empty-set)
(.add "xyz")
(.remove "xyz")))))
And you would then run them in a variety of ways. Build tools like Maven using the maven clojure plugin run them automatically as part of "mvn test". In a repl, you can do something like:
example.test-java-util> (run-tests 'example.test-java-util)
Testing example.test-java-util
Ran 1 tests containing 4 assertions.
0 failures, 0 errors.
{:type :summary, :test 1, :pass 4, :fail 0, :error 0}
Here is an example using Leiningen, test.check and assuming a standard Maven layout:
pom.xml
project.clj
src
main
java
quicktest
Discontinuities.java
test
clojure
quicktest
test_discontinuities.clj
The Java function to test:
package quicktest;
public class Discontinuities {
public static double f5(double x) {
return x / (x-5);
}
}
The Clojure test case:
(ns quicktest.test-discontinuities
(:import [quicktest Discontinuities])
(:require [clojure.test :refer :all]
[clojure.test.check :as tc]
[clojure.test.check.generators :as gen]
[clojure.test.check.properties :as prop]
[clojure.test.check.clojure-test :as ct :refer (defspec)]))
(deftest test-single-case
(is (= 2.0 (Discontinuities/f5 10))))
(defspec test-discontinuities 1e4
(prop/for-all [x gen/nat ]
(let [y (Discontinuities/f5 x)]
(is (<= y x)))))
The project:
(defproject quicktest/discontinuities "0.1"
:dependencies [[org.clojure/clojure "1.8.0"]
[org.clojure/test.check "0.9.0"]]
:java-source-paths ["src/main/java"]
:test-paths ["src/test/clojure"])
The pom:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>quicktest</groupId>
<artifactId>discontinuities</artifactId>
<version>0.1</version>
</project>
Run with:
mvn compile
lein deps
lein test
Results
The flaw in the function is quickly found:
FAIL in (test-discontinuities) (test_discontinuities.clj:13)
expected: (<= y x)
actual: (not (<= Infinity 5))
{:test-var "test-discontinuities",
:result false,
:seed 1431128331945,
:failing-size 23,
:num-tests 24,
:fail [5],
:shrunk {:total-nodes-visited 3, :depth 0, :result false, :smallest [5]}}