I am looking at the unittest docs and I see that I could put multiple suite blocks in one test file.
However, I would like to have multiple test files and run them all with a single command. I could write a bash script to compile and run each script one after another:
#!/bin/bash
nim c -r test1.nim
nim c -r test2.nim
...
But is there a better way? For example in Python I can automatically discover and run all files of the form test*.py.
Put all your unit tests in a tests directory, running nimble test will run all of them.
Related
I'm writing unit tests using rackunit and running them using raco through the command raco test ., which recursively finds all racket files and executes them, including files that do not contain unit tests at all. It just runs everything.
According to what I investigated, the idiomatic way to write unit tests in racket is to write a test module inside the implementation file, and not have a separate file for them. The problem is that I have multiple different implementations of the same specification that share the same unit tests, so I need to import all the implementations in a separate file and run the unit tests there.
So I end up with just a few unit test files, and a lot of racket files that do not contain unit tests, but raco test ends up scanning everything anyway.
I'm looking for a way to use raco test to only run files that have a specific pattern in the name. Like *.test.rkt, *.spec.rkt, test-*.rkt, or similar (like in jest for js or pytest for python).
Is this possible?
I looked into test-include-paths in info.rkt files, but it seems to be for files that are not racket files, and not for whitelisting which files to run.
If you're using bash, zsh, or ksh93 as your shell, you can use a recursive glob to find all the matching files:
raco test **/*.test.rkt
(bash might need a shopt -s globstar first)
or any Unix-y system:
find . -name "*.test.rkt" -exec raco test \{\} +
If I have a bunch of tests for my project, I can run them - after cmakeing and makeing to build, building - with make test.
But what if I only want to run one of my tests? That is, one of the items for which I have a add_test() in the tests CMakeFile.txt ?
tl;dr - Do this:
cd $YOUR_BUILD_DIRECTORY
ctest -R name_of_your_test
Explanation
Here is how you likely got confused:
You were trying to do this with make, since make test runs all the tests for you. This won't work for a single test (although there's a workaround - see #danger89's answer). ctest is the cross-platform way to do it.
You started using ctest, e.g. in your main project directory, and you probably got something like:
*********************************
No test configuration file found!
*********************************
Usage
ctest [options]
which wasn't helpful.
... So you thought "Ok, maybe ctest has a -C switch, like CMake and GNU Make" - and indeed, it has a -C switch! but that didn't do what you expected:
[joeuser:/home/joeuser/src/myproj]$ ctest -C build
Test project /home/joeuser/src/myproj
No tests were found!!!
What you actually need to do:
cd $YOUR_BUILD_DIRECTORY
ctest -R name_of_your_test
(note that -R matches a regular expression.) This should work. Note that you can list the tests to be run rather than actually run them by passing -N to ctest.
Thanks goes to #RTsyvarev for pointing me in the right direction
einpoklum is not fully correct. Actually you can use make test if you like. CMake will generate a Makefile and I found out that it accepts an ARGS variable.
Meaning you are able run specific tests via for example the -R (regex) parameter. Which will look like this when using make test:
make test ARGS="-R '^test_'"
This will run only the testcases files that starts with test_ in their filename.
Note: -R above is just an example, it will accept all the arguments that ctest command will accept as well.
Anyway the make test example above will do exactly the same as running:
ctest -R '^test_'
I have a Go application with a number of unit and benchmark tests both in the root and in a subfolder called "message".
I execute the following command to run all unit tests from the root including the ones in the messages and any other subfolder:
go test ./...
I want to achieve the same for the benchmark tests, i.e. run them all. The following works for the ones in the root directory:
go test -bench .
The benchmark tests in the /messages folder are ignored which is expected. So I run the following from the root:
go test -bench ./...
That's not recognised at all, Go seems to execute the unit tests that are located in the root dir. I even tried to specify the message folder in the command as follows:
go test -bench ./message
...but it also failed. Currently if I want to run the benchmark tests in the message folder I have to cd into that folder and execute
go test -bench .
like above.
So what's the correct way then? How can I tell Go to find the benchmark tests both in the root and the subfolders? How does the regexp arg work in the case of the -bench flag? Apparently it's different from the regexp for the unit test runner.
You should use ./... to bench all the files from the current working directory and all of its subdirectories. If you wish to get a more verbose output you can use the -v flag. Also it's good to list the memory allocation by using -benchmem.
go test -v ./... -bench=. -run=xxx -benchmem
-bench flag takes regex so to run all benchmarks (-bench .) in all packages: go test -bench=. ./...
I have a project structure like this:
pkg
|
--pkg.go
--pkg_test.go
--a.go
--a_test.go
--b.go
--b_test.go
--c.go
--c_test.go
I wish to get the coverage for all the source files belonging to the package i.e.(pkg.go, a.go, b.go and c.go). However, when I run:
go test -v pkg
tests are run for only 1/4 go files.
Is there any way I can test my package without moving all the test codes within one file and keeping the file structure intact ?
if your working directory is that of your package, to test all of the files you could run:
go test ./...
if you wanted to get test coverage, you could run:
go test ./... -cover
Is there any way to provide some command-line argument in order to skip all tests but one on some module? So I will not need to change pom.xml every time I will need to run another test?
For example, I want to create build configuration on TeamCity, and provide command-line arguments to run only single test in some module. Next time I will need to change it and run another test, and so on.
Perhaps it is not how CI is intended to be used, but still.
I assume you've read the docs about running a single test under surefire? What they don't tell you is how to do that in a sub-module:
mvn test -Dtest=testname -pl subproject
Where subproject is the project containing that test. From the mvn man page:
-pl,--projects arg Comma-delimited list of specified reactor projects to build instead of all projects. A project can be specified by [groupId]:artifactId or by its relative path.
Other answers I see are not fully complete, for projects that depend on other sub-modules to be built. One option is to run mvn install to have the required jars to be installed into ~/.m2/..., but that option is not very "clean".
Following command will build the sub-modules, and run only the test class that is specified. This is to be run at parent module level. Also, no need to specify sub-module name.
mvn test -DfailIfNoTests=false -Dtest={test_class_name} -am
As an aside, this can also be mvn clean test -Dfa...... I have a habit of always running clean when running tests.
References..
-am will make all the other sub-modules.
-DfailIfNoTests=false does not fail the entire process since we are not intending to run tests in other modules.
-pl option is not needed since -am is already building everything
In case the module to be tested depends on other projects, solution works by changing commands as:
mvn test -DfailIfNoTests=false -Dtest=testname -pl subproject
FWIW, if you have a multi-module project, you can run all tests with this command at parent directory.
mvn test -pl subproject
And the subproject's name can be found by running the following command, usually in the form of group-id:artifact-id.
mvn help:active-profiles