Is there any way to provide some command-line argument in order to skip all tests but one on some module? So I will not need to change pom.xml every time I will need to run another test?
For example, I want to create build configuration on TeamCity, and provide command-line arguments to run only single test in some module. Next time I will need to change it and run another test, and so on.
Perhaps it is not how CI is intended to be used, but still.
I assume you've read the docs about running a single test under surefire? What they don't tell you is how to do that in a sub-module:
mvn test -Dtest=testname -pl subproject
Where subproject is the project containing that test. From the mvn man page:
-pl,--projects arg Comma-delimited list of specified reactor projects to build instead of all projects. A project can be specified by [groupId]:artifactId or by its relative path.
Other answers I see are not fully complete, for projects that depend on other sub-modules to be built. One option is to run mvn install to have the required jars to be installed into ~/.m2/..., but that option is not very "clean".
Following command will build the sub-modules, and run only the test class that is specified. This is to be run at parent module level. Also, no need to specify sub-module name.
mvn test -DfailIfNoTests=false -Dtest={test_class_name} -am
As an aside, this can also be mvn clean test -Dfa...... I have a habit of always running clean when running tests.
References..
-am will make all the other sub-modules.
-DfailIfNoTests=false does not fail the entire process since we are not intending to run tests in other modules.
-pl option is not needed since -am is already building everything
In case the module to be tested depends on other projects, solution works by changing commands as:
mvn test -DfailIfNoTests=false -Dtest=testname -pl subproject
FWIW, if you have a multi-module project, you can run all tests with this command at parent directory.
mvn test -pl subproject
And the subproject's name can be found by running the following command, usually in the form of group-id:artifact-id.
mvn help:active-profiles
Related
I have this build :
It works fine. The only issue is that the Test Results are overridden. So I actually end up with the test results for the last test project executed.
This is executed by build engine;
C:\Program Files\dotnet\dotnet.exe test C:/agent/_work/4/s/test/Services.UnitTests/project.json --configuration release -xml ./TEST-tle.xml
C:\Program Files\dotnet\dotnet.exe test C:/agent/_work/4/s/test/Web.UnitTests/project.json --configuration release -xml ./TEST-tle.xml
What could help:
1) having "dotnet test" generate XML output file - did not find a way how to do that
2) Use a variable for -xml output file in Build Task. That variable could be a random string/number or just a project name being tested - like what Build engine feeds to "dotnet.exe test". No way how to do that.
Any ideas? Thanks.
I think that, although you're running the task against all of the projects in one go, as the .Net Core (Preview) task doesn't have a working directory, that the test results are being generated at solution root (or similar) and done for each project in turn.
I set mine up using simple command line tasks...
Tool: dotnet
Arguments: test -xml testresults.xml
Working folder: {insert the folder for the project to test here}
These work fine but I have one set up for each project. You could try creating a task for each library and adding the full path to the test results argument (or name them appropriately as starain suggested).
This feels like a minor bug to me.
Based on my test, it doesn’t recognize the date variable as Build Number.
To deal with this issue, you can add another .Net Core (Test) step to run xunit test with different result file.
For example:
I'm using VSCode and the Ionide suite of packages to create a console application in F#. I need to add unit tests to the application so that when I ctrl+shift+p FAKE: Build the project, the tests are run during the build process.
I've created a dummy project in Github as an example.
Initially, the test dir was not there. I created the test dir and into that folder created a second project TestProj.Test (in hindsight, I should have used more descriptive names) for testing purposes. I added the .fsproj file from TestProj to this project so that I could reference the SimpleFunctions.fs. NUnit.Framework and FsUnit are added to the TestProj.Test. Test.fs contains two simple tests.
I intentionally created the TestProj.Test as an F# library because I read on SO that the testing project needed to be a library rather than a console app.
I added lines 9, 31-37, and 47 to the default build.fsx file that comes from Ionide.. However, when I build the whole project (i.e., TestProj), the build fails and I get the following error:
1) System.Exception: NUnit: cannot run tests (the assembly list is empty).
at Fake.NUnitSequential.NUnit(FSharpFunc`2 setParams, IEnumerable`1 assemblies) in C:\code\fake\src\app\FakeLib\UnitTest\NUnit\Sequential.fs:line 22
at FSI_0005.Build.clo#31-3.Invoke(Unit _arg3)
at Fake.TargetHelper.runSingleTarget(TargetTemplate`1 target) in C:\code\fake\src\app\FakeLib\TargetHelper.fs:line 492
Line 22 of the Sequential.fs suggests that assemblies is empty.
What am I doing wrong? How should I set up the build.fsx file so that the tests in TestProj.test run successfully? Alternatively, is there something wrong with the Tests.fs file in TestProj.Test? This seems particularly difficult; is there an easier way to include tests that run automatically with VSCode, Iondide, and F#?
There are a few issues in your project:
trying to test before build "Clean" ==> "Test" ==> "Build" ==> "Deploy"
=> change target dependencies to "Clean" ==> "Build" ==> "Test" ==> "Deploy"
separate paket configuration for test (paket.dependencies, paket.lock in test subfolder) which leads to inconsistent versions of referenced dependencies
=> remove paket.dependencies and paket.lock from test
poisonous mix of NUnit versions
=> remove explicit references to NUnit.Framework from paket.dependencies and run paket.exe install
invalid type extension in test project
=> change to type Test() or delete useless file
building creates output of all projects (and not just src/app) in ./build but tests look for DLLs in ./test
=> change test file pattern to buildDir + "**/*.Test.dll"
if you want to use NUnit3
=> open Fake.Testing and use NUnit3 instead of NUnit
finally, you should commit paket.bootstrapper.exe
I recommend you either use a predefined template or start small and make sure you understand each step and check that it is working as expected. Once you've run over the point of a non-working solution it is extremely hard to get back on track.
In a meson build, is it possible to set the working directory before executing unit tests? It uses ninja by default to run the tests, so perhaps there's an option you can pass to ninja to set the directory?
Why I'm asking is sometimes unit tests need access to config/data files (I usually try to avoid this, but sometimes it's just not possible) and they need to know the relative path in order to load them.
It appears the appropriate syntax for this is to append workdir to arguments passed to the test() method.
exe = executable('unit_test', 'test.c')
test('basic', exe, workdir : meson.source_root())
I am maintaining an autoconf package and wanted to integrate automatic testing. I use the Boost Unit Test Framework for my unit tests and was able to sucessfully integrate it into the package.
That is it can be compiled via make check, but is is not run (although I read that make check both compiles and runs the tests). As result, I have to run it manually after building the tests which is cumbersome.
Makefile.am in the test folder looks like this:
check_PROGRAMS = prog_test
prog_test_SOURCES = test_main.cpp ../src/class1.cpp class1_test.cpp class2.cpp ../src/class2_test.cpp ../src/class3.cpp ../src/class4.cpp
prog_test_LDADD = $(BOOST_FILESYSTEM_LIB) $(BOOST_SYSTEM_LIB) $(BOOST_UNIT_TEST_FRAMEWORK_LIB)
Makefile.am in the root folder:
SUBDIRS = src test
dist_doc_DATA = README
ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS} -I m4
Running test/prog yields the output:
Running 4 test cases...
*** No errors detected
(I don't think you need the contents of my test cases in order to answer my question, so I omitted them for now)
So how can I make automake run my tests every time I run make check?
At least one way of doing this involves setting TESTS variable. Here's what documentation on automake says about it:
If the special variable TESTS is defined, its value is taken to be a list of programs or scripts to run in order to do the testing.
So adding the line
TESTS = $(check_PROGRAMS)
should instruct it to run the tests on make check.
I'm trying to use test support classes within my tests. I want these classes to be available for all different test types.
My directory structure is as follows;
/test/functional
/test/integration
/test/unit
/test/support
I have test helper classes within the /test/support folder that I would like to be available to each of the different test types.
I'm using GGTS and I've added the support folder to the classpath. But whenever I run my integration tests running 'test-app' I get a compiler 'unable to resolve class mypackage.support.MyClass
When I run my unit tests from within GGTS the support classes are found and used. I presume this is because the integration tests run my app in its own JVM.
Is there any way of telling grails to include my support package when running any of my tests?
I don't want my test support classes to be in my application source folders.
The reason that it works for your unit tests inside the IDE is that all source folders get compiled into one directory, and that is added to your classpath along with the jars GGTS picks up from the project dependencies. This is convenient but misleading, because it doesn't take into account that Grails uses different classpaths for run-app and each of the test phases, which you see when you run the integration tests. GGTS doesn't really run the tests; it runs the same grails test-app process that you do from the commandline, and captures its output and listens for build events so it can update its JUnit view.
It's possible to add extra jar files to the classpath for tests because you can hook into an Ant event and add it to the classpath before the tests start. But the compilation process is a lot more involved and it looks like it would be rather ugly/hackish to get it working, and would likely be brittle and stop working in the future when the Grails implementation changes.
Here are some specifics about why it'd be non-trivial. I was hoping that you could call GrailsProjectTestCompiler.compileTests() for your extra directory, but you need to compile it along with the test/unit directory for unit tests and the test/integration directory for integration tests, and the compiler (GrailsProjectTestCompiler) presumes that each test phase only needs to compile that one directory. That compiler uses Gant, and each test phase has its own Grailsc subclass (org.grails.test.compiler.GrailsTestCompiler and org.grails.test.compiler.GrailsIntegrationTestCompiler) registered as taskdefs. So it should be possible to subclass them and add logic to compile both the standard directory and the shared directory, and register those as replacements, but that requires also subclassing and reworking GrailsProjectTestRunner (which instantiates GrailsProjectTestCompiler), and hooking into an event to replace the projectTestRunner field in _GrailsTest.groovy with your custom one, and at this point my brain hurts and I don't want to think about this anymore :)
So instead of all this, I'd put the code in src/groovy and src/java, but in test-specific packages that make it easy to exclude the compiled classes from your WAR files. You can do that with a grails.war.resources closure in BuildConfig.groovy, e.g.
grails.war.resources = { stagingDir ->
println '\nDeleting test classes\n'
delete(verbose: true) {
// adjust as needed to only delete test-specific classes
fileset dir: stagingDir, includes: '**/test/**/*.class'
}
println '\nFinished deleting test classes\n'
}