I'd like to be able to run all the tests under a folder structure like below. Anything under "tests" I would like to run. Is this possible?
We have a bunch of other tests in our project that haven't been maintained. We're migrating tests into this new tests folder.
Would a test category work?
Since VS2012 SP1, there are new test filters. You can use them to run your tests.
Of course, you could also use a Category for this : you just need to tag every test class.
Related
I use Intellij(ultra version) IDE to develop Grails application.
If I have a controller named UserController and then use Intellij to create both unit and integration tests for this.
Intellij will create two files have same name but in different folder.
e.g /unit/xxx/yyy/UserControllerSpec
/integration/xxx/yyy/UserControllerSpec
However, Intellij IDE has some error alert for file already exists. I know it is acceptable and I run the tests pass. Just the annoying error.
Anyone knows how to disable this Intellij annoying error?
You shouldn't have two classes with the same name in the same package even if they're in different folders. Rename one, e.g. UserControllerIntegrationSpec or UserControllerUnitSpec.
I want to run all the ignored tests in my solution in a different build configuration in TeamCity.
I didn't see any explicit why to do it in the build step configuration page.
Can it be done?
The Ignore attribute is the wrong thing to use here. It should be used for tests you do not wish to run at all. Try using Categories instead. TeamCity has two properties you can set within an NUnit build step.
NUnit categories include and NUnit categories exclude control which tests will be run.
How did you mark your tests - your question is not quite clear in this point? Did you use the Explicit or Ignored attribute? If the latter then these tests will not run at all.
I have just created a small project with AppHarbor, composed of a single .csproj that uses xUnit. It's using NuGet to recover the xUnit framework. AppHarbor recognized it was a test project and executed the tests that were present.
I want to know how AppHarbor select which projects to unit test. Does it look at the referenced assemblies of the dll/exe (perhaps through reflection)? (for example if the dll/exe references xunit.dll then it's probably a test project). Does it look at the .csproj file looking for the references? Does it run all the assemblies through all the runners?
Appharbor uses Galio http://www.gallio.org/ I'm pretty sure its going to run everything that is marked with [Fact]
I have a maven build process that publishes executable jars and their tests to Nexus.
I have another maven build process that needs to access these jars (executable + test) and run the tests.
How do I go about it? So far I have managed to do this only if the jar is exploded to class files.
I am new to maven and completely lost in the documentation.
Update 2022-03-11
The feature has been implemented, see https://stackoverflow.com/a/17061755/1589700 for details
Original answer
Surefire and failsafe do not currently support running tests from within a jar.
This is largely a case of not being able to identify the tests.
There are two ways to get the tests to run.
Use a test suite that lists all the tests from the test-jar. Because the test suite will be in src/test/java (more correctly will be compiled into target/test-classes) that will be picked up and all the tests in the suite will be run by Surefire/failsafe (assuming the suite class name matches the includes rule: starts or ends with Test)
Use the maven dependency plugin's unpack-dependencies goal to unpack the test-jar into target/test-classes (this screams of hack, but works quite well)
The main issue with the first option is that you cannot easily run just one test from the suite, and you need to name every test from the test-jar
For that reason I tend to favour option 2... There is the added benefit that option 2 does not mean writing code to work around a limitation in a build tool plugin... The less you lock yourself into a specific build tool, the better IMHO
This actually works quite fine with the newer surefire and failsafe plugins, see related questions:
Run JUnit Tests contained in dependency jar using Maven Surefire
run maven tests from classpath
So you don't need to unpack the jar anymore, you just provide the group and artifact id for the dependencies to scan (this works with both "main jar" dependencies, as well as "test-jar" dependencies)
The attached test-jar can be used as a usual dependency in other project which supports reuse of code in the test area but you can't run tests out of the jar. If you really need the solution you have to write at least a single suite (etc.?) to start the tests from the jar.
What are the steps to get Team Foundation Server running unit tests when a given build runs?
What are the caveats / pitfalls / workarounds a dev or sysadmin should be aware of when setting up a TFS server to do this for the first time?
What are common troubleshooting steps for unit test problems during builds?
it depends on which version of TFS you are running, so I will assume it is 2008.
Firstly, you must have Team Edition for Testers installed on the computer that will act as your build agent, as stated in How To: Create a Build Definition
There are a couple of ways to tell Team Build to run tests for your build.
Unit tests can be run from a defined Test List within the Solution being built. This list is referenced by the build definition and all tests within the chosen list(s) are executed. More info here
WildCard test exectution is also available by defining a wildcard mask (i.e. Test*.dll) that instructs Team Build to run any tests present in assemblies that match the mask. This is done when defining the build definition as well.
Things to note:
If you intend to use the wildcard method and want to enable code coverage for your test configuration, you must add the following to your build definition file to enable it.
<RunConfigFile>$(SolutionRoot)\TestRunConfig.testrunconfig</RunConfigFile>
See my previous question on this for more info here
If you don't want to use test configs (A Pain in the ass to manage) just run all the test in a .dll by adding this to the build config:
<ItemGroup>
<TestContainerInOutput Include="MyProject.UnitTests.dll" />
</ItemGroup>
The whole process is smooth and fairly simple. You can inspect the unit tests that filaed on the build server by opening the test result file locally (a bit of a pain) but generally you'll just run the unit tests locally and you can see the results immediately.
If you are used to NUnit, you may opt to sort the tests by classname, it gives a similar view.
Careful with code coverage, it makes complete copies of your binaries on compile. If your binaries are sufficiently large and you compile often, it will eat through drive space quickly.
http://msdn.microsoft.com/en-us/library/cc981972(v=vs.90).aspx
I like this defination as it gives you a complete 'walkthrough' from
Creating the Project
Creating the Unit Test Project
To configuring Team build to use it Unit Test