I have 6 unit tests projects in my solution, what will be execution order of unit tests projects? The reason why I am asking this is because I have 1 integration test project and 5 unit test project and in our ci/cd pipeline I want to execute unit test projects first and then integration tests. so we want to fail pipeline if the any of the unit test projects are failed. So how can I specify the order i.e. to execute 5 unit tests project first and then integration test project.
Does unit tests projects are run in parallel? Meaning will the 5 unit tests projects of the will execute in parallel?
Is there a way to execute unit tests of a class by attribute based ordering in MsTest framework? I know MsTest have alphabetical ordering but some people do not like it.
Related
I have a solution, with three projects
Solution
-Project 1
-Project 2
-Project 3
Recently wanted to implement a unit test (or integration test) to make sure everything still works, and I have experience with CppUTest, but wanted to see if anything VS has to offer might be "better" integrated for this purpose.
So I followed a guide and it led me to another project.
-Unit test project (aka project 4), which is only a class from the looks of it.
Solution
-Project 1
-Project 2
-Project 3
-Project 4 (Unit test, testing Project 1)
So I'm wondering what's a good practice to maintain unit/integration testing environment?
It could be...
One unit test project, multiple unit tests classes, one for each project
Solution
-Project 1
-Project 2
-Project 3
-Project 4 (Unit test, testing Project 1, 2 and 3)
A Unit test project per working project
One unit test project, multiple unit tests classes, one for each project
Solution
-Project 1
-Project 2
-Project 3
-Project 4 (Unit test, testing Project 1)
-Project 5 (Unit test, testing Project 2)
-Project 6 (Unit test, testing Project 3)
Just apply unit test class to existing projects (not sure if it's possible)
Solution
-Project 1
-Code
-Unit Test
-Project 2
-Code
-Unit Test
-Project 3
-Code
-Unit Test
Is the third option possible?
Contrary to the accepted answer, I'd say to definitely put your test code into a separate project. Whether you create a single or multiple test projects is a matter of opinion, but there are good reasons to not put your test code into your release code.
The main reason is that you don't want to link unit test libraries into release code. That's simply unacceptable in my book and as far as I know Visual Studio has a clear "1 project = 1 library" approach so I don't see how you'd avoid that.
Using one unit test library per project means you can develop projects independent of others and you can easily specific which tests you want to run instead of having to run the tests for the whole solution. You could even set up continuous testing to get feedback much quicker about the code while you're writing (actually not sure if anybody has that working for C++, it's quite common for managed languages at least).
Also if you're using Microsoft's unit test framework you can just use the template they provide which also forces you to create a separate project for your tests.
In my solution I have c++ and c# projects with corresponding Unit test projects mixed. During a TFS build I only want to execute the C# unit tests. Unfortunately I can't find a way to exclude these test assemblies to be being used in the unit test runner. I could identify all c++ unit tests projects based on a naming patter like Native.Tests.dll.
I can't find a way to explicitly exclude certain test assemblies from being tested/executed from the test runner.
Is there a way through either
Test Case Filter
Test assembly file specification?
Environment:
TFS 2013
Process template: ReleaseTfvcTemplate.12.xaml
In case you are wondering: Why do I want to exclude certain assemblies?
I want to use the Test Category feature to exclude certain unit tests from being executed on the build server, which you do through the TestCaseFilter feature in the TFS template. You specify that per "Test batch". When I run all my unit tests (c# + c++) in one test settings the native unit tests throw an error, because they don't understand/support the TestCategory-filter feature (remember: Test Case Filter). Therefore I want to split the it in 2 test runs/batches: c++ and c#.
Test Category is only worked in C# test project. (Test in my environment: TFS2015 VS2015)
If you only want to execute the C# unit test. The simplest way you can specify the C# test assembly file in Test assembly file specification.
You just need to identify all C# unit tests projects based on a naming patter totally different as C++ tests projects.
Note: You must make sure you are using totally different naming rules between C++ an C# test project.
For example:
C++test project name : including ABC like ABC1 ,ABC2, ABC3
C# test project name: including XYZ like XYZ1, XYZ2 ,XYZ3
Then using ***XYZ*.dll in Test assembly file specification to run the all C# test project.
I have a project that was version 0.2.7 and generating component tests as unit tests. My new projects create integration tests by default. Is there a way to change it over?
This is the default behavior now and as far as I know, can't be overwritten, as components should be run in isolation. However, you can simply generate a unit test with:
ember g component-test your-component -unit
If you want to read up on why you should try integration tests, I found this article helpful:
http://alisdair.mcdiarmid.org/ember-component-integration-tests/
As an additional note, we manage our project builds with Maven.
Let's say we have three GWT modules, only one of which has an entry point. The other two modules are inherited by the module containing the entry point.
We practice MVP and strive to keep the majority of our testing as normal JUnit tests, but some integration tests require us to use GWTTest. We keep these to a minimum.
We would like to keep the actual tests with the modules containing the code they are testing. My thought was to ensure that maven only executes JUnit tests for the sub-module builds. The GWTTests in the submodules would be rolled up into a test suite. Then, the entry point project would contain a GWTTestSuite which rolls up the submodule test suites along with and other GWTTests that the entry point module may have.
At this point, my problem is that the entry point GWTTestSuite can't resolve the tests from the submodules ("no source code is available........did you forget to inherit a required module"). I've tried creating the test-jars for the submodules and adding them as dependencies to the test scope, but when GWT is executing the primary GWTTestSuite, it can't find the submodule test suites (or their underlying gwt tests). I can see the GWTTests in the .m2 repo test jars.
So, how do I get the GWT compiler to "see" the submodule tests?
Update:
I have not yet been able to figure out how to do this. The cleanest thing I can manage is to keep the normal JUnit tests co-located with the submodules, but keep all GWT Tests for all modules in the main module.
I'm using TFS 2010, VS2010 and C#. I need to link each requirement to the unit test or tests associated with that requirement. I've found that in order to link Tests to requirements, TFS requires a Test Case for each [TestMethod] unit test. Only then can I link the Requirement to the Test Case which in turn is associated with the automated unit test. I'm looking for a way to shortcut this so that Test Cases can be created automatically for all exiting Tests. Ideally the TFS Test Case description would come from the [TestMethod] header comments.
If you just want to create a couple of test cases, you can do it directly from Visual Studio by opening the Test Window, select your test, right-click and choose "Create Test Case From Test". You must have your test projects opened in order to see your automated tests in the Test Window.
http://msdn.microsoft.com/en-us/library/dd380741.aspx#NewTestCase
If you want to import a bunch of automated tests from a test assembly, you can use the command-line tool for Test Management, called tcm.exe (you can find it under Common7\IDE\ of your Visual Studio location). The following command will do the trick:
tcm testcase /import /storage:[test assembly] /collection:[url] /teamproject:[your team project]
You can also specify the categories or the priorities of tests you want to import. For more information, see:
http://msdn.microsoft.com/en-us/library/dd465191.aspx#CreateTestCases
Hope this helps.