Here is an example of my source control structure in TFS 2010:
TFS Project
Development
Branches
Source
Tests
Under my Source directory I have created a directory for my Data Access Layer. The solution is currently not dependent on any other project outside its own solution. I did this so I could branch just this code. I wanted to avoid having to branch the entire Source folder.
I personally like having my unit tests as part of a solution so I can easily create and run tests against the code without having to go into another solution.
If I put my unit test projects in the same solution as my DAL solution, what is the best way to branch my entire DAL without the unit test project dependencies? Should I create another solution without unit Tests for branching? Should I include the unit tests in the branch after I branch the solution?
Since the Tests are located in the Tests folder, there isn't a way to selectively branch folders in TFS.
How do other users approach this?
The unit tests also help to act as documentation for the code and examples of how it behaves and how to use it.
Therefore, if you're branching to change the code's behavior, I recommend branching the unit tests along with the code. That way your documentation and examples are always up to date.
Related
Currently I am implementing client side unit testing using karma jasmine for a big project I've been working on.
I'm looking for a best practice to structure our testing code in a Visual Studio solution. The project has over 50 different js files that should all have their own unit test script. Also the directory structure of the code goes up to 8 directories deep. I've thought about a couple of different approaches to structure the unit tests files.
Unit test files in the same directory as the code files that should be tested
Unit test files in a subdirectory "tests" in each of the directories with code
A directory called "tests" in the root of the project with the same directory structure as the actual code
A new project with a directory with the same directory structure as the actual code
What is a good way to structure my client side unit tests files in a big project and still keep it maintainable and "clean"?
Definitely keep the tests close to the code files. It's easy to forget to also change a paralell directory structure when doing refactoring, and even if you remember, it is tedious and error-prone to have to do the same change twice. Don't Repeat Yourself.
I would put each test in a subdirectory tests/ below each code file, just like you would do it in a smaller project. That way it will be easy for someone new picking up the project to find the tests, since it is a common layout. Most test runners should be able to run all tests in a directory structure, so it doesn't matter that they are spread out.
I have a big code base in TFS which has multiple .sln files, each with many projects and at least one unit test project. Most of the unit tests rely on common XML and XSD files, and there are several other types of files (.config, .xaml, etc) that are needed by the code when unit testing.
Because of the way that TFS builds and gathers the files for unit testing, most of those files are missing from the TestResults folder, so the tests are failing during our CI builds [this has been happening for a while, but I'm new to the project, and am trying to fix the errors]. What TFS appears to do is this: First, it checks out all the code to a src folder (with Solution1, Solution2, etc) and builds it, just like the developers do locally. Second, it copies the build outputs to a bin\Binaries folder. Third, it looks for all the test.dll files, copies them and their dependencies (but only the dependencies), plus the App.config file to TestResults\Deploy_[date/time]\Out folder, and runs the unit tests there.
I am encountering two problems with this. Because the second step is combining all the build outputs into one folder, all the files with duplicate names are overwriting each other. So, there is only one App.config file, even though each solution has its own. This is happening with other config/xml files too, and with two poorly named unit test .dlls. I can live with this if I have to because most of those config files are duplicates, and other files can be renamed.
The second problem is that most of those extra files don't make it into the TestResults folder, and when they aren't there the unit tests will fail. I know about using the [DeploymentItem] attribute; if that is the only solution, I will go that route, but there are so many extra files that I am looking for a different approach.
So my question is, how can I configure my tfs-run builds & unit tests to include all of the files that they need, without all the work & maintenance problems of adding a lot of [DeploymentItem] attributes, and also without affecting the local builds & unit tests for the developers?
Update
One thing I've found is that adding a [DeploymentItem] attribute to a unit test actually causes the deployment folder to be used. Without that, it runs the unit tests in the binaries folder. See http://msdn.microsoft.com/en-us/library/ms182475.aspx, under "When is a separate deployment folder used?"
Also on that page, it says you can specify files to deploy in a .testsettings file, but then says you should avoid using it because your tests will run slower. The newer alternative, a .runsettings file, does not let you list what files to deploy.
It also appears that deployment becomes enabled if code coverage is enabled, which we don't currently do, but plan to once the tests are passing.
In TFS 2013 you can execute a powershell pre-test to organize the files in a way that you need. You can get files to be deployed ad part of the tests with a test settings file for pre2013.
If however the test setting file is not enough you can use the Community Build tools to call a powershell directly in previous versions to 2013.
If you are stuck on 2012 then you will need to use the .testsettings file to push the bits you need. Yes it will make your build slower but that's your only choice other than customising the build process as above.
I've just started working on a Java project (as I usually use .net), and one of the first things that strikes me as odd is that in the Maven project there is a /src and a /test directory where obviously the source code and the tests should go.
In .net I preferred to have the tests in a separate assembly/project, so for example I would have:
MyProject
MyProject.Tests
That way I dont have to bloat my deployed code with any tests and it makes it easier to test my code in true isolation and in a lot of cases I didn't bother writing tests per project, I would just have solution wide unit/integration/acceptance tests i.e MySolution.UnitTests, MySolution.IntegrationTests.
However in Java it just seems to be bundled together, and I would rather separate it out, however I hear that Maven is a cruel mistress when you want to do things differently to the default structures.
So to reign this post back in, my main questions are:
Is there a way to separate out the tests from the project
Based on the above information are there any pros for actually having the tests within the project? (other than when you check it out you always have the tests there)
I dont have to bloat my deployed code with any tests
The deployable artifacts (jar file, war files) will not contain the test classes or data.
Is there a way to separate out the tests from the project
You could split it into two projects, with a "Test" project containing only the tests, and depending on the "real" project.
However, especially with Maven, you probably want to follow the same project layout conventions that everyone else (or at least the majority) has. This will make your life easier (less configuration).
Again, the tests will not make it into the product to be deployed, so the current layout should not be a problem.
I would just have solution wide unit/integration/acceptance tests i.e MySolution.UnitTests, MySolution.IntegrationTests.
For integration tests, that actually makes sense. In this case, define a "Test" project that depends on all the other projects that make up the solution (I'd still keep the unit tests with the project they test).
In a default maven setup, the tests are only executed, but not deployed.
Per convention, everything inside src/main lands in the target archive, while everything else doesn't.
Per default, a maven JAR project creates a jar with just the classes that are compiled from src/main/java. You can use different plugin goals to create:
test jar (jar of compiled test classes)
source jar (jar of main sources)
test source jar (jar of test sources)
javadoc jar (jar of javadoc api documentation)
But all of these require extra steps.
Long time viewer, first time questioner here. Using Unit Tests in a separate project, I can't figure out how to separate them at the release build?
Surely when I remove the reference to the Unit Test project, all the references to the Unit Test interfaces, won't be able to find the interfaces and will cause compiler errors?
Thanks for any help.
Edit: By separating I mean removing them on the release build.
Assuming .NET, your unit test project should have a reference to the production project - but not the other way round. Your production code shouldn't depend on your unit tests at all.
You should have built your two projects, so that the base code (That in which you are testing) has no dependencies on the unit test. It is the Unit Test that has the dependencies on your code.
From what you wrote, I gather you wrote your code in .NET? (i.e. Assemblies?). Your project dependencies should look like this.
// Business code
My_Project.dll
-> References
---> System.dll etc..
// Test code
My_Project_Test.dll
-> References
---> My_Project.dll
If you make your projects that way, than I think you won't have to do anything special in your release build configuration.
What are the steps to get Team Foundation Server running unit tests when a given build runs?
What are the caveats / pitfalls / workarounds a dev or sysadmin should be aware of when setting up a TFS server to do this for the first time?
What are common troubleshooting steps for unit test problems during builds?
it depends on which version of TFS you are running, so I will assume it is 2008.
Firstly, you must have Team Edition for Testers installed on the computer that will act as your build agent, as stated in How To: Create a Build Definition
There are a couple of ways to tell Team Build to run tests for your build.
Unit tests can be run from a defined Test List within the Solution being built. This list is referenced by the build definition and all tests within the chosen list(s) are executed. More info here
WildCard test exectution is also available by defining a wildcard mask (i.e. Test*.dll) that instructs Team Build to run any tests present in assemblies that match the mask. This is done when defining the build definition as well.
Things to note:
If you intend to use the wildcard method and want to enable code coverage for your test configuration, you must add the following to your build definition file to enable it.
<RunConfigFile>$(SolutionRoot)\TestRunConfig.testrunconfig</RunConfigFile>
See my previous question on this for more info here
If you don't want to use test configs (A Pain in the ass to manage) just run all the test in a .dll by adding this to the build config:
<ItemGroup>
<TestContainerInOutput Include="MyProject.UnitTests.dll" />
</ItemGroup>
The whole process is smooth and fairly simple. You can inspect the unit tests that filaed on the build server by opening the test result file locally (a bit of a pain) but generally you'll just run the unit tests locally and you can see the results immediately.
If you are used to NUnit, you may opt to sort the tests by classname, it gives a similar view.
Careful with code coverage, it makes complete copies of your binaries on compile. If your binaries are sufficiently large and you compile often, it will eat through drive space quickly.
http://msdn.microsoft.com/en-us/library/cc981972(v=vs.90).aspx
I like this defination as it gives you a complete 'walkthrough' from
Creating the Project
Creating the Unit Test Project
To configuring Team build to use it Unit Test