Currently I am implementing client side unit testing using karma jasmine for a big project I've been working on.
I'm looking for a best practice to structure our testing code in a Visual Studio solution. The project has over 50 different js files that should all have their own unit test script. Also the directory structure of the code goes up to 8 directories deep. I've thought about a couple of different approaches to structure the unit tests files.
Unit test files in the same directory as the code files that should be tested
Unit test files in a subdirectory "tests" in each of the directories with code
A directory called "tests" in the root of the project with the same directory structure as the actual code
A new project with a directory with the same directory structure as the actual code
What is a good way to structure my client side unit tests files in a big project and still keep it maintainable and "clean"?
Definitely keep the tests close to the code files. It's easy to forget to also change a paralell directory structure when doing refactoring, and even if you remember, it is tedious and error-prone to have to do the same change twice. Don't Repeat Yourself.
I would put each test in a subdirectory tests/ below each code file, just like you would do it in a smaller project. That way it will be easy for someone new picking up the project to find the tests, since it is a common layout. Most test runners should be able to run all tests in a directory structure, so it doesn't matter that they are spread out.
Related
I would like to use Boost for unit testing. Writing the unit tests is not a problem. However, I am a little unsure how to structure the makefiles relating to each unit test. I am not an expert on makefiles so I would like to explain how my code base is, what I would like to do and then ask for the best solution.
I have a code base which is a mixture of Python, C++ utility functions in headers and one C++ application/library, but it uses the classes/functions located in other folders (which are not sub folders). So unsurprisingly I only have one makefile for my application.
I would like to add unit tests in each of the various folders my application uses.
I would also like to be able to run all the unit tests (across the multiple folders) by running one executable.
Do I write a boost unit test source file in each folder and add a corresponding makefile, one per each folder I wish to unit test? How do I structure this so that I can run all the unit tests from one executable?
Also, do I need to link my unit test makefiles to my application object file, or just gcc -I flag the headers it requires?
(I am using regular GNU Make)
I'm currently working on a project that uses autotools to generate a makefile. As usual, this makefile supports make check, to run the entire test suite. However, I'm working on a small part of the program, where only a few unit tests apply. Is it possible, using make check, to run a single unit test or a selection of unit tests, instead of the entire suite?
Edit:
Evan VanderZee suggests that it depends on the directory structure, so let me explain a bit more about the directory structure. The directory structure for this project is pretty flat. Most source files are in the src/ directory, with some grouped in a subdirectory under src/. The unit tests are in src/tests. None of the source directories contain a Makefile. There is only a makefile (generated by configure) at the top level.
If you are using Automake's TESTS variable to list the tests that make check runs, then there is an easy shortcut: make check TESTS='name-of-test1 name-of-test2' The TESTS variable passed on the command line overrides the one in the Makefile.
Alternatively, export TESTS='names-of-tests'; make -e check to take the value of TESTS from the environment.
If you are not using TESTS but instead check-local or some other target, then this won't work.
The answer to that depends on the specific makefiles you are using. If the tests have a directory structure, you can often navigate to a subdirectory and run make check in the subdirectory.
Since you explained later that your directory structure is flat, it is hard to say without actually seeing some of the makefile. If you know the name of the executable that is created for the specific test you want to run, you might be able to run make name_of_test to build the test that you want to run. This will not run the test, it will only build it. The test may reside in the test directory after it is built. After this you can go into the test directory and run the test in the typical way you would run an executable, but if the test depends on libraries, you may need to tell the test where to find those libraries, perhaps by adding some libraries to the LD_LIBRARY_PATH.
If this is something you would want to do often, the makefile can probably be modified to support running the specific tests that you want to run. Typically this would involve editing a Makefile.am or Makefile.in and then reconfiguring, but I don't have enough information yet to advise what edits you would need to make.
I have a big code base in TFS which has multiple .sln files, each with many projects and at least one unit test project. Most of the unit tests rely on common XML and XSD files, and there are several other types of files (.config, .xaml, etc) that are needed by the code when unit testing.
Because of the way that TFS builds and gathers the files for unit testing, most of those files are missing from the TestResults folder, so the tests are failing during our CI builds [this has been happening for a while, but I'm new to the project, and am trying to fix the errors]. What TFS appears to do is this: First, it checks out all the code to a src folder (with Solution1, Solution2, etc) and builds it, just like the developers do locally. Second, it copies the build outputs to a bin\Binaries folder. Third, it looks for all the test.dll files, copies them and their dependencies (but only the dependencies), plus the App.config file to TestResults\Deploy_[date/time]\Out folder, and runs the unit tests there.
I am encountering two problems with this. Because the second step is combining all the build outputs into one folder, all the files with duplicate names are overwriting each other. So, there is only one App.config file, even though each solution has its own. This is happening with other config/xml files too, and with two poorly named unit test .dlls. I can live with this if I have to because most of those config files are duplicates, and other files can be renamed.
The second problem is that most of those extra files don't make it into the TestResults folder, and when they aren't there the unit tests will fail. I know about using the [DeploymentItem] attribute; if that is the only solution, I will go that route, but there are so many extra files that I am looking for a different approach.
So my question is, how can I configure my tfs-run builds & unit tests to include all of the files that they need, without all the work & maintenance problems of adding a lot of [DeploymentItem] attributes, and also without affecting the local builds & unit tests for the developers?
Update
One thing I've found is that adding a [DeploymentItem] attribute to a unit test actually causes the deployment folder to be used. Without that, it runs the unit tests in the binaries folder. See http://msdn.microsoft.com/en-us/library/ms182475.aspx, under "When is a separate deployment folder used?"
Also on that page, it says you can specify files to deploy in a .testsettings file, but then says you should avoid using it because your tests will run slower. The newer alternative, a .runsettings file, does not let you list what files to deploy.
It also appears that deployment becomes enabled if code coverage is enabled, which we don't currently do, but plan to once the tests are passing.
In TFS 2013 you can execute a powershell pre-test to organize the files in a way that you need. You can get files to be deployed ad part of the tests with a test settings file for pre2013.
If however the test setting file is not enough you can use the Community Build tools to call a powershell directly in previous versions to 2013.
If you are stuck on 2012 then you will need to use the .testsettings file to push the bits you need. Yes it will make your build slower but that's your only choice other than customising the build process as above.
Here is an example of my source control structure in TFS 2010:
TFS Project
Development
Branches
Source
Tests
Under my Source directory I have created a directory for my Data Access Layer. The solution is currently not dependent on any other project outside its own solution. I did this so I could branch just this code. I wanted to avoid having to branch the entire Source folder.
I personally like having my unit tests as part of a solution so I can easily create and run tests against the code without having to go into another solution.
If I put my unit test projects in the same solution as my DAL solution, what is the best way to branch my entire DAL without the unit test project dependencies? Should I create another solution without unit Tests for branching? Should I include the unit tests in the branch after I branch the solution?
Since the Tests are located in the Tests folder, there isn't a way to selectively branch folders in TFS.
How do other users approach this?
The unit tests also help to act as documentation for the code and examples of how it behaves and how to use it.
Therefore, if you're branching to change the code's behavior, I recommend branching the unit tests along with the code. That way your documentation and examples are always up to date.
I've just started working on a Java project (as I usually use .net), and one of the first things that strikes me as odd is that in the Maven project there is a /src and a /test directory where obviously the source code and the tests should go.
In .net I preferred to have the tests in a separate assembly/project, so for example I would have:
MyProject
MyProject.Tests
That way I dont have to bloat my deployed code with any tests and it makes it easier to test my code in true isolation and in a lot of cases I didn't bother writing tests per project, I would just have solution wide unit/integration/acceptance tests i.e MySolution.UnitTests, MySolution.IntegrationTests.
However in Java it just seems to be bundled together, and I would rather separate it out, however I hear that Maven is a cruel mistress when you want to do things differently to the default structures.
So to reign this post back in, my main questions are:
Is there a way to separate out the tests from the project
Based on the above information are there any pros for actually having the tests within the project? (other than when you check it out you always have the tests there)
I dont have to bloat my deployed code with any tests
The deployable artifacts (jar file, war files) will not contain the test classes or data.
Is there a way to separate out the tests from the project
You could split it into two projects, with a "Test" project containing only the tests, and depending on the "real" project.
However, especially with Maven, you probably want to follow the same project layout conventions that everyone else (or at least the majority) has. This will make your life easier (less configuration).
Again, the tests will not make it into the product to be deployed, so the current layout should not be a problem.
I would just have solution wide unit/integration/acceptance tests i.e MySolution.UnitTests, MySolution.IntegrationTests.
For integration tests, that actually makes sense. In this case, define a "Test" project that depends on all the other projects that make up the solution (I'd still keep the unit tests with the project they test).
In a default maven setup, the tests are only executed, but not deployed.
Per convention, everything inside src/main lands in the target archive, while everything else doesn't.
Per default, a maven JAR project creates a jar with just the classes that are compiled from src/main/java. You can use different plugin goals to create:
test jar (jar of compiled test classes)
source jar (jar of main sources)
test source jar (jar of test sources)
javadoc jar (jar of javadoc api documentation)
But all of these require extra steps.