Unit tests aren't run with multiple build configurations in TFS 2010 - unit-testing

I am using MSBuild in my TFS Buildprocess using the DefaultTemplate to build multiple configurations(Debug/Release) of the same solution. When I build only release or only debug the unit tests are run but when I run the build with both configurations the unit tests are not run.
When I am running multiple configurations I have each of them Defined in the build definition under Process-> Items to Build-> Configurations to Build which is Mixed Platforms|Release,Mixed Platforms|Debug. When I run a single configuration I am using Mixed Platforms|Release or Mixed Platforms|Debug.
I am unable to put the entire log here(It is over 6000 lines long) but it appears that it isnt finding any of the testAssemblies.
Edit:
Here are the actual sections of the log. I'm not sure where Mixed Platforms\Release is coming from in the searchpathroot.
I should also add that we compile into our Source's folder because a few of our scripts only work in that layout.
Debug Only(works)
Run MSTest for Test Assemblies
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"C:\Builds\6\ProductName\BuildName\Sources\Product\Local.testsettings" /searchpathroot:"C:\Builds\6\ProductName\BuildName\Sources" /resultsfileroot:"C:\Builds\6\ProductName\BuildName\TestResults" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestCommon\bin\Debug\UnitTestCommon.dll" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestCommon\bin\Release\UnitTestCommon.dll" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestWCF\bin\Debug\UnitTestWCF.dll" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestWCF\bin\Release\UnitTestWCF.dll" /publish:"http://dctfs2010.company.dc:8080/tfs/Product" /publishbuild:"vstfs:///Build/Build/964" /teamproject:"ProductName" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\6\ProductName\BuildName\Sources\Product\Local.testsettings...
Loading C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestCommon\bin\Debug\UnitTestCommon.dll...
Loading C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestCommon\bin\Release\UnitTestCommon.dll...
Debug and Release(Does not work)
Run MSTest for Test Assemblies
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"C:\Builds\6\ProductName\BuildName\Sources\Product\Local.testsettings" /searchpathroot:"C:\Builds\6\ProductName\BuildName\Sources\Mixed Platforms\Release" /resultsfileroot:"C:\Builds\6\ProductName\BuildName\TestResults" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestCommon\bin\Debug\UnitTestCommon.dll" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestCommon\bin\Release\UnitTestCommon.dll" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestWCF\bin\Debug\UnitTestWCF.dll" /testcontainer:"C:\Builds\6\ProductName\BuildName\Sources\Test\UnitTestWCF\bin\Release\UnitTestWCF.dll" /publish:"http://dctfs2010.company.dc:8080/tfs/Product" /publishbuild:"vstfs:///Build/Build/962" /teamproject:"ProductName" /platform:"Mixed Platforms" /flavor:"Release"
Directory "C:\Builds\6\ProductName\BuildName\Sources\Mixed Platforms\Release" not found.

If you click on the properties of your solution (the solution that you build as part of your build). You will have configuration manager. At the configuration manager you will be able to see 'Debug', 'Released', 'Mixed', etc any other configurations you may have defined. Can you see the test project checked to be build in the mixed configuration? If this is not being build, the test.dll will not be available which means, MSBuild will not have any tests to run.
Another way to verify why the tests are not running, is to look at the build drop directory and see if you can see the test.dll there. If the test.dll is not there then the problem is with how you have the projects configured in the configuration manager.

Turns out my outputDirectory was setup wrong for how our projects are compiled. Which is where the searchpathroot problem came from.

Related

Visual Studio 2019 C++ Reference Project Not Built Before Dependent Project Tries To Link in Release Configuration using MSBuild

I have a solution that contains both C++ and C# projects that is built in a nightly CI build on a remote machine. The build script checks out a clean copy of the source and builds both debug and release configurations of the solution using MSBuild and runs the test suite on each configuration.
About every other build, the release configuration fails to build properly. An analysis of the build log reveals that C++ Project Q, which depends on C++ Project D, tries to link before Project D is done. This error only happens for the release configuration on this particular build machine - the debug configuration builds without error. I have a separate nightly build process that runs on a separate machine where the release configuration is built with a similar script that uses MSBuild (it just does not run the test suite), and it builds the same source revision without issue. Multiple team members build the solution without issue either from update or clean checkout with one or both configurations, always from the Visual Studio 2019 IDE, on various operating systems.
Project Q is configured with Project D as a project reference and Project D is also listed as a hard-dependency for Project Q. As I mentioned, the build script is using MSBuild.
An additional item of interest from analysis of the release and debug build logs: The build of Project D is initiated differently between the two configuration builds. It is started by its own metaproject in the release configuration (as item 60, for instance), but started earlier (as item 44, for instance), by a different project in the debug configuration. Not sure why the dependency algorithm would run with such different results in the two cases since the solution and working source being built are the same.
Any ideas or suggestions would be appreciated.
Update: Inspection of differences between the release and debug build logs reveals some interesting facts. In the failure case I did a search for ") is building" in each log - should be an indication of how many projects were built, including metaproj entries. For the failure case, debug had 282 occurrences, whereas release had 175. For the success case, debug had 280 occurrences and release had a whopping 559! A similar search for "Done Building Project" yields similar results, only off by 1 or 2. That might partly explain the differences in build order between the solutions. I also need to check for conditional build entries.
Visual Studio 2019 Reference Project Not Built Before Dependent
Project Tries To Link in Release Configuration using MSBuild
It seems that project buid order was broken and Project D built later than Project Q which need the output content of Project D, so the whole build failed.
Not sure that if you use Project Dependency(Right-click on project-->Build Dependencies-->Project Dependencies), if so, only VS IDE Build Will recognize their build order while MSBuild command line will lose the relationship about them.
Besides, I wonder if your main project under Release mode references a dependency project under Release mode. If you did these, the build will definitely go wrong.
All of these above is that I think you made some changes to your project.
You could follow these steps:
1) I suggest you could try to use Project Reference and it will add these xml node in ProjectQ.csproj file to strongly specify build relationship:
<ItemGroup>
<ProjectReference Include="..\ProjectD\ProjectD.csproj">
<Project>{26c26cdd-a5e0-40c7-b0c9-4563f969424f}</Project>
<Name>ProjectD</Name>
</ProjectReference>
</ItemGroup>
Also, check if there are any conditions that distinguish between Debug or Release mode when referring to a project like this:
<ProjectReference Include="..\ProjectD\ProjectD.csproj" Condition="'$(Configuration)'=='Debug'">
If so, please remove that condition Condition="'$(Configuration)'=='Debug'" to make sure it is the same between Debug and Release mode.
2) close VS Instance, delete .vs hidden folder under solution folder.
3) check your CI Build and cloud build server and ensure that cloud parameters are consistent with other servers. And examine any of your xxx.csproj files to check if you have any other operations which causes this bahavior.
In addition, if necessary, you can share your xxx.csproj file and the build script with us to troubleshoot it.

Compiling projects in different configurations in a multi-projects Visual Studio Solution

I have a visual studio solution containing multiple projects. Just to simplify the testing, I have created 2 sub-projects. I need to compile and run one sub-project in Win-32 configuration because it uses an external DLL (win-32 config) and another sub-project in x64.
How can I do that?
UPDATE-1: I already tried to set the configuration of both the projects as shown in the screenshot below. But as soon as I build the proect then both the projects are build in the configuration which is selected in the main window.
Open the Configuration Manager (menu Build > Configuration manager). There, you can set the configuration and platform for individual projects for each solution configuration+platform pair.

XAML build not able to run Unit Test

I am trying to run Nunit Test using Visual Studio test Runner from XAML build in TFS,but i am getting error saying
TF900547: The directory containing the assemblies for the Visual Studio Test Runner is not valid ''.
Based on your history, I assume you are using VSTS, you can refer to these steps to run Nunit test:
Create a unit test project via VS
Install NUnit and NUnit3TestAdapter packages
Add solution to source control (if the package files not added to the source control, you can refer to the steps below)
Open Source Control Explorer in VS
Add a new folder (e.g. Tools) and add Nuget.exe to the source control
Add bat file to your test project (For example: Tools\nuget35.exe restore NUTest2\NUTest2.sln)
Open XAML build definition and map Tools source folder to agent
Select Process section
Choose TfvcTemplate.12.xaml build template
In the Advance section of Build, specify Pre-build script path with previous bat file.

Working directory for google test in Visual Studio

I have a Visual Studio 2012 C++ solution generated using CMake in which I use google test for unit tests. This works mostly fine, but in one of my tests I want to read a settings file from a local directory. To find the file I copy the file as a post build step from my source code tree to the build and install directory using the following CMake commands:
install(FILES ./adapters/settingFile.txt DESTINATION .)
add_custom_command(TARGET testAdapters POST_BUILD
COMMAND "${CMAKE_COMMAND}" -E copy
"${CMAKE_CURRENT_SOURCE_DIR}/adapters/settingFile.txt"
"${CMAKE_CURRENT_BINARY_DIR}"
COMMENT "Copying elastix parameter files")
This works fine: after building my test the settingFile.txt is in the same location as the testAdapters.exe. Using a right click on the testAdapters project and starting a Debug session also works find.
However if I choose to run the test from within the "Test Explorer" window, either by "Run All" or by right clicking the test and choosing "Run selected tests", the test cannot find settingsFile.txt. By right clicking and choosing "Debug selected tests" I found that running the test from the "Test Explorer" the working directory defaults to the visual studio program directory: C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE. I can think of several possible solutions, but don't know how to achieve this:
Set the working directory for the "Test Explorer"
Set the working directory for each test executable
Set the working directory for all google tests
Using CMake set some define that points to a user specified location and use that in the test code. (I consider this a rather ugly solution)
I need a solution that is platform independent. Does anyone know how to achieve (1) or (2) or do you know of a better solution?
With the current version 0.12.3 of GTA you can at least achieve (1):
Tools
Options
Google Test Adapter (or use Search Options)
General
Working directory (at the bottom)
Unfortunately GTA seems to only support $(ExecutableDir) (the default) and $(SolutionDir). It seems that GTA cannot tell which project is the unit test project, so it is not possible to use the project directory as a start directory.

Teamcity and register dll

I'm trying to build a project with TeamCity Professional 6.5.2
I have a MVC Visual Studio 2010 project using a visual foxpro dll.
The build fails because the dll is not registered.
How can I register that dll in the build machine, as part of the build process?
Thanks!!
There are two different scenarios Rodrigo, your scenario will fall into one of these:
Assemblies which can be referenced by the project: Wherever possible, include dependent assemblies in your source control repository (the 10th Commandment), usually in a "libs" folder which is referenced by the necessary projects. This centralises all your dependencies, makes it easy for new developers to get started and solves your TeamCity build problem.
Assemblies which need GAC installation: There are times when the assembly simply has to be installed in the GAC (i.e. RightFax). Frankly, the easiest thing to do in this case is just to install the damn thing in the GAC. It's an exception and whilst some people will philosophically argue against it, alternatives can get a bit complex (i.e. automating installation as part of the build), and for money I'd rather invest the time elsewhere.
Installing assemblies on each machine (dev, build, prod) will make future updates and any continuous integration system difficult to maintain in the future.
It would be better to create a folder in your working directory (I call it "Resources") which contains a folder for each group of DLLs. Then you just reference the assembly directly from the Project each time. You get duplicate DLLs across different projects, but it keeps everything very clean and simple.
In Visual Studio 2010 you can download a little utility called NuGut which I believes helps to manage assemblies in a better way.
It is a bad idea to register all DLLs on build agent machine, sometime you can find out that you have to install Visual Studio on a build agent machine, for instance ;)
I would suggest to place such libraries along with your code base in Source Control system (I assume you are using it), and just reference lib's folder by declaring a MSBUild property like
<PropertyGroup>
<LibFolder>$(PathFromCommandLineOrJustHardCodedPath)</LibFolder>
</PropertyGroup>
And then before doing a Build/Compile just copy files from $(LibFolder) into the build $(OutputFolder) using standard MSBuild command Copy by specifying source and destination files, and thats it.
To force TeamCity to pickup Lib (dlls) files from VCS folder just add path mapping into the:
TeamCity Configuration Settings -> Edit Build Configuration -> VCS Root settings -> Client Mapping
EDIT:
In your case looks like you have integration test which depends on external COm Server which you need to launch, so you can programatically registr this DLL, for instance in TestSetup, and then do not forget to unregister it in test TearDown.
Anyway you have to just reference this DLL as
VS Project -> Add Existing Item -> Add as Link
+ set
Copy to Output Directory -> Copy if newer
How-To register COM dll: see this SO post
Finally I added a prebuild event in Visual Studio running the regsvr32 command. The dll was in a known location, so I referenced this way:
regsvr32 /s $(SolutionDir)Lib\ProjDataAccess\ProjDataAccess.dll
Agree with Troy Hunt.
For a Continuous Integration with TeamCity 8.0.2 where you need an assembly (custom made shared assembly) reference from GAC, do the following.
1. Add the assembly to the build pack (In my case its nuget package).
2. Before starting the build process (MSBuild, Visual Studio etc), register the assembly to GAC with the help of a command line build runner.
3. Continue with the build process.
By following the above steps, if you are using MSBuild, the build process can link to a valid reference from GAC, which can produce an expected build result.
In my case, where the code refers a custom component from GAC, MSBuild ignored the assembly reference and completed the build process without an exception. But the deployment with the build output failed. To overcome this situation, I've used the give steps.
Note: It is mandatory that the TeamCity agent needs to run under an elevated user account. Else the addition to GAC won't be permitted and the build process can fail.