We have been using TeamBuild and test for our continuous integration build for about 4 months and this issue just popped up the other day in one of our test assemblies when running tests on one of our test assemblies on the build server only.
{NameSpace}.Order_Accessor, Version=0.0.0.0, Culture=neutral, PublicKeyToken='{xxx}' or one of its dependencies. The system cannot find the file specified.
This is not being caused by problems with Publicize.exe noted elsewhere. The unit tests run fine locally and the generated assembly does get created.
The part that is confusing is that we use generated private accessor assemblies in numerous projects including the one with this issue above without any problems. Investigating this issue further, the generated assembly never gets copied to the TestResults//Out directory which appears to be the problem. I determined this was the issue by adding the assembly from SolutionRoot using the TestRunConfig deployment settings which results in an assembly manifest does not match error.
I am wondering if anyone has had issues with these assemblies being copied properly. We are running TeamSystem 2008 on Windows Server 2008 if that helps.
Below is the test run deployment error for the file in question as well
Run has the following issue(s):
TESTTOOLSTASK : warning : Test Run deployment issue: Failed to get the file for deployment item 'E-mail Templates\OrderConfirmation.txt' (output directory 'E-mail Templates') specified by the test 'EmailServiceTest.EnqueueTest': System.IO.DirectoryNotFoundException: Could not find a part of the path 'Continuous Integration Build\Binaries\Continuous Integration Test\E-mail Templates\OrderConfirmation.txt'.
TESTTOOLSTASK : warning : Test Run deployment issue: Failed to get the file for deployment item 'E-mail Templates\OrderConfirmation.htm' (output directory 'E-mail Templates') specified by the test 'Email.Tests.EmailServiceTest.EnqueueTest': System.IO.DirectoryNotFoundException: Could not find a part of the path 'Continuous Integration Build\Binaries\Continuous Integration Test\E-mail Templates\OrderConfirmation.htm'
TESTTOOLSTASK : warning : Test Run deployment issue: The assembly or module 'Services.Order_Accessor' directly or indirectly referenced by the test container '\continuous integration build\binaries\continuous integration test\services.order.supplierintegration.tests.dll' was not found.
Looks like your TestRunConfig deployment expects that the files are in "Continuous Integration Build\ Binaries\"
I've highlighted the "Binaries" part for a reason. This is a folder the $BinariesRoot property points to (unless overridden), and it is used by the compiler. Therefore, I'd suggest you double-check that:
The Publicize tool (or whatever tool you use to generate the assemblies) is configured to copy the output into ($BinariesRoot)Continuous Integration Test
There are provisions in the team build type to copy the .htm and the .txt files to ($BinariesRoot)Continuous Integration Test
Related
How can I set the path to my external binaries during test discovery in visual studio's Test Explorer? After that how to make sure, it uses the correct paths?
I use windows 10 and VS 2019. I have a solution that builds some binaries and some tests into different folders. Also, I have some 3rd party dependencies, each in its own folder.
Something like:
solutionDir/
-ownBinaries/
-testBinaries/
-externalBinaries/
I'd like to use the Test Explorer to run my tests. For this purpose, I use a .runsettings file. I installed Google Test adapter via NuGet (later it will run on CI, so this is the only option). The automatic runsetting discovery is disabled, and this file is selected as the runsettings file. It overrides the workingDir to my ownBinaries folder, and extend the PATH enviroment variable with the externalBinaries. The relevant parts are:
<SolutionSettings>
<Settings>
<AdditionalTestExecutionParam>-testdirectory=$(SolutionDir)</AdditionalTestExecutionParam>
<WorkingDir>$(SolutionDir)ownBinaries</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
This is works fine, after my tests are discovered, but I have problems when it tries to discover my tests.
I use google test and c++, so the test discovery tries to run those tests with the --gtest-list-tests argument, then populate the view with the test name, case, etc. The binaries are just fine, builds without error, I can run them from the debugger, and they produce the output I want.
But the test explorer won't show them, because it doesn't set the externalBinaries path.
This is what lead me to this situation.
First I copied every binaries next to my test exe, namely into the testBinaries folder. Then, I could run it in the cmd with the --gtest-list-tests argument. Everything was fine, all my test names showed up. Started VS, and Test Explorer discovered all my tests, it was able to run them.
Then I done a clean build, so the external stuff deleted from the testBin folder. The Test Explorer cached the test names, so it was able to run them.
Restart VS. Test Explorer tries to discover my tests. but it fails whit this helping message: (removed date+time)
Google Test Adapter: Test discovery starting...
Failed to run test executable 'D:\MySolution\testBinaries\SBCUnitTest.exe': One or more errors occurred.
Check out Google Test Adapter's trouble shooting section at https://github.com/csoltenborn/GoogleTestAdapter#trouble_shooting
In particular: launch command prompt, change into directory '..\ownBinaries', and execute the following command to make sure your tests can be run in general.
D:\MySolution\testBinaries\SBCUnitTest.exe --gtest_list_tests -testdirectory=
Found 0 tests in executable D:\MySolution\testBinaries\SBCUnitTest.exe
Test discovery completed, overall duration: 00:00:00.3022924
Have you noticed that -testDirectory= is empty despite it is set in the runsettings file?
I'm completely lost how I can proceed with it. This workaround is quite heavy to copy all files, then delete all but the test binaries each time when I start VS.
Here is the link for the Troubleshooting section mentioned in the error message.
I've read through the readme file on github, also the runsetting docs on Microsoft's website.
Edit
I made progress with the VsTest.console.exe, I can successfully run all my tests with the proper arguments as below:
& "VSTest.console.exe" *_uTest.exe /Settings:..\MySolution.gta.runsettings /TestAdapterPath:"..\packages\GoogleTestAdapter.0.18.0\build\_common\"
I use the same *.runsettings and *.gta_settings_helper files. Those files are used to get absolute paths for the dependencies. I could run this from different folders, but then I had to adjust the arguments (test discovery pattern, relative path to runsettings, and relative path to GTA).
Great news, that it successfully runs on Azure (it uses vstest.console).
Edit 2
Tried to merge the workingDir and pathExtension nodes, so only one needed (the pathExtension). No success.
Tried to install Test adapter for google test in the VS installer, delete the runsetting file, and set the properties in VS->Tools->Options then Test adapter for google test. Even the example pathExtension didn't worked for me.
Found the extended logs under %AppData%/Local/Temp/TestAdapter/someNumber/*.txt and in that log I've found one line as the runsettings file. I paste here the formatted version of the log
<RunSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
<WorkingDir>$(SolutionDir)</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
</GoogleTestAdapterSettings>
</GoogleTestAdapterSettings>
</RunSettings>
Does anybody know why is there an empty google test adapter setting? Where does it comes from? I think this is overwrites my settings.
It turned out, before first run the relative paths are not known.
Trivial solution
Add the full path to the PATH Extension under Visual Studio -> Options -> Test Adapter for Google Test settings. Meanwhile the custom *.runsetting file is not selected.
Using this method all my tests are discoverable, but it is a manual setting for each repo cloned.
I'v some DB facing unit tests that take their connection string from an app.config file. I'm grabbing the strings thru ConfigurationManager class.
var conString = ConfigurationManager.ConnectionStrings["DB"].ConnectionString;
The tests run fine on dev machine but not on TFS. There is a null refernce error. If I jsut replace the connection string in unit test with a hard coded string - identical to that in app.config - then the test works. Oh yeah, and I've made sure that app.config is set to "copy to output folder".
Is there a known issue with TFS and ConfigurationManager?
pom
I recently ran into the same problem with TFS 2015 (on premises with update 2). We have multiple test projects that access a connection string from the app.config file however one particular project failed to access the connection string which caused our build to fail.
I altered the .csproj file to match other working test projects but this didn't provide any success, instead the following worked for me:
Copied the contents of the App.config to notepad
Deleted the App.config file from the project
Added a new Application Configuration File (App.config) to the project
Pasted the contents from notepad into the new App.confg file
Right click the App.config file and altered its properties:
Build Action: Content
Copy to output directory: Copy always
Upon check-in all unit tests within the project passed successfully.
It's an odd solution but it got our tests running.
Note: Deleting the App.config file was a necessary step. Changing the properties on the file (prior to deleting the file) didn't make a difference.
Update:
After applying the above solution I noticed all unit tests were running twice and after some investigation I discovered the entire issue was down to our build configuration: -
On our Visual Studio Build task we use the following parameter on our MS Build Arguments:
/p:OutDir=$(Build.StagingDirectory)
Our Visual Studio Test task was configured to look for Test Assembly in the following location:
**\$(BuildConfiguration)\*test*.dll;-:**\obj\**
Altering the Test Assembly location to the following fixed everything:
$(Build.StagingDirectory)\*test*.dll;-:**\_PublishedWebsites\**
No more issues with App.config files and unit tests ran once only.
Actually, I made another test project in my solution and added the app.config there again with the conn strings and now it works also on TFS. I don't know what the cause was but I'll leave the question here for possible reference and solution for others. Thank you.
My Test project is a separate project from the main project. Therefore it needs its own copy of App.config. Follow the steps in previous response to add new Application Configuration file and give it the same contents as in your main project.
I have created a TFS2013 Build Definition using the template TfvcTemplate.12.xaml
I have specified a test run using VSTestRunner and enabled code coverage.
I am integrating this build with sonar analysis by specifying pre-build and post-test execution script.
Prebuild script arguments: begin /name:PrjName /key:PrjKey /version:1.0 /d:sonar.cs.vstest.reportsPaths="tst*.trx"
I have the "Unit Test Coverage" widget on my sonar dashboard.
It shows Unit Test Coverage %
However, it does not show the unit tests (ie how many tests were run, how many failed ,etc).
I looked in the build output. There is a "tst" folder, however it is empty.
I cannot find the trx files.
I believe that either the trx files are not properly generated or
I am not setting the "sonar.cs.vstest.reportsPaths" correctly.
Please help !!
Relative paths are not well supported: Specify an absolute path wildcard to your *.trx reports. See https://jira.sonarsource.com/browse/SONARMSBRU-100 for details on the bug.
Note that you probably can use the TFS 2013 environment variables to construct this absolute path wildcard: https://msdn.microsoft.com/en-us/library/hh850448.aspx#env_vars
I have a Xcode 5.1 project (projA) which included a private framework (frameworkB) via Cocoapods. Everything was working, building and testing and even the weather was nice. But, because the frameworkB is being developed in parallel with the projA, I decided to include the project of the frameworkB (proj B) in projA, again via Cocoapods but as a reference with :path ='path/to/projB'
In the result the projB compiles and builds and runs on device, the tests target also compiles and builds but doesn't run, the simulator start and this message is displayed:
2014-04-14 11:08:34.990 xctest[98973:303] The executable for the test bundle at
/Users/myNameHere/Library/Developer/Xcode/DerivedData/projB-manyLettersHere/Build
/Products/Debug-iphonesimulator/projB.xctest could not be found.
Program ended with exit code: 1
Also the weather is not so nice anymore.
Google didn't help. Other stackoverflow question are more about transition from Sentest cu XCTest.
Any hint that will put me on the right path will be greatly appreciated.
The cause and solution to this problem is (as usual) very simple:
The framework that was being built contained a bundle, which was required, but (of course) was not generated by the Pod project, even if in the podspec i specified the spec.resources pram. The solution was to create a spec.resource_bundle with the required name and resource files.
Why it would throw this error and not a compile-time or runtime error i still don't know :|
(the projB.xctest was present at the path in error)
We are writing unit tests for our business layer running under .NET 4.0. The business layer is a straightforward C# class library that usually runs within SOAP and REST web services. Our application uses log4net within a separate wrapper assembly for logging. The C# code in the logging assembly has an assembly info directive that tells log4net the name of the configuration file, a la-
[assembly: log4net.Config.XmlConfigurator(ConfigFile="AcmeLogging.config", Watch=true)]
Initializing the log4net through the wrapper works fine in the web services. When we initialize it from with our unit test assembly it does not appear to see the configuration file. The configuratino file is configured through properties to be copied to the execution directory, and we do see it in the bin\debug directory. A quick console test program using the logging assembly running from within that same folder works fine. The curious thing is that the behavior problems are intermittent, and pop up on different developers' machines at different times and cannot be cured in any deterministic way.
Stepping through the wrapper assembly code, the log4netLogManager.GetLogger() call appears to return correctly, but the list of appenders returned by log.Logger.Repository.GetAppenders() is empty. Since this incorrect behavior is the same whether the file is in the Bin\Debug folder or not, we believe that it is not seeing the file.
Any clues as to what we're missing about running log4net in the Microsoft Test Framework would be greatly appreciated.
Unit tests are unique in the fact that if you need configuration files in your unit tests you need to include them as deployment items. Here is an example of how I do this within my test class:
[TestClass]
[DeploymentItem("hibernate.cfg.xml")]
public class AsyncForwardingAppenderTest
{
}
In addition to the Deployment attribute you need to Enable Deployment in your test settings. To do this go to Test->Edit Test Settings->. Then click on Deployment area on right. Click the checkbox Enable Deployment.
After doing this and running your test your config file should be located in your test results folder. TestResults\username_machine date stamp\Out. If your config file is not located in this folder it will not work. This is what the DeploymentItem attribute does. It sticks the file in this Out folder.
If you don't want to include the DeploymentItem attribute on every test class what I did was to create a base test class that all of the tests that use log4net inherit from and mark it with the DeploymentItem attribute.