If a C# project with a unit tests that uses MS fakes and a 3rd party library in .dll I'll refer to as X. In my unit test project the .dll is referenced locally and is packed in the Azure GIT. Now when building the solution via an Azure DevOps pipeline using a MS hosted agent everything works fine, but the units tests keep failing at every test.
The unit test yaml looks as follows:
- task: VSTest#2
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
**\*UnitTests*.dll
!**\*TestAdapter.dll
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
The error:
System.IO.FileNotFoundException : Could not load file or assembly 'Y, Version=4.0.0.0, Culture=neutral, PublicKeyToken=...' or one of its dependencies. The system cannot find the file specified.
Y is another .dll which I assume the first library depends on, although using dumpbin /dependents X.dll doesn't show it. I've tried adding Y to the project as reference yielding the same result. I've tried using the following Powershell script to add it:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$dllpath = 'dll'
$files = Get-ChildItem -Path $dllpath -Name
[System.Reflection.Assembly]::Load("System.EnterpriseServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a")
$publish = New-Object System.EnterpriseServices.Internal.Publish
Foreach ($file in $files)
{
$publish.GacInstall($dllpath + "\" + $file)
}
which changes the errors as follows:
System.Runtime.InteropServices.COMException : Retrieving the COM class factory for component with CLSID {...} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).
I've checked online for the error, but couldn't find any solution to solve the issue.
Anybody have any idea's how I can get the unit tests working on the MS Agent?
I've tried adding Y to the project as reference yielding the same result.
When you add the Y to the unit test project, make sure the path of the dll file in the project file is relative path:
<ItemGroup>
<Reference Include="Y">
<HintPath>..\Y.dll</HintPath>
</Reference>
</ItemGroup>
And confirm the properties Copy Local is set as True:
which changes the errors as follows:...
It seems dll is registered on the 32/64 bit version of the windows registry, but the application is using the different bit version. Try to change your project build properties Platform target from Any CPU to x86 or x64 based on your system and it may fix the issue.
Check the similar thread for some more details.
Hope this helps.
Related
How can I set the path to my external binaries during test discovery in visual studio's Test Explorer? After that how to make sure, it uses the correct paths?
I use windows 10 and VS 2019. I have a solution that builds some binaries and some tests into different folders. Also, I have some 3rd party dependencies, each in its own folder.
Something like:
solutionDir/
-ownBinaries/
-testBinaries/
-externalBinaries/
I'd like to use the Test Explorer to run my tests. For this purpose, I use a .runsettings file. I installed Google Test adapter via NuGet (later it will run on CI, so this is the only option). The automatic runsetting discovery is disabled, and this file is selected as the runsettings file. It overrides the workingDir to my ownBinaries folder, and extend the PATH enviroment variable with the externalBinaries. The relevant parts are:
<SolutionSettings>
<Settings>
<AdditionalTestExecutionParam>-testdirectory=$(SolutionDir)</AdditionalTestExecutionParam>
<WorkingDir>$(SolutionDir)ownBinaries</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
This is works fine, after my tests are discovered, but I have problems when it tries to discover my tests.
I use google test and c++, so the test discovery tries to run those tests with the --gtest-list-tests argument, then populate the view with the test name, case, etc. The binaries are just fine, builds without error, I can run them from the debugger, and they produce the output I want.
But the test explorer won't show them, because it doesn't set the externalBinaries path.
This is what lead me to this situation.
First I copied every binaries next to my test exe, namely into the testBinaries folder. Then, I could run it in the cmd with the --gtest-list-tests argument. Everything was fine, all my test names showed up. Started VS, and Test Explorer discovered all my tests, it was able to run them.
Then I done a clean build, so the external stuff deleted from the testBin folder. The Test Explorer cached the test names, so it was able to run them.
Restart VS. Test Explorer tries to discover my tests. but it fails whit this helping message: (removed date+time)
Google Test Adapter: Test discovery starting...
Failed to run test executable 'D:\MySolution\testBinaries\SBCUnitTest.exe': One or more errors occurred.
Check out Google Test Adapter's trouble shooting section at https://github.com/csoltenborn/GoogleTestAdapter#trouble_shooting
In particular: launch command prompt, change into directory '..\ownBinaries', and execute the following command to make sure your tests can be run in general.
D:\MySolution\testBinaries\SBCUnitTest.exe --gtest_list_tests -testdirectory=
Found 0 tests in executable D:\MySolution\testBinaries\SBCUnitTest.exe
Test discovery completed, overall duration: 00:00:00.3022924
Have you noticed that -testDirectory= is empty despite it is set in the runsettings file?
I'm completely lost how I can proceed with it. This workaround is quite heavy to copy all files, then delete all but the test binaries each time when I start VS.
Here is the link for the Troubleshooting section mentioned in the error message.
I've read through the readme file on github, also the runsetting docs on Microsoft's website.
Edit
I made progress with the VsTest.console.exe, I can successfully run all my tests with the proper arguments as below:
& "VSTest.console.exe" *_uTest.exe /Settings:..\MySolution.gta.runsettings /TestAdapterPath:"..\packages\GoogleTestAdapter.0.18.0\build\_common\"
I use the same *.runsettings and *.gta_settings_helper files. Those files are used to get absolute paths for the dependencies. I could run this from different folders, but then I had to adjust the arguments (test discovery pattern, relative path to runsettings, and relative path to GTA).
Great news, that it successfully runs on Azure (it uses vstest.console).
Edit 2
Tried to merge the workingDir and pathExtension nodes, so only one needed (the pathExtension). No success.
Tried to install Test adapter for google test in the VS installer, delete the runsetting file, and set the properties in VS->Tools->Options then Test adapter for google test. Even the example pathExtension didn't worked for me.
Found the extended logs under %AppData%/Local/Temp/TestAdapter/someNumber/*.txt and in that log I've found one line as the runsettings file. I paste here the formatted version of the log
<RunSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
<WorkingDir>$(SolutionDir)</WorkingDir>
<PathExtension>$(SolutionDir)externalBinaries</PathExtension>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
<GoogleTestAdapterSettings>
<SolutionSettings>
<Settings>
</Settings>
</SolutionSettings>
<ProjectSettings>
</ProjectSettings>
</GoogleTestAdapterSettings>
</GoogleTestAdapterSettings>
</RunSettings>
Does anybody know why is there an empty google test adapter setting? Where does it comes from? I think this is overwrites my settings.
It turned out, before first run the relative paths are not known.
Trivial solution
Add the full path to the PATH Extension under Visual Studio -> Options -> Test Adapter for Google Test settings. Meanwhile the custom *.runsetting file is not selected.
Using this method all my tests are discoverable, but it is a manual setting for each repo cloned.
I have this build :
It works fine. The only issue is that the Test Results are overridden. So I actually end up with the test results for the last test project executed.
This is executed by build engine;
C:\Program Files\dotnet\dotnet.exe test C:/agent/_work/4/s/test/Services.UnitTests/project.json --configuration release -xml ./TEST-tle.xml
C:\Program Files\dotnet\dotnet.exe test C:/agent/_work/4/s/test/Web.UnitTests/project.json --configuration release -xml ./TEST-tle.xml
What could help:
1) having "dotnet test" generate XML output file - did not find a way how to do that
2) Use a variable for -xml output file in Build Task. That variable could be a random string/number or just a project name being tested - like what Build engine feeds to "dotnet.exe test". No way how to do that.
Any ideas? Thanks.
I think that, although you're running the task against all of the projects in one go, as the .Net Core (Preview) task doesn't have a working directory, that the test results are being generated at solution root (or similar) and done for each project in turn.
I set mine up using simple command line tasks...
Tool: dotnet
Arguments: test -xml testresults.xml
Working folder: {insert the folder for the project to test here}
These work fine but I have one set up for each project. You could try creating a task for each library and adding the full path to the test results argument (or name them appropriately as starain suggested).
This feels like a minor bug to me.
Based on my test, it doesn’t recognize the date variable as Build Number.
To deal with this issue, you can add another .Net Core (Test) step to run xunit test with different result file.
For example:
I'v some DB facing unit tests that take their connection string from an app.config file. I'm grabbing the strings thru ConfigurationManager class.
var conString = ConfigurationManager.ConnectionStrings["DB"].ConnectionString;
The tests run fine on dev machine but not on TFS. There is a null refernce error. If I jsut replace the connection string in unit test with a hard coded string - identical to that in app.config - then the test works. Oh yeah, and I've made sure that app.config is set to "copy to output folder".
Is there a known issue with TFS and ConfigurationManager?
pom
I recently ran into the same problem with TFS 2015 (on premises with update 2). We have multiple test projects that access a connection string from the app.config file however one particular project failed to access the connection string which caused our build to fail.
I altered the .csproj file to match other working test projects but this didn't provide any success, instead the following worked for me:
Copied the contents of the App.config to notepad
Deleted the App.config file from the project
Added a new Application Configuration File (App.config) to the project
Pasted the contents from notepad into the new App.confg file
Right click the App.config file and altered its properties:
Build Action: Content
Copy to output directory: Copy always
Upon check-in all unit tests within the project passed successfully.
It's an odd solution but it got our tests running.
Note: Deleting the App.config file was a necessary step. Changing the properties on the file (prior to deleting the file) didn't make a difference.
Update:
After applying the above solution I noticed all unit tests were running twice and after some investigation I discovered the entire issue was down to our build configuration: -
On our Visual Studio Build task we use the following parameter on our MS Build Arguments:
/p:OutDir=$(Build.StagingDirectory)
Our Visual Studio Test task was configured to look for Test Assembly in the following location:
**\$(BuildConfiguration)\*test*.dll;-:**\obj\**
Altering the Test Assembly location to the following fixed everything:
$(Build.StagingDirectory)\*test*.dll;-:**\_PublishedWebsites\**
No more issues with App.config files and unit tests ran once only.
Actually, I made another test project in my solution and added the app.config there again with the conn strings and now it works also on TFS. I don't know what the cause was but I'll leave the question here for possible reference and solution for others. Thank you.
My Test project is a separate project from the main project. Therefore it needs its own copy of App.config. Follow the steps in previous response to add new Application Configuration file and give it the same contents as in your main project.
I have a VS2010 unit test project set to using SpecFlow 1.8.1 and mstest. In order to get the SpecFlow unit tests working, I've done the following:-
I added the references to the following files in my project:-
Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
TechTalk.SpecFlow.dll
Note that the TechTalk.SpecFlow.dll has been added into my project and the reference points to that file.
I've set the "Copy Local" property of the TechTalk.SpecFlow.dll reference to True.
I've also added an App.Config that specifies "MsTest.2010" as the provider, and regenerated all code-behinds for the SpecFlow features.
Everything works in my VS2010, the tests run successfully in both the SpecFlow testrunner and the mstest test runner. BUT when I try to run the mstests in TFS 2008 (using a .vsmdi test list file), it failed with the following exception:-
Class Initialization method MyNamespace.MyTestFeature.FeatureSetup threw exception.
System.Configuration.ConfigurationErrorsException:
System.Configuration.ConfigurationErrorsException: An error occurred creating the
configuration section handler for specFlow: Could not load file or assembly
'TechTalk.SpecFlow' or one of its dependencies. The system cannot find the file
specified. (D:\Projects\TestProject\TestResults\administrator_MYPC 2012-06-27
18_30_05_Any CPU_Debug\Out\TestProject.DLL.config line 4) --->
System.IO.FileNotFoundException: Could not load file or assembly 'TechTalk.SpecFlow'
or one of its dependencies. The system cannot find the file specified.
Note that the TFS built the project fine and it runs other unit tests in the same project (normal mstests, not SpecFlow) without problems. It only failed for the SpecFlow test runs.
So what am I doing wrong?
Edit: The contents of my App.Config file looks like this:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section
name="specFlow"
type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
<unitTestProvider name="MsTest.2010" />
<runtime detectAmbiguousMatches="true"
stopAtFirstError="false"
missingOrPendingStepsOutcome="Inconclusive" />
<trace traceSuccessfulSteps="true"
traceTimings="false"
minTracedDuration="0:0:0.1" />
</specFlow>
</configuration>
Following the instruction on this site and this site:
the command Tools > Library Package Manager > Package Manager Console allows you to type in PM> Install-Package SpecFlow
when the prompts returns "installed successfully", the SpecFlow Assembly now appears in the references of your project. And the MSTest project now compiles succesfully (at least for me).
I got this error as well, in my case the problem was that I was using the \...\obj\Debug||Release\ folder as target and not the \...\bin\Debug||Release\ folder. Looking in these folders I saw that the TechTalk.dll assembly was missing from the former. Simply switching in my .bat file the problem was fixed.
Sometimes VS2013 is looking for SpecRun dlls not in project folder, but in C:\Users\**YOUR_USER**\AppData\Local\Temp\VisualStudioTestExplorerExtensions\SpecRun.Runner.1.3.0\tools. So you just need to put all necessary SpecFlow libraries therel
One hack I found to get it working is to add another class for EVERY single SpecFlow feature that I created in the project. The class looks like this:-
[DeploymentItem(#"TechTalk.SpecFlow.dll")]
partial class MyTestFeature { }
// The above class-name needs to come from the auto-generated code behind
// (.feature.cs) for each SpecFlow feature.
I consider this as a very nasty hack, but it does provide a clue as to why it didn't work. It would be good if anyone comes up with a more elegant solution.
I finally found the more proper fix for this issue. I just need to add a post-build event to remove the .config file from the build output. (The App.config file is used only to generate the code-behind during design time. It is not used at all during runtime, so it can be removed.)
The command for the post-build event looks like this:-
del /f /q "$(TargetDir)$(TargetFileName).config"
Correction: The .config file is used for generating inconclusive results, so a better post-build event command is as follows:-
if "$(IsDesktopBuild)"=="false" del /f /q "$(TargetDir)$(TargetFileName).config"
We have been using TeamBuild and test for our continuous integration build for about 4 months and this issue just popped up the other day in one of our test assemblies when running tests on one of our test assemblies on the build server only.
{NameSpace}.Order_Accessor, Version=0.0.0.0, Culture=neutral, PublicKeyToken='{xxx}' or one of its dependencies. The system cannot find the file specified.
This is not being caused by problems with Publicize.exe noted elsewhere. The unit tests run fine locally and the generated assembly does get created.
The part that is confusing is that we use generated private accessor assemblies in numerous projects including the one with this issue above without any problems. Investigating this issue further, the generated assembly never gets copied to the TestResults//Out directory which appears to be the problem. I determined this was the issue by adding the assembly from SolutionRoot using the TestRunConfig deployment settings which results in an assembly manifest does not match error.
I am wondering if anyone has had issues with these assemblies being copied properly. We are running TeamSystem 2008 on Windows Server 2008 if that helps.
Below is the test run deployment error for the file in question as well
Run has the following issue(s):
TESTTOOLSTASK : warning : Test Run deployment issue: Failed to get the file for deployment item 'E-mail Templates\OrderConfirmation.txt' (output directory 'E-mail Templates') specified by the test 'EmailServiceTest.EnqueueTest': System.IO.DirectoryNotFoundException: Could not find a part of the path 'Continuous Integration Build\Binaries\Continuous Integration Test\E-mail Templates\OrderConfirmation.txt'.
TESTTOOLSTASK : warning : Test Run deployment issue: Failed to get the file for deployment item 'E-mail Templates\OrderConfirmation.htm' (output directory 'E-mail Templates') specified by the test 'Email.Tests.EmailServiceTest.EnqueueTest': System.IO.DirectoryNotFoundException: Could not find a part of the path 'Continuous Integration Build\Binaries\Continuous Integration Test\E-mail Templates\OrderConfirmation.htm'
TESTTOOLSTASK : warning : Test Run deployment issue: The assembly or module 'Services.Order_Accessor' directly or indirectly referenced by the test container '\continuous integration build\binaries\continuous integration test\services.order.supplierintegration.tests.dll' was not found.
Looks like your TestRunConfig deployment expects that the files are in "Continuous Integration Build\ Binaries\"
I've highlighted the "Binaries" part for a reason. This is a folder the $BinariesRoot property points to (unless overridden), and it is used by the compiler. Therefore, I'd suggest you double-check that:
The Publicize tool (or whatever tool you use to generate the assemblies) is configured to copy the output into ($BinariesRoot)Continuous Integration Test
There are provisions in the team build type to copy the .htm and the .txt files to ($BinariesRoot)Continuous Integration Test