I'm making use of the TestDriven.NET plugin for visual studio to run all my tests to check for code coverage with NDepend. All is working fine apart from my tests with a HostType attribute. These aren't getting run.
Does anybody knopw how to fix this or is there a specific reason for it?
Thanks in advance
If you look at the release notes for TestDriven.NET, they explicitly skip test with the HostType attribute. I'm not sure why they do this exactly.
I can actually not find another test runner outside of the built in runner in Visual Studio that will execute those tests, Resharper/dotCover gives me an Aborted message when running a specific test with a HostType
EDIT: Linked to release notes (look at item 3203) http://www.testdriven.net/downloads/releasenotes.html
Related
We are trying to run Code Coverage tests using cloud test and it fails with the below exception:
System.Security.VerificationException: Operation could destabilize the runtime.
The Tests pass if we run without the code coverage, but when code coverage is enabled it fails with the above error.
Things tried:
We tried running the code by adding : [assembly: SecurityRules(SecurityRuleSet.Level1, SkipVerificationInFullTrust = true)] in the AssemblyInfo.cs files, but did not help.
Could someone help shed some light why this might be happening or any fix for this?
Thanks in Advance.
You can try to set True in runsettings for the vstest task and enable the code coverage check box. (https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019)
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
The fix is present in the version of the vstest task greater than 2.153.0.
vstestplatform version needs to be greater than 16.1 (anything greater than 16.0.2 also works)
You can check https://github.com/microsoft/vstest/pull/1997 for more details.
You can use the the combination of:
Latest Newtonsoft.Json 12.0.2 (doesn't work with 12.0.1)
Latest stable Test platform: 16.2 (should be >16.0.2)
Latest VSTest task 2.156.2 (should be >2.153.0)
Here is a ticket with the same issue you can refer to.
I have a collection of MsTest and NUnit tests that are being run in TFS 2015 using the VS Test step. I'm using NUnit Test Adapter 3.4.1 to run the tests on the build agent.
At the end, even though the tests pass, Nunit seems to croak and the build step fails with these two errors.
2016-09-04T09:59:44.7209787Z ##[error]Error: Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
2016-09-04T09:59:44.7209787Z ##[error]
2016-09-04T09:59:44.7209787Z ##[error]Error: Exception encountered unloading AppDomain
2016-09-04T09:59:44.7209787Z ##[error]
2016-09-04T09:59:44.7209787Z Information: NUnit Adapter 3.4.1.0: Test execution complete
2016-09-04T09:59:44.8615975Z Results File: C:\agent\_work2\1\TestResults\SRV-BLD1 2016-09-04 01_22_45.trx
2016-09-04T09:59:44.8615975Z Total tests: 139. Passed: 134. Failed: 0. Skipped: 5.
I've checked that there are indeed 139 tests in the suite, and 5 are set to ignore (2 are MSTest, and 3 are NUnit).
I'm not sure if there's a place to get more detailed explanation of the error. Searching this site and google seems to suggest that the NUnit.Engine.NUnitEngineException is linked to test discovery (here, and here for instance), but my tests are being discovered, so I'm not sure if this is related (Pretty new to NUnit, so not sure of a lot of things).
I also saw two links dealing with Adapter failures (here and here), but the errors don't quite match up, although maybe just because I don't have the same level of verbosity.
In TFS, the step doesn't have any configuration on it, just that it's enabled and the path to the DLLs.
Does anyone know what is causing the errors (which I'm assuming are causing the build to fail)? Alternatively, what should be the next steps in getting a more precise/verbose error stack to investigate the issue?
As a side note, I saw this SO answer, which states this:
MSTest.exe returned an exit code of 1 indicating that not all tests
passed.
I wasn't able to find any confirmation that VSTests fails when it encounters Skipped tests, but could this also be an issue?
Thank you for any help.
UPDATE
As suggested below, I tried running this from the IDE directly, and got this output (folders redacted)
------ Discover test started ------
NUnit Adapter 3.4.1.0: Test discovery starting
NUnit Adapter 3.4.1.0: Test discovery complete
========== Discover test finished: 139 found (0:00:00.8820879) ==========
------ Run test started ------
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
NUnit Adapter 3.4.1.0: Test execution started
Running all tests in C:\agent\_work2\1\s\codePorject\bin\Debug\codeProjectTests.dll
NUnit3TestExecutor converted 37 of 37 NUnit test cases
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
Exception encountered unloading AppDomain
NUnit Adapter 3.4.1.0: Test execution complete
========== Run test finished: 139 run (1:20:10.3290294) ==========
I've found a similar xUnit issue open, but it doesn't seem to have a solution.
This StackOverflow answer suggests to use a sleep timer, so I might try that.
In the end, it looks like some sort of a race condition between Firefox browser windows and nUnit. In my cleanup code, I'm killing firefox and iisexpress processes. Adding a sleep call has eliminated the issue:
public static void AssemblyCleanup()
{
foreach (var process in Process.GetProcessesByName("firefox")) process.Kill();
foreach (var process in Process.GetProcessesByName("iisexpress")) process.Kill();
System.Threading.Thread.Sleep(5000);
}
An NUnitEngineException is what it says: an exception that was discovered by the engine. It can be caused by many things and the message indicates the problem. In your case, the message says "Exception encountered unloading AppDomain" which means... well what it says.
The test adapter catches and handles the exception, producing the message you see. There is some indication that TFS also sees the exception and fails test run as a result. If you were to run this under the VS IDE, I think you would see the message from the adapter but the run would not fail. There is an NUnit3 Test Adapter issue about this, but it's not clear if the solution lies within the adapter or if it's a TFS problem.
If you do try this under the IDE I hope you will post the output window text from that run as well.
Simply put:
When I execute the tests using boost runner I get the following error and the c/c++ unit testing hangs.
XML parse error: The entity name must immediately follow the '' in the entity reference.
How do I avoid this from happening?
FYI:
The test runners run perfectly fine from the command line outside of eclipse, or in a debug window in eclipse, so it must be an eclipse issue isolated in the "c/c++ unit console".
Im running Kepler.
Gracias.
If you put a BOOST_CHECK inside a test stub, and the tested code is multithreaded, the asynchronous <Info> output will probably corrupt the XML output. Check the XML output carefully to spot what is the offending test.
At least up to 1.57, while validating stub usage is also important for unit-testing modules, Boost.Test is not considered thread-safe in this context.
You could probably work around the problem by redirecting the output stream and overloading boost::unit_test::unit_test_log.set_stream.
I have created simple MS Unit Test Project. Assert.Equal(true, true).
When I want to run a test I receive a error from Resharper and Visual Studio.
"Unit test Runner failed to run tests
System.Xml.XmlException: An Error occured while parsing EnityName. Line 1, position 17."
Do I missing something.
Edit:
I can't add screenshot and I can't select text to copy. Ok I will write it the best I can.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.xml.XmlTextReaderImpl.Throw(String res,String arg)
at System.Xml.XmlText.ParseEnityName()
at System.Xml.XmlTextReaderImplparseEnityReference
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextLoader.ParsePartialContent(XmlNode parentNode,String innerxmltext, xmlNodeType nt)
at System.Xml.XmlElement.set_InnerXml(string value)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingHelper.AppendOrModifyChild(XpathNavigator,String nodePatH,String nodeNAME,sTRING INNERxML)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.UpdatedRunConfiguration(XPathNavigator, ArchitectureeffectivePlatform,FrameworkVersion effectiveFramework,String resultsDirectory,String solutionDirectory,String binariesRootDirectory)
Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.MergeRunSettingsAndFindCompatibleSource()
at JetBrains.ReSharper.UnitTestProvider.MSTest11.MsTest11Runner.Run(IRemoteTaskServer server,String assemblyLocation,String runConfigurationFilename,IUnitTestRun run)
To solve this problem you don't need to reinstall Visual Studio or Resharper.
The problem was in a file path to my projects. This would explains why other Unit Test projects were fine because there were in a different location.
I guess this a bug inside Visual Studio Unit Testing Framework. If you have unit test project inside folders with special charts like '&' then Visual Studio will crash or Resharper will return this message.
I wonder where I can report this bug.
I have just run into this problem too and followed the advice above.
First my path had a folder named ...\m&e...
Then I copied the whole project folder to a new location without an '&' in the path.
After that everything was working well.
In my project I write tests using Microsoft's unit testing framework. All of my tests pass when I run them from Visual Studio but when I run the tests from MSBuild all of the tests fail with the following erorr message:
Unit Test Adapter threw exception:
Type is not resolved for member
SomeType,SomeAssembly Version=assemblyVersion,
Culture=neutral, PublicKeyToken=..
The assembly not found is a 3rd party assembly referenced by all of the projects.
The build script is used by TFS so I've aded the following lines:
<RunTest>true</RunTest>
<ItemGroup>
<MetaDataFile Include="$(BuildProjectFolderPath)myproject.vsmdi">
<TestList>CI_Tests</TestList>
</MetaDataFile>
</ItemGroup>
I've found the this post that shows a solution to this issue but unfortunatly I cannot chnage the files on the TFS server.
Help!
I encountered the same problem in my unit tests. The linked article above indicates that the problem is that VSTS causes copying of some objects in the thread's CallContext.
For what it's worth, in my case the problem was that I had manually placed an object in the thread's CallContext, which caused this exception. I was able to resolve this by clearing the CallContext in my TestCleanup routine. I didn't have to change any files anywhere.
I had also run into the same issue but where I had StructureMap initialisation being performed within the constructor for a base test class.
I was able to get around the problem by moving the call from the constructor to the [TestInitialize] method. I also ensured that the [TestCleanUp] method disposed of the created StructureMap container. After this MSBuild (2010) would run through the tests without raising this error.
The first thing to check would be if this assembly is copied to the folder from which msbuild runs the tests. It might be the case that you have a copy in your bin/Debug folder because of some historic reasons, but the dependency is not set up properly in the project.
Had this error
Unit Test Adapter threw exception:
Type is not resolved for member 'NHibernate.HibernateException,NHibernate
As it turned out the problem was in exception thrown in static constructor for the test.
It was completely unrelated to the looks of message and was happening during DB creation using BuildSchema.
Very uninformative error message by MSTest which cost me a lot of hours and stress. putting migration to something better like NUnit in our TODO list.
This article solved my problem with this error:
To recap, MyCustomException was thrown at very early stage of test execution. The dll that contained it was not loaded yet so Unit Test Adapter indeed could not reach it.
Happening when the assemblies been tested have loaded instances of their class marked as [Serializable]. Workaround by deriving from MarshalByRefObject so the deserialization is no longer attempted from mstest host.