I use Visual Studio adapter for Boost.Test. I built boost.test[v1.59] project using cmake.
I didn't have any compile problems but when I tried to run tests I got the output like below:
Executing: -> [BoostUnitTestSample]
filestream was found to be null when handling path: D:\Work\Project\dev\DesktopMSVC13_Project\src\ProjectTest\Debug\ProjectTest.exe.test.report.xml
Exception caught while running test batch D:\Work\Project\dev\DesktopMSVC13_Project\src\ProjectTest\Debug\ProjectTest.exe [BoostUnitTestSample]
(File 'D:\Work\Project\dev\DesktopMSVC13_Project\src\ProjectTest\Debug\ProjectTest.exe.test.report.xml' not found.)
========== Run test finished: 0 run (0:00:03,6081912) ==========
Everything goes fine if I add a boost.test project manually
My experience with the Boost.Test extension for Visual Studio (see comment) tells me that you forgot to set references to dependencies required to run the tests (typically DLLs). I think the message is rather created by the standard test execution engine than by Boost.Test extension. It is just a clumsy way of saying "oh, the test did not run, so I could find the results..."
Related
I have been using a number of c++ projects for gtest unit cases. While all these were executing fine all these years, now after updating the openssl version 1.1.1a, seeing some strange problem. Bilt in VS2017
the test project exe is not executing or not starting at all. I tried executing the exe from a command prompt. It just comes out of the execution immediately without any execution or message. It is not taking or considering any argument like --shuffle-tests.
I tried adding trace cout logs in main function but still its has no effect.
The error i get is
6: Test command: plugin-tests.exe "--gtest_shuffle" "--gtest_output=xml:TESTS-plugin.xml"
6: Test timeout computed to be: 10000000
6/8 Test #6: plugin-tests ...........***Failed 0.01 sec
I am helpless in debugging in the gtest exe as there are no proper documentation for this as well.
Could any body please shed some light on this ?
It has the standard GTest functions to init and runn all tests. This has been working all these years.
The change i made in the code is,
Modified
EVP_CIPHER_CTX m_context_encrypt
to
EVP_CIPHER_CTX *m_context_encrypt
and did
m_context_encrypt = EVP_CIPHER_CTX_new()
as per the structure changes in openssl1.1 onwards.
And, the openssl is statically linked to the project exe which required two of the windows libraries to be linked with
crypt32 ws2_32
so added those dependency the cmake like,
if( WIN32 )
target_link_libraries(${PROJECT_NAME} crypt32 ws2_32)
endif()
The gtest unit test cases are executing successfully in linux build.
The problem is with the windows execution.
However out of 8 test projects, one test project is able to execute properly in windows, and i find no difference in the cmake files.
It is kindof strange
Note: Exit code of the exe seems to be 255
This question is related to the problem where my unit test procedure is crashing but I don't know on which unit test.
I realise this question is a duplicate of How can I find out which test method in a batch of test methods fails to run? but I need to try anyway:
I'm running some unit tests (about 118 of them), but one of them seems to make the Visual Studio unit test environment crash. This is what I see in the "output" window's "Tests" tab:
[5/02/2018 11:13:18 Informational] ------ Run test started ------
[5/02/2018 11:13:38 Error] The active Test Run was aborted because the execution process exited unexpectedly. The test execution process crashed while running the tests. To investigate further, open file:///C:/Users/DominiqueDS/AppData/Local/CrashDumps/vstest.executionengine.x86.exe.19136.dmp file in Visual Studio and choose "Debug in mixed mode".
[5/02/2018 11:13:38 Informational] ========== Run test finished: 65 run (0:00:19,6926824) ==========
The reason that I have that dump file is thanks to my Windows configuration which automatically creates such dumpfiles in case of a crashing application. (The procedure for this configuration is found under this URL)
Examining the dump file gives me more information on the test, which is failing: I have a hunch where I can find it, and in another thread, I find a function, calling a method which seems to contain the name of the unit test I'm running.
Although I know which test is failing, it's based on hunches and on dumpfiles I should even not generate, so I'm looking for another solution:
In the function TEST_METHOD in the file "c:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\VS\UnitTest\include\CppUnitTest.h", I've tried to add the following lines:
cout << "Test : ";\
cout << methodName;\
cout << "\n";\
I was hoping that this would show every test I wanted to run, but instead I get the error message that this is wrong (also using std::cout and OutputDebugString() function seems not to be allowed).
Hence my question: does anybody know a way to find out, in case one of my unit tests makes my test environment crash, how can I find out which test this is, without needing to generate and debug the Visual Studio dumpfile?
For your information: looking in the "Test Explorer" is not helping: I have a list of disabled tests, a list of successful ones, and a list of some which are not finished, and my failing test is not the first of the "not finished" ones.
Found it: I've added the following line in the definition of TEST_METHOD():
Microsoft::VisualStudio::CppUnitTestFramework::Logger::WriteMessage(L#methodName);\
I have a collection of MsTest and NUnit tests that are being run in TFS 2015 using the VS Test step. I'm using NUnit Test Adapter 3.4.1 to run the tests on the build agent.
At the end, even though the tests pass, Nunit seems to croak and the build step fails with these two errors.
2016-09-04T09:59:44.7209787Z ##[error]Error: Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
2016-09-04T09:59:44.7209787Z ##[error]
2016-09-04T09:59:44.7209787Z ##[error]Error: Exception encountered unloading AppDomain
2016-09-04T09:59:44.7209787Z ##[error]
2016-09-04T09:59:44.7209787Z Information: NUnit Adapter 3.4.1.0: Test execution complete
2016-09-04T09:59:44.8615975Z Results File: C:\agent\_work2\1\TestResults\SRV-BLD1 2016-09-04 01_22_45.trx
2016-09-04T09:59:44.8615975Z Total tests: 139. Passed: 134. Failed: 0. Skipped: 5.
I've checked that there are indeed 139 tests in the suite, and 5 are set to ignore (2 are MSTest, and 3 are NUnit).
I'm not sure if there's a place to get more detailed explanation of the error. Searching this site and google seems to suggest that the NUnit.Engine.NUnitEngineException is linked to test discovery (here, and here for instance), but my tests are being discovered, so I'm not sure if this is related (Pretty new to NUnit, so not sure of a lot of things).
I also saw two links dealing with Adapter failures (here and here), but the errors don't quite match up, although maybe just because I don't have the same level of verbosity.
In TFS, the step doesn't have any configuration on it, just that it's enabled and the path to the DLLs.
Does anyone know what is causing the errors (which I'm assuming are causing the build to fail)? Alternatively, what should be the next steps in getting a more precise/verbose error stack to investigate the issue?
As a side note, I saw this SO answer, which states this:
MSTest.exe returned an exit code of 1 indicating that not all tests
passed.
I wasn't able to find any confirmation that VSTests fails when it encounters Skipped tests, but could this also be an issue?
Thank you for any help.
UPDATE
As suggested below, I tried running this from the IDE directly, and got this output (folders redacted)
------ Discover test started ------
NUnit Adapter 3.4.1.0: Test discovery starting
NUnit Adapter 3.4.1.0: Test discovery complete
========== Discover test finished: 139 found (0:00:00.8820879) ==========
------ Run test started ------
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
NUnit Adapter 3.4.1.0: Test execution started
Running all tests in C:\agent\_work2\1\s\codePorject\bin\Debug\codeProjectTests.dll
NUnit3TestExecutor converted 37 of 37 NUnit test cases
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
Exception encountered unloading AppDomain
NUnit Adapter 3.4.1.0: Test execution complete
========== Run test finished: 139 run (1:20:10.3290294) ==========
I've found a similar xUnit issue open, but it doesn't seem to have a solution.
This StackOverflow answer suggests to use a sleep timer, so I might try that.
In the end, it looks like some sort of a race condition between Firefox browser windows and nUnit. In my cleanup code, I'm killing firefox and iisexpress processes. Adding a sleep call has eliminated the issue:
public static void AssemblyCleanup()
{
foreach (var process in Process.GetProcessesByName("firefox")) process.Kill();
foreach (var process in Process.GetProcessesByName("iisexpress")) process.Kill();
System.Threading.Thread.Sleep(5000);
}
An NUnitEngineException is what it says: an exception that was discovered by the engine. It can be caused by many things and the message indicates the problem. In your case, the message says "Exception encountered unloading AppDomain" which means... well what it says.
The test adapter catches and handles the exception, producing the message you see. There is some indication that TFS also sees the exception and fails test run as a result. If you were to run this under the VS IDE, I think you would see the message from the adapter but the run would not fail. There is an NUnit3 Test Adapter issue about this, but it's not clear if the solution lies within the adapter or if it's a TFS problem.
If you do try this under the IDE I hope you will post the output window text from that run as well.
I have created simple MS Unit Test Project. Assert.Equal(true, true).
When I want to run a test I receive a error from Resharper and Visual Studio.
"Unit test Runner failed to run tests
System.Xml.XmlException: An Error occured while parsing EnityName. Line 1, position 17."
Do I missing something.
Edit:
I can't add screenshot and I can't select text to copy. Ok I will write it the best I can.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.xml.XmlTextReaderImpl.Throw(String res,String arg)
at System.Xml.XmlText.ParseEnityName()
at System.Xml.XmlTextReaderImplparseEnityReference
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextLoader.ParsePartialContent(XmlNode parentNode,String innerxmltext, xmlNodeType nt)
at System.Xml.XmlElement.set_InnerXml(string value)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingHelper.AppendOrModifyChild(XpathNavigator,String nodePatH,String nodeNAME,sTRING INNERxML)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.UpdatedRunConfiguration(XPathNavigator, ArchitectureeffectivePlatform,FrameworkVersion effectiveFramework,String resultsDirectory,String solutionDirectory,String binariesRootDirectory)
Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.MergeRunSettingsAndFindCompatibleSource()
at JetBrains.ReSharper.UnitTestProvider.MSTest11.MsTest11Runner.Run(IRemoteTaskServer server,String assemblyLocation,String runConfigurationFilename,IUnitTestRun run)
To solve this problem you don't need to reinstall Visual Studio or Resharper.
The problem was in a file path to my projects. This would explains why other Unit Test projects were fine because there were in a different location.
I guess this a bug inside Visual Studio Unit Testing Framework. If you have unit test project inside folders with special charts like '&' then Visual Studio will crash or Resharper will return this message.
I wonder where I can report this bug.
I have just run into this problem too and followed the advice above.
First my path had a folder named ...\m&e...
Then I copied the whole project folder to a new location without an '&' in the path.
After that everything was working well.
I have a C++ project in NetBeans using generated Makefiles. I set up a job in Jenkins (continuous integration server) to run the tests configured in NetBeans. Now Jenkins runs the tests and captures their output, but it considers the build successful even when a test fails.
I'm using the Boost Unit Test Framework which of course returns a non-zero code on failure as any proper *nix program would. So I wondered why Jenkins didn't understand when a test failed. Then I found this in the generated Makefile-Debug.mk from NetBeans:
# Run Test Targets
.test-conf:
#if [ "${TEST}" = "" ]; \
then \
${TESTDIR}/TestFiles/f1 || true; \
${TESTDIR}/TestFiles/f2 || true; \
else \
./${TEST} || true; \
fi
So it seems like they deliberately ignore the return value of all tests. But this doesn't make sense, because then what are your tests testing?
I tried to find a setting in NetBeans to say "Let failing tests break the build" but didn't find anything. I also tried to find a bug in the NetBeans tracker for this but didn't see any in my brief search.
Is there any other reasonable solution? I want Jenkins to fail my build if any test fails. Right now it only fails if a test fails to build, but if it builds and fails to run, success is reported.
It turns out that NetBeans (up to version 8 at least) cannot support this. What I did to work around it is to do make build-tests rather than make test in Jenkins, followed by a loop over all the generated test files (TestFiles/f* in the build directory) to run them.
This is a major shortcoming in NetBeans' Makefile generator, as it is fundamentally incompatible with running tests outside of NetBeans itself. Thanks to #HEKTO for the link which led me to this page about writing NetBeans testing plugins: http://wiki.netbeans.org/CND69UnitTestsPluginTutotial
What that page tells you is basically that NetBeans relies on parsing the textual output of tests to determine success or failure. What it doesn't tell you is that NetBeans generates defective Makefiles which ignore critical failures in tests, including aborts, segmentation faults, assertion failures, uncaught exceptions, etc. It assumes you will use a test framework that it knows about (which is only CppUnit), or manually write magic strings at the right moments in your test programs.
I thought about taking the time to write a NetBeans unit test plugin for the Boost Unit Test Framework, but it won't help Jenkins at all: the plugins are only used when tests are run inside NetBeans itself, to display pretty status indicators.