I am testing my c++11 static lib in vs 2013 environment.
I followed this nice tutorial and testing + code coverage are working fine.
Now I need step-by-step : I added breakpoints and executed the "Debug Selected Test" command and I get the following message : vstest.executionengine.exe has triggered a breakpoint and the callstack brings me at CppUnitTest.h:465 : (static_cast<ThisClass *>(this)->*method2)();, eg at the root call of the method I want to break into. No way to see the code inside this call.
My question : how to break into my code during debugging Ms cpp unit test ?
I found the problem. Some breakpoints were activated inside the static lib used by the test dll. It seems breakpoints inside static lib generates something wrong for the test framework.
To reproduce :
create c++ static lib project with a simple function, int foo(){ return 0;}
create a test-dll project
add a test function which calls foo
set a breakpoint in foo
execute "run selected test" : it will work
execute "debug selected test" : it will block just before calling the test-method test-generated class.
I am still interested in explaining this non-intuitive behaviour.
Related
Trying out C++ unit testing in Visual Studio 2019. Have added a C++ Native Unit Test Project unit test project to my solution. Then added the following Logger messages:
TEST_CLASS_INITIALIZE(ClassInitialize)
{
Logger::WriteMessage("Setup on test class. Where will this appear?");
}
// ...
TEST_METHOD(TestMethod1)
{
Logger::WriteMessage("Test method. Where is this message going??");
}
The tests are picked up by the Visual Studio Test Explorer and run successfully. Breakpoints are also hit if the tests are debugged. According to the Microsoft documentation on Logger, output should appear in the Visual Studio Output Window, but the messages are not being displayed.
Where is Logger sending its' output?
Although the MS documentation states "The logger class contains static methods to write to the Output Window" this no longer seems to be the case.
The output from the Logger Class can now be seen in the Test Detail Summary, after the unit tests have been run.
Output from the Logger::WriteMessage that is contained in TEST_METHOD(TestMethod1) will appear in the Test Detail Summary for that given unit test.
Output from the Logger::WriteMessage that is contained in TEST_CLASS_INITIALIZE(ClassInitialize) will appear in the first test method that is executed during that test run.
I am writing to run a test case locally, while doing so. thr code never halts at the breakpoints set into the #processElement . I tried Alt+shift+F7 but failed to break inside the code.
This question is related to the problem where my unit test procedure is crashing but I don't know on which unit test.
I realise this question is a duplicate of How can I find out which test method in a batch of test methods fails to run? but I need to try anyway:
I'm running some unit tests (about 118 of them), but one of them seems to make the Visual Studio unit test environment crash. This is what I see in the "output" window's "Tests" tab:
[5/02/2018 11:13:18 Informational] ------ Run test started ------
[5/02/2018 11:13:38 Error] The active Test Run was aborted because the execution process exited unexpectedly. The test execution process crashed while running the tests. To investigate further, open file:///C:/Users/DominiqueDS/AppData/Local/CrashDumps/vstest.executionengine.x86.exe.19136.dmp file in Visual Studio and choose "Debug in mixed mode".
[5/02/2018 11:13:38 Informational] ========== Run test finished: 65 run (0:00:19,6926824) ==========
The reason that I have that dump file is thanks to my Windows configuration which automatically creates such dumpfiles in case of a crashing application. (The procedure for this configuration is found under this URL)
Examining the dump file gives me more information on the test, which is failing: I have a hunch where I can find it, and in another thread, I find a function, calling a method which seems to contain the name of the unit test I'm running.
Although I know which test is failing, it's based on hunches and on dumpfiles I should even not generate, so I'm looking for another solution:
In the function TEST_METHOD in the file "c:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\VS\UnitTest\include\CppUnitTest.h", I've tried to add the following lines:
cout << "Test : ";\
cout << methodName;\
cout << "\n";\
I was hoping that this would show every test I wanted to run, but instead I get the error message that this is wrong (also using std::cout and OutputDebugString() function seems not to be allowed).
Hence my question: does anybody know a way to find out, in case one of my unit tests makes my test environment crash, how can I find out which test this is, without needing to generate and debug the Visual Studio dumpfile?
For your information: looking in the "Test Explorer" is not helping: I have a list of disabled tests, a list of successful ones, and a list of some which are not finished, and my failing test is not the first of the "not finished" ones.
Found it: I've added the following line in the definition of TEST_METHOD():
Microsoft::VisualStudio::CppUnitTestFramework::Logger::WriteMessage(L#methodName);\
I've added tests (XCTests) to an existing C++ command line app in Xcode 5
via Test Navigator > (+),
changed the extension of the test class to .mm,
added the XCTest framework to the project.
All compiles fine. Now running the tests just gives me a 'Test failed' message, nothing in the console and neither green nor red lights in the Navigator (i.e. no tests executed).
Starting with a fresh Xcode 5 project and changing the extension of the test class to .mm just works fine so I'd assume it's not just about lacking support for Objective-C++ in XCTest.
Even with a plain, vanilla test target added to the existing C++ project the tests fail before ever running.
Any more gotchas to watch out for when adding XCTests to existing (Objective-)C++ targets?
Update #1
With Xcode 5.0.2 (on 10.8.5) xctest now crashes in the same scenario with an
*** NSTask: Task create for path '/Users/XXX/Library/Developer/Xcode/DerivedData/RM_Next_Gen-gpihzjouhxvifqcslmywktktizer/Build/Products/Debug/YYY Tests.xctest/Contents/MacOS/YYY Tests' failed: 22, "Invalid argument". Terminating temporary process.
objc[3478]: GC: forcing GC OFF because OBJC_DISABLE_GC is set
*** multi-threaded process forked ***
Turns out it was a command line C++ project which apparently isn't currently supported by XCTest. Testing the C++ code from an Objective-C(++) project works just fine..
Just hoping for better documentation of XCTest at some point in the (near) future.
I couldn't find any documentation on using XCTest to test code that is not Objective C (C / C++). I am wanting to test a command line C project, and I was concerned that your answer suggested that doing this is unsupported by Xcode. I'm a novice to XCTest and TDD, but I thought others might appreciate what I discovered.
I created a New Project > Command Line Tool, based on C. Project is called foo; Xcode will call the target foo by default too.
-Files created: main.c
Add test target by going to Test Navigator > (+) > New Test Target. Call the target fooTests. Xcode will create this target and an example test within it (all within the file called fooTests.m). The example test will be called testExample and it will contain the test function below:
- (void)testExample
{
XCTFail(#"No implementation for \"%s\"", __PRETTY_FUNCTION__);
}
If you run the tests now (either from the Product>Test menu item or ⌘U) then the test will fail (it's supposed to). For the sake of brevity, I won't go through the TDD cycle, but simply list the files and contents that need to be added to test functions that are not within Objective-C classes.
Let's say that we have a function that we want to test, called bar(), and called from main.c. Go to the project file explorer and add a new header file called bar.h. Add the function prototype:
int bar();
If you want to use XCTest to test a function, it can't be in main.c, so add it to a new C Source file called bar.c. Important: when you add the file, you must add it to the target foo and to fooTests. if you don't add it to fooTests then when compiling the tests the linker won't find the function.
int bar() {
return 0;
}
Within fooTests.m, add the include:
#include "bar.h"
You can now refer to any function contained in bar.h within your tests. Now create 2 unit tests, one that will fail, the other that will pass.
Delete the method testExample and replace with the following code:
- (void)testBarWillFail
{
int rc = bar();
XCTAssertTrue(rc == -1, #"Expected rc==-1, rc==%d", rc);
}
- (void)testBarWillPass
{
int rc = bar();
XCTAssertTrue(rc == 0, #"Expected rc==0, rc==%d", rc);
}
Running the tests will produce the following in the debug window:
Test Suite 'All tests' finished at 2014-01-12 13:38:45 +0000.
Executed 2 tests, with 1 failure (0 unexpected) in 0.000
Within the Issue Navigator pane you'll see:
file: /foo/fooTests/fooTests.m: test failure: -[fooTests testBarWillFail] failed: ((rc == -1) is true) failed - Expected rc==-1, rc==0
This is all as expected! Repeat the process with all other C code you wish to test.
If anyone knows a better way of doing this then I'd be delighted to hear it!
I'm making use of the TestDriven.NET plugin for visual studio to run all my tests to check for code coverage with NDepend. All is working fine apart from my tests with a HostType attribute. These aren't getting run.
Does anybody knopw how to fix this or is there a specific reason for it?
Thanks in advance
If you look at the release notes for TestDriven.NET, they explicitly skip test with the HostType attribute. I'm not sure why they do this exactly.
I can actually not find another test runner outside of the built in runner in Visual Studio that will execute those tests, Resharper/dotCover gives me an Aborted message when running a specific test with a HostType
EDIT: Linked to release notes (look at item 3203) http://www.testdriven.net/downloads/releasenotes.html