I've a c++ test class implemented using googletest, so it is perfectly passing on Xcode with following output:
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from test_fixture
[ RUN ] test_fixture.test_name
[ OK ] test_fixture.test_name (65609 ms)
[----------] 1 test from test_fixture (65609 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (65609 ms total)
[ PASSED ] 1 test.
But when I try to run the same test on terminal by using ctest command it is giving an strange error Unexpected format: with following output:
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from test_fixture
[ RUN ] test_fixture.test_name
Unexpected format: [ RUN ] test_fixture.test_name (it is always printing the above line)
<end of output>
Test time = 29.72 sec
For both Xcode and ctest, the same iOS simulator used, I don't understand why the test output giving conclusion as "FAILED" or "PASSED". I've searched a lot for that error and couldn't find any reasonable answers, so please help.
Update:
It has turned out the ios simulator tool ios-sim has a problem with apps that runs longer than 30 secs (I tried with a googletest class with three empty functions which has delay of 15 secs between them, app crashed on second one), simulator crashes the app but didn't find why yet.
It turns out iOS Simulator tool I was using https://github.com/phonegap/ios-sim.
I've emptied our test classes and added 3 test functions with delay of 15 secs between, if the app runs longer than 30 seconds, ios-sim crashes it.
The solution for this case is to use real devices with the similar project: https://github.com/phonegap/ios-deploy
Related
I have read a few topics about errors: cygwin_exception::open_stackdumpfile: Dumping stack trace to program.exe.stackdump
But I can't find something that fits my situation.
This is my project: https://github.com/InzynierDomu/LED_tester
I got error in UT:
Building...
Testing...
[==========] Running 9 tests from 3 test suites.
[----------] Global test environment set-up.
0 [main] program 751 cygwin_exception::open_stackdumpfile: Dumping stack trace to program.exe.stackdump
0 [main] program 751 cygwin_exception::open_stackdumpfile: Dumping stack trace to program.exe.stackdump
[----------] 3 tests from PWM_controller_test
[ RUN ] PWM_controller_test.active
[ OK ] PWM_controller_test.active (0 ms)
[ RUN ] PWM_controller_test.set_pwm_up
================================================================================================================================================================================= [FAILED]
I have very similar test in test/native/Color_tester_controller_test and test/native/PWM_controller_test. The first one works properly, the second not with that test.
TEST_F(PWM_controller_test, set_pwm_up)
{
EXPECT_CALL(hal_mock, set_PWM_output(1));
EXPECT_CALL(view_mock, update_duty());
uut.keyboar_reaction(Cursor_move::up);
}
I stack, and maybe anyone has some idea?
EDIT:
It's only Cygwin gcc problem, on MinGW gcc it's work fine /:
I have a project with unit tests and when I run ctest (cmake version 3.18.2 on macos), success is reported for all tests. However, if I run one of the tests by itself, it has exit status 1. As far as I know, this shouldn't happen, so what is causing this and how can I fix it?
The issue was a small careless mistake, but for the benefit of others running into the issue, I provided an answer for which the rest of the question can be skipped.
Unfortunately, I cannot reproduce this behavior with a smaller minimum working example. I will try to provide as much relevant information as possible, please let me know if I am missing something. Here is the code of the unit test:
#include "alignment_reader.h"
#define CATCH_CONFIG_MAIN
#define CATCH_CONFIG_COLOUR_NONE
#include "catch.h"
#include "string_conversions.h"
#include "exceptions.h"
namespace paste_alignments {
namespace test {
namespace {
SCENARIO("Test correctness of AlignmentReader::FromFile.",
"[AlignmentReader][FromFile][correctness]") {
GIVEN("The name of a valid input file.") {
std::string input_file{"test_data/valid_alignment_file.tsv"};
WHEN("Constructed with default number of fields and chunk size.") {
CHECK_NOTHROW(AlignmentReader::FromFile(input_file));
/*AlignmentReader reader{AlignmentReader::FromFile(input_file)};
THEN("Field number and chunk size are at default values.") {
CHECK(reader.NumFields() == 12);
CHECK(reader.ChunkSize() == 128 * 1000 * 1000);
}*/
}
}
}
} // namespace
} // namespace test
} // namespace paste_alignments
when I uncomment the commented part, the exit code changes to 2 and ctest still reports success.
Here is what happens when I run ctest (both with and without the commented portion):
$ ctest
Test project /Users/Jasper/cpp_projects/PasteAlignments/debug
Start 1: alignment_reader_test
1/1 Test #1: alignment_reader_test ............ Passed 0.16 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.17 sec
Here is what I get if I run the test individually and check exit status (with the stuff commented out):
$ ./test/alignment_reader_test
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
alignment_reader_test is a Catch v2.12.2 host application.
Run with -? for options
-------------------------------------------------------------------------------
Scenario: Test correctness of AlignmentReader::FromFile.
Given: The name of a valid input file.
When: Constructed with default number of fields and chunk size.
-------------------------------------------------------------------------------
/Users/Jasper/cpp_projects/PasteAlignments/test/alignment_reader_test.cc:64
...............................................................................
/Users/Jasper/cpp_projects/PasteAlignments/test/alignment_reader_test.cc:65: FAILED:
CHECK_NOTHROW( AlignmentReader::FromFile(input_file) )
due to unexpected exception with message:
Unable to open file: 'test_data/valid_alignment_file.tsv'.
===============================================================================
test cases: 1 | 1 failed
assertions: 1 | 1 failed
$ echo $?
1
The only things that change when I include the commented portion is that two tests fail and the exit status is 2 instead of 1.
Here with --verbose flag:
$ ctest --verbose
UpdateCTestConfiguration from :/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
Parse Config file:/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
UpdateCTestConfiguration from :/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
Parse Config file:/Users/Jasper/cpp_projects/PasteAlignments/debug/DartConfiguration.tcl
Test project /Users/Jasper/cpp_projects/PasteAlignments/debug
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
Start 1: alignment_reader_test
1: Test command: /Users/Jasper/cpp_projects/PasteAlignments/debug/test/alignment_reader_test
1: Test timeout computed to be: 1500
1: ===============================================================================
1: All tests passed (1 assertion in 1 test case)
1:
1/1 Test #1: alignment_reader_test ............ Passed 0.36 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.36 sec
I didn't change the ctest configurations (as suggested here). CMakeLists.txt looks like this:
add_executable(alignment_reader_test
"${PROJECT_SOURCE_DIR}/test/alignment_reader_test.cc"
"${PROJECT_SOURCE_DIR}/src/alignment_reader.cc"
"${PROJECT_SOURCE_DIR}/src/alignment_batch.cc"
"${PROJECT_SOURCE_DIR}/src/scoring_system.cc"
"${PROJECT_SOURCE_DIR}/src/alignment.cc"
"${PROJECT_SOURCE_DIR}/src/helpers.cc")
target_include_directories(alignment_reader_test PUBLIC
"${PROJECT_SOURCE_DIR}/test"
"${PROJECT_SOURCE_DIR}/include"
"${PROJECT_SOURCE_DIR}/lib/catch/include")
add_test(NAME alignment_reader_test COMMAND alignment_reader_test)
I found the issue thanks to #Tsyvarev.
The build directory structure is as follows:
build/
|--test/
|--test_executable
|--test_data/
|--test_datafile
When CMake runs test_executable it runs it from within the directory test.
When I ran the executable separately, I ran it from the build directory. (./test/test_executable).
Inside the unit-test code, the test_datafile is referred to as test_data/test_datafile. This is not recognized when the test is run from the build executable (as I did) as opposed to when it's run from the build/test directory.
Therefore, when ctest ran the test, it actually succeeded as it should.
Indeed, if I cd into test first, the test has exit code 0, as it should.
[----------] 2 tests from VideoMgrTest
[Blast] Log directory is: C:\ProgramData\VMware\VMware Blast
[ RUN ] VideoMgrTest.createVideoMgr
unknown file: error: SEH exception with code 0xc0000005 thrown in TearDown().
[ FAILED ] VideoMgrTest.createVideoMgr (1 ms)
[ RUN ] VideoMgrTest.getFBAddress
unknown file: error: SEH exception with code 0x5 thrown in the test body.
[ FAILED ] VideoMgrTest.getFBAddress (3 ms)
[----------] 2 tests from VideoMgrTest (8 ms total)
[----------] Global test environment tear-down
[==========] 10 tests from 4 test cases ran. (226 ms total)
[ PASSED ] 8 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] VideoMgrTest.createVideoMgr
[ FAILED ] VideoMgrTest.getFBAddress
I am getting a failure in Google UT. I have provided the funtion definition as given below. Ita failing with a SEG exception. I know it is access denied but i dont know what is being acceseed
void VideoMgrTest::TearDown()
{
VideoProducerInstance::DestroyInstance();
mVideoMgr = NULL;
}
Runtime exceptions are hard to pin down, and the code snipped you provided are unfortunately insufficient to be of any help.
If you are using Visual Studio to run your unit tests (e.g. using the google test adapter plugin), remember that you CAN debug those tests and that you can enable the capture of run time exections (they are not captured by default). The answer given to a similar question may be of help to you here.
In any case, if I had to give a guess, I'd suspect that a access of a nullpointer is the source of your troubles.
I use GoogleTest extensively and the output in the cosole looks like this:
When I run the same from within Eclipse (using Eclipse Mars latest as of today) I get the following output with no colors and junk characters (are not visible just pasting the output here):
Running main() from gtest_main.cc
[0;32m[==========] [mRunning 3 tests from 1 test case.
[0;32m[----------] [mGlobal test environment set-up.
[0;32m[----------] [m3 tests from NloptAdapterSuite
[0;32m[ RUN ] [mNloptAdapterSuite.testQuadraticFunction1
[0;32m[ OK ] [mNloptAdapterSuite.testQuadraticFunction1 (1 ms)
[0;32m[ RUN ] [mNloptAdapterSuite.testQuadraticFunction1WithNoise
[0;32m[ OK ] [mNloptAdapterSuite.testQuadraticFunction1WithNoise (1 ms)
[0;32m[ RUN ] [mNloptAdapterSuite.testQuadraticFunction2
[0;32m[ OK ] [mNloptAdapterSuite.testQuadraticFunction2 (1 ms)
[0;32m[----------] [m3 tests from NloptAdapterSuite (3 ms total)
[0;32m[----------] [mGlobal test environment tear-down
[0;32m[==========] [m3 tests from 1 test case ran. (3 ms total)
[0;32m[ PASSED ] [m3 tests.
How can this be fixed to have the same output?
The ANSI Escape in Console plugin that greg-449 linked above worked great for me. After installing it, the gtest output looks like:
I'm on Neon.2 (Eclipse 4.6.2).
I'm running into this weird problem with Google Test using Visual Studio 2012. I can successfully set up tests and they run just fine. However, when I make change to any code in the test project and build the project, I receive LNK2005 errors, like this one:
Error 3 error LNK2005: "private: virtual void __thiscall writeStreamTest_writeRandomDouble_Test::TestBody(void)" (?TestBody#writeStreamTest_writeRandomDouble_Test##EAEXXZ) already defined in UtilsTest.obj C:\Users\cye\Desktop\codeCPP\unitTest\unitTest.obj unitTest
However, when I change the name of the test, the test runs fine again. But it not only runs the tests after the name change, but also the tests with the old names. Like this:
[==========] Running 4 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 4 tests from writeStreamTest
[ RUN ] writeStreamTest.writeRandomDouble
[ OK ] writeStreamTest.writeRandomDouble (0 ms)
[ RUN ] writeStreamTest.writeRandomInt
[ OK ] writeStreamTest.writeRandomInt (0 ms)
[ RUN ] writeStreamTest.writeRandomDouble2
[ OK ] writeStreamTest.writeRandomDouble2 (0 ms)
[ RUN ] writeStreamTest.writeRandomInt2
[ OK ] writeStreamTest.writeRandomInt2 (0 ms)
[----------] 4 tests from writeStreamTest (3 ms total)
[----------] Global test environment tear-down
[==========] 4 tests from 1 test case ran. (5 ms total)
[ PASSED ] 4 tests.
It appears that the tests that were included in the last build would be left over. Therefore when I build the code with the same test name, compiler would complaint that the test with the same name already exists.
If I try to comment out all the tests, then call RUN_ALL_TESTS(), the tests that were included in the last build would still run. If I make change and build again, no test will run.
I'm very confused about this issue. Even after I clean and rebuild the project this issue would still occur.
Anyone else ran into this problem before? Thanks very much!