googletest: How to skip test from an inner method - c++

With GTEST_SKIP, we can (conditionally) skip a test. The way it is implemented is as follows:
#define GTEST_SKIP_(message) \
return GTEST_MESSAGE_(message, ::testing::TestPartResult::kSkip)
If we are in the main test body, all is fine.
How can I skip a test from deep within my code base? This does not work:
int skip_this(int x) {
if (x == 3) GTEST_SKIP();
return x;
}
TEST(X, Y) {
skip_this(2);
skip_this(3);
throw std::runtime_error("Did not skip");
}
Error:
/Users/markus/Projekte/Opossum/Git/src/test/operators/aggregate_test.cpp:183:15: error: cannot initialize return object of type 'int' with an rvalue of type 'void'
if (x == 3) GTEST_SKIP();
^~~~~~~~~~~~
In other words: How can I modify skip_this so that the test passes or is marked as "skipped"?

This is not really nice, but works for my purpose. The idea is to unwind the stack, search for the googletest method that called the test and reset the instruction pointer to that stack frame.
Note that the stack is not properly destroyed, meaning that, e.g., resources can leak.
#define UNW_LOCAL_ONLY
#include <libunwind.h>
#include <cxxabi.h>
#include <iostream>
#include <gtest/gtest.h>
void skip() {
unw_cursor_t cursor;
unw_context_t context;
unw_getcontext(&context);
unw_init_local(&cursor, &context);
while (unw_step(&cursor)) {
unw_word_t off;
char symbol[256] = {"<unknown>"};
char* name = symbol;
if (!unw_get_proc_name(&cursor, symbol, sizeof(symbol), &off)) {
int status;
if ((name = abi::__cxa_demangle(symbol, nullptr, nullptr, &status)) == nullptr) name = symbol;
}
if (std::string(name) == "testing::Test::Run()") {
::testing::internal::AssertHelper(::testing::TestPartResult::kSkip, __FILE__, __LINE__, "skipped") = ::testing::Message();
unw_resume(&cursor);
}
if (name != symbol) free(name);
}
throw std::runtime_error("Did not find test method on the stack, could not skip test");
}
Example test (can be pasted below the code above):
int foo(int x) {
if (x == 3) skip();
return 11;
}
TEST(SkipTest, ShouldNotBeSkipped) {
foo(2);
EXPECT_TRUE(false);
}
TEST(SkipTest, ShouldBeSkipped) {
foo(3);
EXPECT_TRUE(false);
}
TEST(SkipTest, TestExecutionContinues) {
EXPECT_FALSE(false);
}
int main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
Output:
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from SkipTest
[ RUN ] SkipTest.ShouldNotBeSkipped
skip_test.cpp:44: Failure
Value of: false
Actual: false
Expected: true
[ FAILED ] SkipTest.ShouldNotBeSkipped (0 ms)
[ RUN ] SkipTest.ShouldBeSkipped
[ SKIPPED ] SkipTest.ShouldBeSkipped (0 ms)
[ RUN ] SkipTest.TestExecutionContinues
[ OK ] SkipTest.TestExecutionContinues (0 ms)
[----------] 3 tests from SkipTest (0 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (0 ms total)
[ PASSED ] 1 test.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] SkipTest.ShouldBeSkipped
[ FAILED ] 1 test, listed below:
[ FAILED ] SkipTest.ShouldNotBeSkipped
1 FAILED TEST

You cant use GTEST_SKIP in the inner function. It will not take effect. I can only offer this solution or something similar:
#define SKIPCODE -1
int skip_this(int x)
{
if (x == 3)
return SKIPCODE;
else return x;
}
TEST(Test)
{
int n = 3;
if (skip_this(n) == SKIPCODE)
GTEST_SKIP();
ASSERT_TRUE(false);
}
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

Related

Is there a way to detect command line arguments not running tests?

Using the googletest framework I want to write my own main function. Basically some custom initialization step needs to happen before RUN_ALL_TESTS is called. I'd like to skip this step, if the command line parameters for googletest indicate, no tests should be run (e.g. if --gtest_list_tests is passed).
Is it possible to retrieve this kind of information from the test framework without the need to parse the parameters myself?
What I'd like to accomplish:
#include <gtest/gtest.h>
bool RunAllTestsDoesNotRunTests()
{
// should return false, if and only if RUN_ALL_TESTS() runs any test cases
// implementation?
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
if (!RunAllTestsDoesNotRunTests())
{
DoExpensiveInitialization();
}
return RUN_ALL_TESTS();
}
Is this even possible? I've tried to identify members of ::testing::UnitTest, that would allow me to retrieve this kind of information, without any success so far though.
Am I going at this the wrong way? Preferrably I'd like to avoid lazily doing the initialization via fixture or similar logic requiring me to adjust every test case.
There are various flags that might prevent google test from running any tests. I can think of at least the following:
--gtest_list_tests is passed.
--gtest_repeat=0 is passed.
--help or other forms of it like -h or any unrecognized flag with gtest prefix is passed See here.
You can test these cases by:
The first can be checked using ::testing::GTEST_FLAG(list_tests). See here.
The second can be tested using ::testing::GTEST_FLAG(repeat).
The third can be checked by reading the global variable g_help_flag. While practical, this is not ideal because it's in the internal namespace and might change in future releases.
Another alternative is to parse the command line arguments yourself.
So assuming you want to be practical, one way to get to what you want is this:
// Defining g_help_flag as an extern variable
namespace testing {
namespace internal {
extern bool g_help_flag;
}
}
bool RunAllTestsDoesNotRunTests()
{
// should return false, if and only if RUN_ALL_TESTS() runs any test cases
// implementation?
return ( ::testing::GTEST_FLAG(list_tests) ||
::testing::GTEST_FLAG(repeat) == 0 ||
::testing::internal::g_help_flag);
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
if (!RunAllTestsDoesNotRunTests())
{
// DoExpensiveInitialization();
std::cout << "Expensive initialization!" << std::endl;
} else {
std::cout << "No Expensive initialization!" << std::endl;
}
return RUN_ALL_TESTS();
}
Live example: https://godbolt.org/z/P36fde11T
command line arguments are detected using the GTEST_FLAG macro. An example of what you're trying to do might look like:
#include <gtest/gtest.h>
TEST(equality, always_passes) {
EXPECT_TRUE(true);
}
bool RunAllTestsDoesNotRunTests()
{
return ::testing::GTEST_FLAG(list_tests);
}
void DoExpensiveInitialization() {
std::cout << "Boop Boop, Beep Beep" << std::endl;
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
if (!RunAllTestsDoesNotRunTests())
{
DoExpensiveInitialization();
}
return RUN_ALL_TESTS();
}
When compiled and linked appropriately, it can be run as:
$ ./a.out
Boop Boop, Beep Beep
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from equality
[ RUN ] equality.always_passes
[ OK ] equality.always_passes (0 ms)
[----------] 1 test from equality (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 1 test.
$ ./a.out --gtest_list_tests
equality.
always_passes
i.e. you don't see the Boop Boop, Beep Beep as the function was not called.
Now if you have something that you want to have run once, if you're running tests, then adding it to the testing::Environment would also do the trick:
#include <gtest/gtest.h>
class MyEnvironment: public ::testing::Environment
{
public:
virtual ~MyEnvironment() = default;
// Override this to define how to set up the environment.
virtual void SetUp() { std::cout << "Env Beep Beep" << std::endl; }
// Override this to define how to tear down the environment.
virtual void TearDown() {}
};
TEST(equality, always_passes) {
EXPECT_TRUE(true);
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
auto* env = new MyEnvironment();
::testing::AddGlobalTestEnvironment(env);
return RUN_ALL_TESTS();
}
When executed, it will also cope with filters (the previous example would not be able to cope in that case):
$ ./a.out --gtest_filter=equality\*
Note: Google Test filter = equality*
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
Env Beep Beep
[----------] 1 test from equality
[ RUN ] equality.always_passes
[ OK ] equality.always_passes (0 ms)
[----------] 1 test from equality (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 1 test.
$ ./a.out --gtest_filter=equality\* --gtest_list_tests
equality.
always_passes
Again, not that the Env Beep Beep does not appear if you don't run any tests (you can check with a filter like --gtest_filter=equality, and you won't see the Env output in that case.

When there is a failure in ASSERT_ macro of gtest, the TearDown function will be called or not?

Will the TearDown function be called when the ASSERT_EQ fails in google test? Or it just cancel that test case and move to the next one without TearDown? This is because in my TearDown function, I need to do something to properly shutdown the test function, so I'm afraid that this ASSERT will make my test not independent.
It's not hard to satisfy yourself that TearDown is always run, regardless
of whether an ASSERT_... macro fails:
testcase.cpp
#include <gtest/gtest.h>
#include <iostream>
struct foo : ::testing::Test
{
void SetUp()
{
std::cout << ">>>" << __PRETTY_FUNCTION__ << " was run " << std::endl;
}
void TearDown()
{
std::cout << ">>>" << __PRETTY_FUNCTION__ << " was run " << std::endl;
}
};
TEST_F(foo,bar)
{
ASSERT_EQ(1,0);
}
int main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
Compile and link:
g++ -o testcase testcase.cpp -lgtest -pthread
Run:
$ ./testcase
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from foo
[ RUN ] foo.bar
>>>virtual void foo::SetUp() was run
testcase.cpp:19: Failure
Expected equality of these values:
1
0
>>>virtual void foo::TearDown() was run
[ FAILED ] foo.bar (0 ms)
[----------] 1 test from foo (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] foo.bar
1 FAILED TEST

Test Cases states Failed but value returned is true

I have written up a simple sample code to understand GMOCK used for unit testing:
#include <iostream>
#include "gmock/gmock.h"
#include "gtest/gtest.h"
using ::testing::AtLeast;
class A {
public:
void ShowPub1()
{
std::cout << "A::PUBLIC::SHOW..1" << std::endl;
}
int ShowPub2(int x)
{
std::cout << "A::PUBLIC::SHOW..2" << std::endl;
return true;
}
};
class MockA : public A{
public:
MOCK_METHOD0(ShowPub1, void());
MOCK_METHOD1(ShowPub2, int(int x));
};
Below is my test code - I just wish to call ShowPub2 method of class A. I was expecting the statement A::PUBLIC::SHOW..2 to get printed at console - but it just did not happen and the test case also failed though the method is hardcoded to return true:
TEST(FirstA, TestCall) {
MockA a;
EXPECT_CALL(a, ShowPub2(2))
.Times(AtLeast(1));
a.ShowPub2(2);
EXPECT_TRUE(a.ShowPub2(2));
}
GMOCK test code execution output - I am not sure why the output A::PUBLIC::SHOW..2 did not rendered in console and test case failed:
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from FirstA
[ RUN ] FirstA.TestCall
c:\users\user1\documents\c and c++\gmock\gmock1\gmock1\gmock1.cpp(78): error:
Value of: a.ShowPub2(2)
Actual: false
Expected: true
[ FAILED ] FirstA.TestCall (0 ms)
[----------] 1 test from FirstA (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] FirstA.TestCall
1 FAILED TEST
Press any key to continue . . .
int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
return 0;
}
Some clarification is needed here..
Creating a Mock class means that the Mock methods are generate with its own implementations. These line of code
MOCK_METHOD0(ShowPub1, void());
MOCK_METHOD1(ShowPub2, int(int x));
doesn't mean it will call parent implementation of ShowPub1 / ShowOub2. This only means that you'll get a function (Mock) with same signature as that of the class you're mocking.
The test fails because of this line
EXPECT_TRUE(a.ShowPub2(2));
Since the original implementation is not called, this function fails.
The test function should be written as
TEST(FirstA, TestCall) {
MockA a;
EXPECT_CALL(a, ShowPub2(2)) .Times(AtLeast(1));
a.ShowPub2(2);
}
Here you're testing the behavior that the function is called at least once and the test will be successful with that.
If you want to test the functionality of a function, write separate tests. Mocking will not solve the problem in this case.

Is it possible to mock a called function return value using gtest or gmock?

I am new to gtest and gmock please let me understand how to mock the called function. Which will also help me in code coverage.
#include <stdio.h>
#include "gtest/gtest.h"
int ret()
{
return 5;
}
class A
{
public:
A();
int a;
int func();
};
A::A()
{
printf("This is constructor\n");
}
int A::func()
{
int iRet = ret(); /* When func() is called by gtest I would like to control the return value of ret() */
int iRet1 = ret();
/* Based on these two values some operations to be done */
printf("This is func. %d, %d\n", iRet, iRet1);
return iRet;
}
TEST (abc, xyz) {
A a;
EXPECT_EQ(5, a.func()); /* Here how to get different values of iRet and iRet1 for ret() function? */
}
int main(int argc, char **argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
In case if it is not possible through gtest and/or gmock, please suggest me any other tool for the same.
Also I tried with the following thing which is not working as expected:
int ret()
{
printf("Returning 5\n");
return 5;
}
int ret1()
{
int iRet = 10;
printf("Inside ret1\n");
iRet = ret();
if (iRet == 5)
{
printf("Original ret is called\n");
}
else if (iRet == 100)
{
printf("This is mocked function call\n");
}
else
{
printf("Opps! This should not happen\n");
}
return iRet;
}
class FooMock {
public:
MOCK_METHOD0(ret, int());
MOCK_METHOD0(ret1, int());
};
TEST (abc, xyz) {
FooMock mock;
EXPECT_CALL(mock, ret()).Times(1).WillOnce(Return(100));
mock.ret1();
}
int main(int argc, char **argv)
{
::testing::InitGoogleMock(&argc, argv);
return RUN_ALL_TESTS();
}
While running giving me the following error:
$ ./a.out
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from abc
[ RUN ] abc.xyz
gtest.cpp:86: Failure
Actual function call count doesn't match EXPECT_CALL(mock, ret())...
Expected: to be called once
Actual: never called - unsatisfied and active
gtest.cpp:87: Failure
Actual function call count doesn't match EXPECT_CALL(mock, ret())...
Expected: to be called once
Actual: never called - unsatisfied and active
[ FAILED ] abc.xyz (0 ms)
[----------] 1 test from abc (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] abc.xyz
1 FAILED TEST
Please let me know if anything I am doing wrong...
So, the idea of unit-testing is that you run some of your code and that you compare the results to what you expect to be the result. Your code makes it more complicated than it has to be. E.g.
#include <stdio.h>
#include "gtest/gtest.h"
int incBy5(const int a) {
return a+5;
}
TEST (abc, xyz) {
int x=17;
EXPECT_EQ(x+5, incBy5(x));
}
int main(int argc, char **argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
See the documentation of unit testing concepts in general and google-test in particular. The mocking aspect of it all comes into place with more complex things to test where you may not have everything readily available (e.g. network resources, databases, ...) and you create mock-ups that can replace these resources.
In your second snippet, you define two mock methods, ret and ret1. Later, you set up an expectation on ret but invoke ret1. At the end of the test, the expectation on ret remains unfulfilled, causing gmock to complain.

googletest SetUp Method not called

I'm using Google Test to unit test my C++ project. The getting started guide says:
If necessary, write a default constructor or SetUp() function to prepare the objects for each test. A common mistake is to spell SetUp() as Setup() with a small u - don't let that happen to you.
SetUp() is spelled correctly, but I still can't get SetUp to work. Any ideas?
#include "gtest/gtest.h"
class SampleTest : public ::testing::Test {
protected:
virtual void SetUp() { std::cout << "SetUp called." << std::endl; }
};
TEST(SampleTest, OneEqualsOne) {
int one = 1;
ASSERT_EQ(1, one);
}
int main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
g++ -g -Wno-deprecated -I gtest/include SampleTest.cpp gtest/libgtest.a -o SampleTest
Output:
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SampleTest
[ RUN ] SampleTest.OneEqualsOne
[ OK ] SampleTest.OneEqualsOne (1 ms)
[----------] 1 test from SampleTest (1 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1 ms total)
[ PASSED ] 1 test.
Change TEST to TEST_F, as SetUp methods and such are called with TEST_F, but not with TEST alone.
Change your TEST macro to TEST_F. (It's listing in the documentation right underneath the quote you provided.)