One of my favorite unit testing frameworks is PHPUnit because it supports test dependencies (i.e. the ability to mark tests as dependent upon other tests, running the dependent tests conditionally on the success of their dependencies). I've been using the Boost testing framework more recently to test my C++ code, and while it suits most of my unit testing needs, it doesn't appear to support test dependencies.
I've scoured the documentation for the Boost testing framework and have found various hints that Boost supports this feature, but I've yet to find a documentation page or any concrete examples of test dependency support in Boost. Are the previously given pages red herrings, or does the Boost testing framework actually support test dependencies?
Well, you've found yourself that the feature is there and according to the links, it's there for end user to use.
It's not documented by accident and "should be added soon" (as per links).
Now, here's a nother post I've found which uses feature:
http://boost.2283326.n4.nabble.com/Unit-Test-Framework-strange-behaviour-of-test-unit-depends-on-td2653654.html
Sample from there (unfortunately, looks like no BOOST_AUTO_TEST_CASE works there).
Also note that code is incorrect because Dependency() is never called and therefore dependent test doesn't run as well.
#include <boost/test/included/unit_test.hpp>
using boost::unit_test::test_suite;
void Dependency()
{
BOOST_MESSAGE( "Dependency!" );
BOOST_CHECK( 1 );
}
void TC_TestCase()
{
BOOST_MESSAGE( "A test case!" );
BOOST_CHECK( 1 );
}
test_suite*
init_unit_test_suite( int argc, char* argv[] )
{
test_suite* ts = BOOST_TEST_SUITE( "Test_Test" );
ts->add( BOOST_TEST_CASE( &TC_TestCase ) );
/*1*/ ts->depends_on( BOOST_TEST_CASE( &Dependency ) );
return ts;
}
Update
Performed some experimenting and here's an example with automatic test/suits and dependencies.
Some notes about code:
Boost here is 1.42, there may be slight differences for newer versions.
If you put test_suite2 after test_suite1 in the cpp file, keeping dependencies the same, test_suite1 will be always skipped because test_suite2 is not run before it.
I made test_case4 to fail so that test_suite1 is skipped, but if test_case4 succeeds, test_suite1 does execute.
I'm pretty sure you'll be able to make dependencies registration much prettier and shorter.
The code:
#include <boost/test/included/unit_test.hpp>
using namespace boost::unit_test;
BOOST_AUTO_TEST_SUITE(test_suite2)
BOOST_AUTO_TEST_CASE(test_case4)
{
BOOST_CHECK(false);
}
BOOST_AUTO_TEST_SUITE_END()
BOOST_AUTO_TEST_SUITE(test_suite1)
BOOST_AUTO_TEST_CASE(test_case1)
{
BOOST_CHECK(true);
}
BOOST_AUTO_TEST_CASE(test_case2)
{
BOOST_CHECK(true);
}
BOOST_AUTO_TEST_SUITE_END()
//____________________________________________________________________________//
test_suite*
init_unit_test_suite( int argc, char* argv[] )
{
const auto testSuite1Id = framework::master_test_suite().get("test_suite1");
if( testSuite1Id != INV_TEST_UNIT_ID ) {
auto test_suite1 = &framework::get<test_suite>( testSuite1Id );
const auto testSuite2Id = framework::master_test_suite().get("test_suite2");
if (testSuite2Id != INV_TEST_UNIT_ID) {
auto testSuite2 = &framework::get<test_suite>( testSuite2Id );
const auto testCase4Id = testSuite2->get("test_case4");
if (testCase4Id != INV_TEST_UNIT_ID) {
// test_suite1 depends on test_suite2/test_case4
test_suite1->depends_on( &framework::get<test_case>( testCase4Id ));
}
}
}
return 0;
}
Output:
Running 3 test cases...
Entering test suite "Master Test Suite"
Entering test suite "test_suite2"
Entering test case "test_case4"
<blahblahblah>/consoleapplication5/consoleapplication5.cpp(10): error in "test_case4": check false failed
Leaving test case "test_case4"; testing time: 14ms
Leaving test suite "test_suite2"
Test suite "test_suite1"is skipped
Leaving test suite "Master Test Suite"
*** 1 failure detected in test suite "Master Test Suite"
Starting on Boost 1.59 you have better tools for this: depends_on: http://www.boost.org/doc/libs/1_59_0/libs/test/doc/html/boost_test/utf_reference/test_org_reference/decorator_depends_on.html
Usage is pretty simple (from Boost docu):
namespace utf = boost::unit_test;
BOOST_AUTO_TEST_CASE(test3, * utf::depends_on("s1/test1"))
{
BOOST_TEST(false);
}
BOOST_AUTO_TEST_CASE(test4, * utf::depends_on("test3"))
{
BOOST_TEST(false);
}
BOOST_AUTO_TEST_SUITE(s1)
BOOST_AUTO_TEST_CASE(test1)
{
BOOST_TEST(true);
}
BOOST_AUTO_TEST_SUITE_END()
Related
Using the googletest framework I want to write my own main function. Basically some custom initialization step needs to happen before RUN_ALL_TESTS is called. I'd like to skip this step, if the command line parameters for googletest indicate, no tests should be run (e.g. if --gtest_list_tests is passed).
Is it possible to retrieve this kind of information from the test framework without the need to parse the parameters myself?
What I'd like to accomplish:
#include <gtest/gtest.h>
bool RunAllTestsDoesNotRunTests()
{
// should return false, if and only if RUN_ALL_TESTS() runs any test cases
// implementation?
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
if (!RunAllTestsDoesNotRunTests())
{
DoExpensiveInitialization();
}
return RUN_ALL_TESTS();
}
Is this even possible? I've tried to identify members of ::testing::UnitTest, that would allow me to retrieve this kind of information, without any success so far though.
Am I going at this the wrong way? Preferrably I'd like to avoid lazily doing the initialization via fixture or similar logic requiring me to adjust every test case.
There are various flags that might prevent google test from running any tests. I can think of at least the following:
--gtest_list_tests is passed.
--gtest_repeat=0 is passed.
--help or other forms of it like -h or any unrecognized flag with gtest prefix is passed See here.
You can test these cases by:
The first can be checked using ::testing::GTEST_FLAG(list_tests). See here.
The second can be tested using ::testing::GTEST_FLAG(repeat).
The third can be checked by reading the global variable g_help_flag. While practical, this is not ideal because it's in the internal namespace and might change in future releases.
Another alternative is to parse the command line arguments yourself.
So assuming you want to be practical, one way to get to what you want is this:
// Defining g_help_flag as an extern variable
namespace testing {
namespace internal {
extern bool g_help_flag;
}
}
bool RunAllTestsDoesNotRunTests()
{
// should return false, if and only if RUN_ALL_TESTS() runs any test cases
// implementation?
return ( ::testing::GTEST_FLAG(list_tests) ||
::testing::GTEST_FLAG(repeat) == 0 ||
::testing::internal::g_help_flag);
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
if (!RunAllTestsDoesNotRunTests())
{
// DoExpensiveInitialization();
std::cout << "Expensive initialization!" << std::endl;
} else {
std::cout << "No Expensive initialization!" << std::endl;
}
return RUN_ALL_TESTS();
}
Live example: https://godbolt.org/z/P36fde11T
command line arguments are detected using the GTEST_FLAG macro. An example of what you're trying to do might look like:
#include <gtest/gtest.h>
TEST(equality, always_passes) {
EXPECT_TRUE(true);
}
bool RunAllTestsDoesNotRunTests()
{
return ::testing::GTEST_FLAG(list_tests);
}
void DoExpensiveInitialization() {
std::cout << "Boop Boop, Beep Beep" << std::endl;
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
if (!RunAllTestsDoesNotRunTests())
{
DoExpensiveInitialization();
}
return RUN_ALL_TESTS();
}
When compiled and linked appropriately, it can be run as:
$ ./a.out
Boop Boop, Beep Beep
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from equality
[ RUN ] equality.always_passes
[ OK ] equality.always_passes (0 ms)
[----------] 1 test from equality (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 1 test.
$ ./a.out --gtest_list_tests
equality.
always_passes
i.e. you don't see the Boop Boop, Beep Beep as the function was not called.
Now if you have something that you want to have run once, if you're running tests, then adding it to the testing::Environment would also do the trick:
#include <gtest/gtest.h>
class MyEnvironment: public ::testing::Environment
{
public:
virtual ~MyEnvironment() = default;
// Override this to define how to set up the environment.
virtual void SetUp() { std::cout << "Env Beep Beep" << std::endl; }
// Override this to define how to tear down the environment.
virtual void TearDown() {}
};
TEST(equality, always_passes) {
EXPECT_TRUE(true);
}
int main(int argc, char** argv) {
testing::InitGoogleTest(&argc, argv);
auto* env = new MyEnvironment();
::testing::AddGlobalTestEnvironment(env);
return RUN_ALL_TESTS();
}
When executed, it will also cope with filters (the previous example would not be able to cope in that case):
$ ./a.out --gtest_filter=equality\*
Note: Google Test filter = equality*
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
Env Beep Beep
[----------] 1 test from equality
[ RUN ] equality.always_passes
[ OK ] equality.always_passes (0 ms)
[----------] 1 test from equality (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 1 test.
$ ./a.out --gtest_filter=equality\* --gtest_list_tests
equality.
always_passes
Again, not that the Env Beep Beep does not appear if you don't run any tests (you can check with a filter like --gtest_filter=equality, and you won't see the Env output in that case.
I have a unit test file using Boost Test, like so:
#include <boost/test/unit_test.hpp>
#include <cppx/auto/Cloner_.hpp>
#include <utility>
namespace {
struct S
{
static auto count() -> int& { static int c; return c; }
S(){ ++count(); }
S( const S& ) { ++count(); }
S( S&& ) { ++count(); }
~S() { --count(); }
};
}
BOOST_AUTO_TEST_SUITE( Cloner_ )
BOOST_AUTO_TEST_CASE( default_construction )
{
cppx::Cloner_<int> cloner1;
BOOST_TEST( cloner1.p() == nullptr );
}
BOOST_AUTO_TEST_CASE( auto_cleanup )
{
BOOST_REQUIRE( S::count() == 0 );
{
cppx::Cloner_<S> cloner1( new S );
BOOST_TEST( S::count() == 1 );
cppx::Cloner_<S> cloner2 = cloner1.clone();
BOOST_TEST( S::count() == 2 );
}
BOOST_TEST( S::count() == 0 );
}
BOOST_AUTO_TEST_SUITE_END()
My purpose with the test suite name Cloner_ is to collect all the test cases for the Cloner_ class, under a node in the Boost test cases hierarchy.
When I choose to run all tests in Visual Studio it detects this hierarchy:
[22.09.2018 06:18:37 Informational] ------ Run test started ------
[22.09.2018 06:18:39 Informational] Found test: Cloner_/auto_cleanup
[22.09.2018 06:18:39 Informational] Found test: Cloner_/default_construction
[22.09.2018 06:18:39 Informational] Executing: -> [Cloner_/auto_cleanup]
[22.09.2018 06:18:40 Informational] Executing: -> [Cloner_/default_construction]
[22.09.2018 06:18:40 Informational] ========== Run test finished: 2 run (0:00:03,1048741) ==========
However, it doesn't display the hierarchy, except that it uses the main Boost test module name cppx_tests (defined in a separate file) as root:
(Visual Studio's Test Explorer's presentation of the test cases is up to the right in the above screenshot.)
I'd like to avoid old-fashioned C style name prefixes since Boost Test does provide a means of defining a hierarchy, and since Test Explorer reports paths in that hierarchy when it searches for tests to execute, so that it apparently knows about it.
So, how can I make VS Test Explorer display the test case hierarchy for tests using Boost Test, so that I can readily identify e.g. default_construction testing of class X versus class Y or class Z, or is that not possible?
Good evening everyone, please I'm writing a Library management application in c++ using virtual studio 2012. I had already writing some codes using Dev c++ it worked but when I switched to visual studio it gives error. It involves creating folders and checking if the folders were actually created. That is using dir and mkdir.
Windows and Linux (POSIX) don't support the same API for most file system functions. You can use Microsoft's platform-specific APIs like CreateDirectory() or use the POSIX-like versions like _mkdir().
If you have a more recent C++ compiler / standard library, you can use the experimental filesystem library that is slated to become part of standard C++, perhaps as early as C++17. If not, you can use Boost.Filesystem from which the pre-standard experimental library was drawn.
Here's a complete, minimal example using Boost.Filesystem, which will work on both Windows and Linux without modification:
#include <iostream>
#include <boost/filesystem.hpp>
namespace fs = boost::filesystem;
int main()
{
if( !fs::exists( "my_dir" ) )
{
if( fs::create_directory( "my_dir" ) )
{
std::cout << "Directory created!\n";
}
}
}
See it run: Coliru.
Here's the same code but with std::experimental::filesystem: Coliru.
You would need the appropriate include and linker paths setup in your build system for either of these to work locally. The biggest "gotcha" using the filesystem is that it throws exceptions for a lot of errors by default. You can either setup try/catch blocks at the appropriate places or pass in an error code param to make it return the status there instead.
#include <stdio.h>
#include <windows.h>
int main() {
if (!CreateDirectoryA("C:\\ABC123", NULL))
{
if (GetLastError() == ERROR_ALREADY_EXISTS)
{
printf("Already Exists");
}else if (GetLastError()== ERROR_PATH_NOT_FOUND)
{
printf("Path not found");
}
}else{
printf("Created..");
}
}
simple function will do.
Thanks alot guys but I found this that solved my problem
#include <iostream>
#include <direct.h>
#include <stdlib.h>
#include <stdio.h>
using namespace std;
void main( void )
{
if( _mkdir( "\\testtmp" ) == 0 )
{
printf( "Directory '\\testtmp' was successfully created\n" );
system( "dir \\testtmp" );
if( _rmdir( "\\testtmp" ) == 0 )
printf( "Directory '\\testtmp' was successfully removed\n" );
else
printf( "Problem removing directory '\\testtmp'\n" );
}
else
printf( "Problem creating directory '\\testtmp'\n" );
int a;
cin >> a;
}
the cin >> a; is just to keep the output screen so I can see the result
I am writing a custom CMake find module to use a 3rd party library and would like to extract its version string to use with:
find_package_handle_standard_args(MySDK
REQUIRED_VARS LIBRARY INCLUDE
VERSION_VAR VERSION
)
However its version number is not available as text in the header so I have to build a small c++ program to print it to stdout:
#include <MySDK.h>
int main(int argc, const char * argv[]) {
MySDK::Version version = MySDK::getVersion();
std::cout << version.text << "\n";
return 0;
}
I have tried adding to my custom FindMySDK.cmake something like this:
file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/version.cpp"
"
#include <MySDK.h>
int main(int argc, const char * argv[]) {
MySDK::Version version = MySDK::getVersion();
std::cout << version.text << \"\\n\";
return 0;
}
"
)
add_executable(MySDKVersion
EXCLUDE_FROM_ALL
"${CMAKE_CURRENT_BINARY_DIR}/version.cpp"
)
target_include_directories(MySDKVersion SYSTEM PRIVATE ${MYSDK_INCLUDE_DIRS})
target_link_libraries(MySDKVersion ${MYSDK_LIBRARIES})
execute_process(COMMAND "$<TARGET_FILE:MySDKVersion>")
But it outputs an empty string. Is there a better way to do this kind of thing?
You need to use the try_run command.
Note, that this won't work when cross-compiling, because when cross-compiling any built binaries are not runnable on the host. However when it's your last option, that's what you use.
As the title says, I want to get all test suites or test cases (name) from a test application, ether in the console or as xml output.
Test framework is the boost test library.
Is there an option to achieve this? I did not found anything useful in the documentation.
There is simply the --list_content command line option, exactly for that purpose.
This can be done without much intrusion using a global fixture. Assuming you have a translation unit (cpp file) that contains main explicitly or auto generated, you can intercept test execution when a certain command line argument is provided. Then you can traverse the test tree using a customized visitor, which lists all available tests. Here is a small working example, which creates a test runner by compiling and linking files main_test.cpp, a.cpp and b.cpp:
main_test.cpp
#include <string>
#include <iostream>
// --- Boost Includes ---
#define BOOST_TEST_MODULE MyTestSuite
#define BOOST_TEST_DYN_LINK
#include <boost/test/unit_test.hpp>
using namespace boost::unit_test;
struct Visitor : test_tree_visitor
{
size_t level = 0;
void visit( test_case const& test )
{
std::string indentation( level, '.' );
std::cout << indentation << test.p_name << std::endl;
}
bool test_suite_start( test_suite const& suite )
{
std::string indentation( level, '.' );
level++;
std::cout << indentation << "Suite: " << suite.p_name << std::endl;
return true;
}
void test_suite_finish( test_suite const& suite )
{
level--;
}
};
struct GlobalFixture
{
GlobalFixture( )
{
int argc = framework::master_test_suite( ).argc;
for ( int i = 0; i < argc; i++ )
{
std::string argument( framework::master_test_suite( ).argv[i] );
if ( argument == "list" )
{
Visitor visitor;
traverse_test_tree( framework::master_test_suite( ), visitor );
exit( EXIT_SUCCESS );
}
}
}
};
BOOST_GLOBAL_FIXTURE( GlobalFixture )
a.cpp
#include <boost/test/unit_test.hpp>
BOOST_AUTO_TEST_SUITE ( TestA )
BOOST_AUTO_TEST_CASE ( TestFoo )
{
BOOST_CHECK(true);
}
BOOST_AUTO_TEST_CASE ( TestBar )
{
BOOST_CHECK(true);
}
BOOST_AUTO_TEST_SUITE_END() // TestA
b.cpp
#include <boost/test/unit_test.hpp>
BOOST_AUTO_TEST_SUITE ( TestB )
BOOST_AUTO_TEST_CASE ( TestFoo )
{
BOOST_CHECK(true);
}
BOOST_AUTO_TEST_CASE ( TestBar )
{
BOOST_CHECK(true);
}
BOOST_AUTO_TEST_SUITE_END() // TestA
Invoking the runner without any arguments results in
./somelib_testrunner1
Running 4 test cases...
*** No errors detected
Passing the argument list used in the fixture defined above results in
Suite: MyTestSuite
.Suite: TestA
..TestFoo
..TestBar
.Suite: TestB
..TestFoo
..TestBar
I am afraid there is no easy way to do this without actually running the tests.
And even then the only thing coming to my mind without touching the code would be --log_level=test_suite and some sort of parsing script that finds the "Entering test suite", "Leaving test suite" and "Entering test case" messages (or parsing the xml given when you also put --log_format=XML, if you have a xml parser at hand).
But even that spoils the log with your added BOOST_TEST_MESSAGEs and of course encountered errors and warnings.
Seeing how the boost test framework is not much more than a bunch of macros, defined in the header, you could of course add the functionality yourself, probably easiest by adding another log format just listing the suites and tests, assuming you are fine with running the tests while finding the list.