I am developing a command line options processor for my app. I have decided to use GTest to test it. It's implementation has been shown below in brief:
int main(int argv, char **argv)
{
if (!ProcessOptions(argc, argv)
{
return 1;
}
// Some more code here
return 0;
}
int ProcessOptions(int argc, char **argv)
{
for (int i = 1; i < argc; ++i)
{
CheckOption(argv[i]);
CheckArgument();
if (Success)
{
EnableOption();
}
}
}
The code runs as expected, but the problem is: I want to test this using GTest by supplying different options (valid and invalid) to it. The GTest manual reads:
The ::testing::InitGoogleTest() function parses the command line for
googletest flags, and removes all recognized flags. This allows the
user to control a test program's behavior via various flags, which
we'll cover in the AdvancedGuide. You must call this function before
calling RUN_ALL_TESTS(), or the flags won't be properly initialized.
But this way, I will be able to test just one sequence. I want to do this multiple times for different options. How do I do that?
Is there any better strategy to achieve this? Can I do this using test fixtures?
Have you considered a value-parameterized test? They sound perfect for your situation:
Value-parameterized tests allow you to test your code with different parameters without writing multiple copies of the same test. This is useful in a number of situations, for example:
You have a piece of code whose behavior is affected by one or more command-line flags.
You want to test different implementations of an OO interface.
You want to make sure your code performs correctly for various values of those flags.
You could write one or more test(s) which define the expected behaviour of the command line argument parser and then pass the command line flags down to it this way.
A full code example is shown in the link to the Google Test GitHub docs, but here's a quick outline:
Create a test class inheriting testing::TestWithParam<T>.
Use TEST_P and within it, the GetParam() method to access the parameter value.
You can instantiate your tests with INSTANTIATE_TEST_SUITE_P. Use the testing::Values method to supply values.
Related
Let me elaborate on the title:
I want to implement a system that would allow me to enable/disable/modify the general behavior of my program. Here are some examples:
I could switch off and on logging
I could change if my graphing program should use floating or pixel coordinates
I could change if my calculations should be based upon some method or some other method
I could enable/disable certain aspects like maybe a extension api
I could enable/disable some basic integrated profiler (if I had one)
These are some made-up examples.
Now I want to know what the most common solution for this sort of thing is.
I could imagine this working with some sort of singelton class that gets instanced globally or in some other globally available object. Another thing that would be possible would be just constexpr or other variables floating around in a namespace, again globally.
However doing something like that, globally, feels like bad practise.
second part of the question
This might sound like I cant decide what I want, but I want a way to modify all these switches/flags or whatever they are actually called in a single location, without tying any of my classes to it. I don't know if this is possible however.
Why don't I want to do that? Well I like to make my classes somewhat reusable and I don't like tying classes together, unless its required by the DRY principle and or inheritance. I basically couldn't get rid of the flags without modifying the possible hundreds of classes that used them.
What I have tried in the past
Having it all as compiler defines. This worked reasonably well, however I didnt like that I couldnt make it so if the flag file was gone there were some sort of default settings that would make the classes themselves still operational and changeable (through these default values)
Having it as a class and instancing it globally (system class). Worked ok, however I didnt like instancing anything globally. Also same problem as above
Instancing the system class locally and passing it to the classes on construction. This was kinda cool, since I could make multiple instruction sets. However at the same time that kinda ruined the point since it would lead to things that needed to have one flag set the same to have them set differently and therefore failing to properly work together. Also passing it on every construction was a pain.
A static class. This one worked ok for the longest time, however there is still the problem when there are missing dependencies.
Summary
Basically I am looking for a way to have a single "place" where I can mess with some values (bools, floats etc.) and that will change the behaviour of all classes using them for whatever, where said values either overwrite default values or get replaced by default values if said "place" isnt defined.
If a Singleton class does not work for you , maybe using a DI container may fit in your third approach? It may help with the construction and make the code more testable.
There are some DI frameworks for c++, like https://github.com/google/fruit/wiki or https://github.com/boost-experimental/di which you can use.
If you decide to use switch/flags, pay attention for "cyclometric complexity".
If you do not change the skeleton of your algorithm but only his behaviour according to the objets in parameter, have a look at "template design pattern". This method allow you to define a generic algorithm and specify particular step for a particular situation.
Here's an approach I found useful; I don't know if it's what you're looking for, but maybe it will give you some ideas.
First, I created a BehaviorFlags.h file that declares the following function:
// Returns true iff the given feature/behavior flag was specified for us to use
bool IsBehaviorFlagEnabled(const char * flagName);
The idea being that any code in any of your classes could call this function to find out if a particular behavior should be enabled or not. For example, you might put this code at the top of your ExtensionsAPI.cpp file:
#include "BehaviorFlags.h"
static const enableExtensionAPI = IsBehaviorFlagEnabled("enable_extensions_api");
[...]
void DoTheExtensionsAPIStuff()
{
if (enableExtensionsAPI == false) return;
[... otherwise do the extensions API stuff ...]
}
Note that the IsBehaviorFlagEnabled() call is only executed once at program startup, for best run-time efficiency; but you also have the option of calling IsBehaviorFlagEnabled() on every call to DoTheExtensionsAPIStuff(), if run-time efficiency is less important that being able to change your program's behavior without having to restart your program.
As far as how the IsBehaviorFlagEnabled() function itself is implemented, it looks something like this (simplified version for demonstration purposes):
bool IsBehaviorFlagEnabled(const char * fileName)
{
// Note: a real implementation would find the user's home directory
// using the proper API and not just rely on ~ to expand to the home-dir path
std::string filePath = "~/MyProgram_Settings/";
filePath += fileName;
FILE * fpIn = fopen(filePath.c_str(), "r"); // i.e. does the file exist?
bool ret = (fpIn != NULL);
fclose(fpIn);
return ret;
}
The idea being that if you want to change your program's behavior, you can do so by creating a file (or folder) in the ~/MyProgram_Settings directory with the appropriate name. E.g. if you want to enable your Extensions API, you could just do a
touch ~/MyProgram_Settings/enable_extensions_api
... and then re-start your program, and now IsBehaviorFlagEnabled("enable_extensions_api") returns true and so your Extensions API is enabled.
The benefits I see of doing it this way (as opposed to parsing a .ini file at startup or something like that) are:
There's no need to modify any "central header file" or "registry file" every time you add a new behavior-flag.
You don't have to put a ParseINIFile() function at the top of main() in order for your flags-functionality to work correctly.
You don't have to use a text editor or memorize a .ini syntax to change the program's behavior
In a pinch (e.g. no shell access) you can create/remove settings simply using the "New Folder" and "Delete" functionality of the desktop's window manager.
The settings are persistent across runs of the program (i.e. no need to specify the same command line arguments every time)
The settings are persistent across reboots of the computer
The flags can be easily modified by a script (via e.g. touch ~/MyProgram_Settings/blah or rm -f ~/MyProgram_Settings/blah) -- much easier than getting a shell script to correctly modify a .ini file
If you have code in multiple different .cpp files that needs to be controlled by the same flag-file, you can just call IsBehaviorFlagEnabled("that_file") from each of them; no need to have every call site refer to the same global boolean variable if you don't want them to.
Extra credit: If you're using a bug-tracker and therefore have bug/feature ticket numbers assigned to various issues, you can creep the elegance a little bit further by also adding a class like this one:
/** This class encapsulates a feature that can be selectively disabled/enabled by putting an
* "enable_behavior_xxxx" or "disable_behavior_xxxx" file into the ~/MyProgram_Settings folder.
*/
class ConditionalBehavior
{
public:
/** Constructor.
* #param bugNumber Bug-Tracker ID number associated with this bug/feature.
* #param defaultState If true, this beheavior will be enabled by default (i.e. if no corresponding
* file exists in ~/MyProgram_Settings). If false, it will be disabled by default.
* #param switchAtVersion If specified, this feature's default-enabled state will be inverted if
* GetMyProgramVersion() returns any version number greater than this.
*/
ConditionalBehavior(int bugNumber, bool defaultState, int switchAtVersion = -1)
{
if ((switchAtVersion >= 0)&&(GetMyProgramVersion() >= switchAtVersion)) _enabled = !_enabled;
std::string fn = defaultState ? "disable" : "enable";
fn += "_behavior_";
fn += to_string(bugNumber);
if ((IsBehaviorFlagEnabled(fn))
||(IsBehaviorFlagEnabled("enable_everything")))
{
_enabled = !_enabled;
printf("Note: %s Behavior #%i\n", _enabled?"Enabling":"Disabling", bugNumber);
}
}
/** Returns true iff this feature should be enabled. */
bool IsEnabled() const {return _enabled;}
private:
bool _enabled;
};
Then, in your ExtensionsAPI.cpp file, you might have something like this:
// Extensions API feature is tracker #4321; disabled by default for now
// but you can try it out via "touch ~/MyProgram_Settings/enable_feature_4321"
static const ConditionalBehavior _feature4321(4321, false);
// Also tracker #4222 is now enabled-by-default, but you can disable
// it manually via "touch ~/MyProgram_Settings/disable_feature_4222"
static const ConditionalBehavior _feature4222(4222, true);
[...]
void DoTheExtensionsAPIStuff()
{
if (_feature4321.IsEnabled() == false) return;
[... otherwise do the extensions API stuff ...]
}
... or if you know that you are planning to make your Extensions API enabled-by-default starting with version 4500 of your program, you can set it so that Extensions API will be enabled-by-default only if GetMyProgramVersion() returns 4500 or greater:
static ConditionalBehavior _feature4321(4321, false, 4500);
[...]
... also, if you wanted to get more elaborate, the API could be extended so that IsBehaviorFlagEnabled() can optionally return a string to the caller containing the contents of the file it found (if any), so that you could do shell commands like:
echo "opengl" > ~/MyProgram_Settings/graphics_renderer
... to tell your program to use OpenGL for its 3D graphics, or etc:
// In Renderer.cpp
std::string rendererType;
if (IsDebugFlagEnabled("graphics_renderer", &rendererType))
{
printf("The user wants me to use [%s] for rendering 3D graphics!\n", rendererType.c_str());
}
else printf("The user didn't specify what renderer to use.\n");
I have a CPP unit test which tests a class which is designed to read configuration: we can call this class
Config
The config class has the capacity of doing
Config c;
c.read("/tmp/random-tmp-directory/test.conf");
The random-temp-directory is created by a bash script and should be passed into the test binary.
#!/bin/bash
TEMPDIR=$(mktemp)
cp files/config/test.conf $TEMPDIR/.
./testConfig $(mktemp)/test.conf
The above creates a temp directory, copies our temporary file and passes the path to the test, so it can load the correct file.
Is there a way to tell CPPUNIT to send the commandline arguments, or any arguments to the test registry?
Here is my testConfig.cpp
#include <all the required.h>
CPPUNIT_TEST_SUITE_REGISTRATION(testConfig);
int main(int argc, char ** argv)
{
CPPUNIT_NS::TestResult testresult;
CPPUNIT_NS::TestRunner runner;
CPPUNIT_NS::TestFactoryRegistry ®istry = CPPUNIT_NS::TestFactoryRegistry::getRegistry();
// register listener for collecting the test-results
CPPUNIT_NS::TestResultCollector collectedresults;
testresult.addListener(&collectedresults);
runner.addTest(registry.makeTest());
runner.run(testresult);
// Print test in a compiler compatible format.
CppUnit::CompilerOutputter outputter( &collectedresults, std::cerr );
outputter.write();
return collectedresults.wasSuccessful() ? 0 : 1;
}
Consider dividing your code into at least three distinct methods: the part that constructs the config file name, the part that reads the config file, and the part that parses what was read from the config file. You can easily and thoroughly unit test both the file name builder and the parser methods. And as long as you can test simply reading data from the file even one time, you should be golden.
[edit]
For example, you might have a method like string & assembleConfigFileName(string basepath, string randompath, string filename) that takes in the different components of your path and filename, and puts them together. One unit test should look like this:
void TestConfig::assembleConfigFileName_good()
{
string goodBase("/tmp");
string goodPath("1234");
string goodName("test.conf");
string actual(assembleConfigFileName(goodBase, goodPath, goodName));
string expected("/tmp/1234/test.conf");
CPPUNIT_ASSERT_EQUAL(expected, actual);
}
Now you can test that you're building the fully qualified config file name exactly correctly. The test is not trying to read a file. The test is not trying to generate a random number. The test is providing an example of exactly what kinds of input the routine needs to take, and stating exactly what the output should look like given that exact input. And it's proving the code does exactly that.
It's not important for this routine to actually read a config file out of a temp directory. It's only important that it generate the right file name.
Similarly, you build a unit test to test for each possible flow through your code, including error scenarios. Let's say you wrote an exception handler that throws if the random path is wrong. Your unit test will test the exception mechanism:
void TestConfig::assembleConfigFileName_null_path()
{
string goodBase("/tmp");
string nullPath;
string goodName("temp.config");
CPPUNIT_ASSERT_THROW(assembleConfigFileName(goodBase, nullPath, goodName), MissingPathException);
}
The tests are now a document that says exactly how it works, and exactly how it fails. And they prove it every single time you run the tests.
Something you appear to be trying to do is to create a system test, not a unit test. In a unit test, you do NOT want to be passing in randomly pathed config files. You aren't trying to test the external dependencies, that the file system works, that a shell script works, that $TMPDIR works, none of that. You're only trying to test that the logic you've written works.
Testing random files in the operating system is very appropriate for automated system tests, but not for automated unit tests.
I've written my own access layer to a game engine. There is a GameLoop which gets called every frame which lets me process my own code. I'm able to do specific things and to check if these things happened. In a very basic way it could look like this:
void cycle()
{
//set a specific value
Engine::setText("Hello World");
//read the value
std::string text = Engine::getText();
}
I want to test if my Engine-layer is working by writing automated tests. I have some experience in using the Boost Unittest Framework for simple comparison tests like this.
The problem is, that some things I want the engine to do are just processed after the call to cycle(). So calling Engine::getText() directly after Engine::setText(...) would return an empty string. If I would wait until the next call of cycle() the right value would be returned.
I now am wondering how I should write my tests if it is not possible to process them in the same cycle. Are there any best practices? Is it possible to use the "traditional testing" approach given by Boost Unittest Framework in such an environment? Are there perhaps other frameworks aimed at such a specialised case?
I'm using C++ for everything here, but I could imagine that there are answers unrelated to the programming language.
UPDATE:
It is not possible to access the Engine outside of cycle()
In your example above, std::string text = Engine::getText(); is the code you want to remember from one cycle but execute in the next. You can save it for later execution. For example - using C++11 you could use a lambda to wrap the test into a simple function specified inline.
There are two options with you:
If the library that you have can be used synchronously or using c++11 futures like facility (which can indicate the readyness of the result) then in your test case you can do something as below
void testcycle()
{
//set a specific value
Engine::setText("Hello World");
while (!Engine::isResultReady());
//read the value
assert(Engine::getText() == "WHATEVERVALUEYOUEXPECT");
}
If you dont have the above the best you can do have a timeout (this is not a good option though because you may have spurious failures):
void testcycle()
{
//set a specific value
Engine::setText("Hello World");
while (Engine::getText() != "WHATEVERVALUEYOUEXPECT") {
wait(1 millisec);
if (total_wait_time > 1 sec) // you can put whatever max time
assert(0);
}
}
I am trying to write a function/method for my project, which will ask to user which all test cases are you going to run?
It looks like below...,
Test_Cases_1
|_TestNo1
|_TestNo2....so on
Test_Cases_2
|_TestNo1
|_TestNo2....so on
....
....so on
Test_Cases_N
|_TestNo1
|_TestNo2....so on
So, now the challenge is while running the project it should prompt me what all test cases you would like to execute?
If I select Test_Cases_1 and Test_Cases_N. Then it should execute these two test cases and should exclude all other from Test_Cases_2 to ..... In result window also I would like to see the results of Test_Cases_1 and Test_Cases_N.
So, if I will see the GoogleTest, there is a method called test_case_to_run_count();
But all the test cases are getting registered with Test_F() method.
So, I did lots of analysis, but still did not find any solution.
Please help me.
You could use advanced options to run Google tests.
To run only some unit tests you could use --gtest_filter=Test_Cases1* command line option with value that accepts the * and ? wildcards for matching with multiple tests. I think it will solve your problem.
UPD:
Well, the question was how to run specific test cases. Integration of gtest with your GUI is another thing, which I can't really comment, because you didn't provide details of your approach. However I believe the following approach might be a good start:
Get all testcases by running tests with --gtest_list_tests
Parse this data into your GUI
Select test cases you want ro run
Run test executable with option --gtest_filter
Summarising Rasmi Ranjan Nayak's and nogard's answers and adding another option:
On the console
You should use the flag --gtest_filter, like
--gtest_filter=Test_Cases1*
(You can also do this in Properties|Configuration Properties|Debugging|Command Arguments)
On the environment
You should set the variable GTEST_FILTER like
export GTEST_FILTER = "Test_Cases1*"
On the code
You should set a flag filter, like
::testing::GTEST_FLAG(filter) = "Test_Cases1*";
such that your main function becomes something like
int main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
::testing::GTEST_FLAG(filter) = "Test_Cases1*";
return RUN_ALL_TESTS();
}
See section Running a Subset of the Tests for more info on the syntax of the string you can use.
Finally I got some answer,
::test::GTEST_FLAG(list_tests) = true; //From your program, not w.r.t console.
If you would like to use --gtest_filter =*; /* =*, =xyz*... etc*/ // You need to use them in Console.
So, my requirement is to use them from the program not from the console.
Updated:-
Finally I got the answer for updating the same in from the program.
::testing::GTEST_FLAG(filter) = "*Counter*:*IsPrime*:*ListenersTest.DoesNotLeak*";//":-:*Counter*";
InitGoogleTest(&argc, argv);
RUN_ALL_TEST();
So, Thanks for all the answers.
You people are great.
I have a section of code that I would not like to run if it is being unit tested. I was hoping to find some #defined flag that is set by the gtest library that I can check. I couldn't find one that is used for that purpose, but after looking through the gtest header, I found one I thought I could use like this:
SomeClass::SomeFunctionImUnitTesting() {
// some code here
#ifndef GTEST_NAME
// some code I don't want to be tested here
#endif
// more code here
}
This doesn't seem to work as all the code runs regardless. Is there another flag I can check that might work?
Google Test doesn't need or provide its own build wrapper. You don't even have to recompile your source files sometimes. You can just link them along with your test code. Your test code calls your already-compiled library code. Your library code probably doesn't even include and Gtest headers.
If you want your library code to run differently under test, then you first need to make sure that your library code is compiled differently under test. You'll need another build target. When compiling for that build target, you can define a symbol that indicates to your code that it's in test mode. I'd avoid the GTEST prefix for that symbol; leave for use by Google's own code.
Another way to achieve what you're looking for is to use dependency injection. Move your special code into another routine, possibly in its own class. Pass a pointer to that function or class into your SomeFunctionImUnitTesting function and call it. When you're testing that code, you can have your test harness pass a different function or class to that code, therefore avoiding the problematic code without having to compile your code multiple times.
In main():
int main(int argc, char** argv)
{
testing::InitGoogleTest(&argc, argv);
setenv("GTEST_RUNNING", "1", true);
ros::init(argc, argv, "tester");
return RUN_ALL_TESTS();
}
Somewhere else:
bool gtestRunning = strcmp(getenv("GTEST_RUNNING"), "1") == 0;
if (gtestRunning)
{
}
else
{
}