I'm setting up a bunch of unit tests using CppUnit but am having the problem that none of the tests are being run. The project is divided up into several small libraries and I planned on dividing the unit test classes up the same way and then linking them all into a single test program. The problem is, then the test classes are in their own libraries, they don't get linked into the main test program unless I explicitly call them, i.e. I have to put in
runner.addTest( TestClass::suite() );
individually for each test class and can't use the TestFactoryRegistry's makeTests() method to get the list of tests. If I just compile them all together in the top directory the makeTests() method works fine but I don't want to have all the test classes in one location if I can help it.
The CppUnit documentation gives the following little hint
Linking problem when using Helper
macros ?
When you create a project and write
its unit test suites, the work is made
easier through the use of the
so-called helper macros :
CPPUNIT_TEST_SUITE_NAMED_REGISTRATION,
CPPUNIT_REGISTRY_ADD and
CPPUNIT_REGISTRY_ADD_TO_DEFAULT. The
problem is that if you use those
macros in the source code file of a
TestFixture class (say MyTest as an
example), and if you use a line like
this one
runner.addTest( CppUnit::TestFactoryRegistry::getRegistry().makeTest()
);
in your main() function in file
main.cpp, there will have no test run
at all !
The reason is simply that the link
stage, one of the step of the build
process, do not insert the object
files (.obj or .o files) in the final
executable if there is no undefined
symbol in your main.cpp.
That way, the object code which
contains the AutoRegister static
variables instantiation is not part of
the final executable and is not able
to insert oneself into the runner in
the main() function.
You have to create an undefined symbol
in main.cpp so that the mytest.o file
is integrated with main.o into the
final executable.
Trick committed by Michel Nolard
but doesn't say how to make this work and I'm just dense enough not to be able to figure it out myself or find an example on-line.
Now I could just make a separate executable test for each library, and in the end I may go that way, but I wanted to try to get this working first so I just had one single test program to run to test the whole thing. Any ideas/examples of how to get this to work?
By adding an undefined symbol to main, he just means create any random external symbol to force the linker into searching your external libraries that contain the test code.
For example, assuming two test libraries fred and barney, in fredTestLib.cpp you'd just add this line:
int fredDummyInt = 0; // declare a unique symbol for the linker to resolve
and in barneyTestLib.cpp, you'd add a similar line:
int barneyDummyInt = 0; // a different unique symbol for the linker to resolve
You could compile each library separately in different steps. In the main test program, you then force the linker to resolve them. So add these lines to main.cpp:
extern int fredDummyInt;
extern int barneyDummyInt;
...
main () {
...
fredDummyInt++; // give the linker some symbols to resolve
barneyDummyInt++;
...
The idea (according to what the author of the trick above is saying) is that because the linker is already searching fredTest.lib for fredDummyInt, it will also find and resolve your automatically registered tests.
Note: I have not tried this to see if it works! I'm just answering your question about externals.
Another approach to consider would be to create your tests in DLLs, and use LoadLibrary() to explicitly bring them in to run. For overkill, if you use the MfcUi::TestRunner, you could probably build a little drop-down GUI thing that lets you pick the library to load, loads it, then displays the tests to run in that library, then runs them.
The solution for this problem is rather simple as statet before (but may not be very elegant).
For each TestFixture that is located in an external library you have to add the following two lines of code in the main module
#include <CppUnitTestFixtureExample.h>
CppUnitTestFixtureExample Test1;
It creates an unused dummy variable that is not used, it just forces the linker to link the test fixture. Now the test runner that is located in the main module is able to run the test.
I realize this post is now quite old, but for anyone else who comes across it:
One way to address this w/out having references in your code is to instruct (force) the linker to include the entire static library in the binary. Details area available from the gcc & ld man pages, and this post covers it too:
How to force gcc to link unreferenced, static C++ objects from a library
Per ld's man page, it's important to consider explicitly turning the option off (also shown in one of the examples above).
Related
I have a C++ project in Visual Studio, and have added another project exclusively for testing. Both of these projects are EXEs (console apps). So how do I use the first project inside the second?
Just to clarify, the question here would be somewhat self-evident if the first project was a library that one could simply include in the second project but, being an EXE, this is where the problem lies.
Per your comments, you have a C++ console application (MyApp) for which you have developed some application-specific classes that you want to unit-test with googletest in
Visual Studio. How?
As you say, if you wanted to unit-test a library the way to do it would be
obvious. You would:
1) Create a project to create a unit-testing application (UnitTest).
2) Configure the include-search directories so that the compiler can find the library's headers.
3) Configure the library-search directories so that the linker can find the library itself.
4) Add the library itself to the linker inputs.
5) Make the UnitTest project dependent on the library project, so that building UnitTest ensures MyApp is up-to-date.
6) Code the UnitTest app per googletest docs.
But since the classes you want to unit-test are specific to MyApp, you don't have any
library.
A drill-sergeant answer to that is: You don't have a library containing the classes you want to unit-test? So make one!
That way you use 3 projects:-
MyAppLib, generating library that contains all the functionality you want to unit-test.
MyApp, generating the same executable as at present, but linking MyAppLib
UnitTest, generating an executable that unit-tests MyAppLib, also linking MyAppLib
However if you don't like the drill-sergeant answer you can work around it.
From the usual build-system point of view (the one designed into Visual Studio),
the important output of the MyApp project is the build-target - the .exe.
The .obj files generated are just intermediate by-products. VS offers you no support
for treating these by-products as automatic linker inputs of a dependent project, and if a dependent project was also an .exe of the same sort - as it is your case - then such automatic linkage would be impossible anyhow because the main entry point would be multiply defined.
But from the unit-testing point of view it's the other way round. The .exe is of no interest, whereas (some of) the .obj files wholly or partly contain the implementations of the classes you want to unit test. In the text-book case where class foo is defined in foo.h and implemented in foo.cpp, the object file foo.obj is needed in the linkage of UnitTest.
For simplicity, assume that MyApp employs just one application-specific class foo,
defined in foo.h and implemented in foo.cpp. Then you have two options for building UnitTest.
a) You can add foo.cpp to the source files of UnitTest. Don't copy it of course. Just Add an existing item from the source folder of MyApp. Then you're done, but this
course has the downside that foo.cpp is exposed to untoward editing within
the UnitTest project.
b) You can treat foo.obj just like a static library required for the linkage of UnitTest and follow steps 1) - 6) above. This means in particular at step 3) that the {Debug|Release} build of UnitTest is configured with library-search directories that include \path\to\MyApp\{Debug|Release} (either in relative or absolute form).
In reality, for option b), there's very likely more than one .obj file from MyApp that you will have to link in UnitTest, and quite likely that their number will grow with time. Maintaining the right linkage of UnitTest could become a chore, and you might come to the conclusion that the drill-sergeant was right after all.
Depends. Google Test is (primarily) a Unit Testing framework (oversimplifying, testing classes). You can absolutely use is for other types of tests, but it doesn't have "built in" functionality for other types of testing, you'll have to write it yourself.
If you are trying to system test your executable, than you can run the process. I suggest using Boost.Process if you are using a multi-platform system or already have a boost dependency. Else, look here: launch an exe/process with stdin stdout and stderr?
The "tests" you write will call the executable, and can input stdin or stdout accordingly.
For example:
std::string path_to_exectuable = "thepath";
TEST(FooTester,CheckHelpScriptReturns0)
{
using bp =::boost::process;
std::vector<std::string> args; args.push_back("--help");
bp::context ctx;
ctx.stdout_behavior = bp::capture_stream();
bp::child c = bp::launch(exec, args, ctx);
bp::status s = c.wait();
ASSERT_TRUE(s.exited())<<"process didn't exit!";
ASSERT_EQ(s.exit_status(),0)<<"Help didn't return 0";
}
I was in a similar situation and I set this up in a way that effectively accomplishes the same goal of Mike Kinghan's answer as far as the compiler is concerned, but goes about it a different way from the user's perspective.
What I did was create a custom Configuration that I called "Testing". You create a new configuration by opening the project settings, choosing "Configuration Manager..." and selecting "New..." in the configuration selection box.
When prompted, I chose to copy the settings from the default "Debug" configuration, so that I can use the debugger with my tests just the same as if I was in the "Debug" configuration.
Under the new Testing configuration, I set the options for the compiler and linker to use google test as you normally would.
The important change in the properties is that I define a preprocessor variable which I have called "TESTING".
I rewrote my "main.cpp" to look something like this:
...
// includes
// functions
// whatever
...
#ifdef TESTING
#include <gtest/gtest.h>
#endif
int main(int argc, char **argv) {
#ifdef TESTING
::testing::InitGoogleTest(&argc, argv);
int val = RUN_ALL_TESTS();
getchar(); // not necessary, but keeps the console open
return val;
#endif
// rest of main() as normal...
}
What I'm trying to indicate is that I only changed a few lines right around where main is defined, I don't have to make gross changes spread throughout the file.
Now that this is all set up I simply made a new source folder for my tests, and create ".cpp" files in there. To avoid bloating the normal executable, I wrap these files with a check for the TESTING variable, so I have something like this:
tests/Test.cpp:
#ifdef TESTING
#include <gtest/gtest.h>
#include "my_class_header.h"
TEST(TestMyClass, test_something) {
// perform some test on class
}
#endif
I think these files still get "hit" by the compiler under Debug and Release configurations, so having a ton of these might slow down the build, but the Debug and Release objects wont get bloated with testing code.
The two takeaways are:
Using this method, the testing code is still organized separately from the application code, but it still resides in the same Visual Studio project, which may or may not be beneficial. Personally I like not having to manage/worry about a second project.
Like Mike Kinghan said, managing and linking .obj files yourself can become a chore, but by using this method, the default Visual Studio settings manage this for you.
One downside is that effectively redundant copies of all object files will get created in the "Testing" output directory. With more configuration, surely there must be a way to "share" the Debug object files, but I didn't have a reason to go that far.
This is a very simple method which may be a lot easier than refactoring your application into separate libraries and a main. I don't love using preprocessor wankery, but in this case it's fairly straightforward, not too much code bloat, and accomplishes exactly what it needs to. You could always trigger the tests another way, without using the preprocessor.
If you are not very rigid about having the tests in a different project, you can write the tests in your application project. Then just make the application execute the tests when receiving certain command line arguments, and execute the normal application logic otherwise, i.e.
int main(int argc, char* argv[])
{
if (argc >= 2 && std::string(argv[1]) == "--tests")
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
else
{
// Application logic goes here
}
}
TEST(ExampleTests, TestSQRTCalculation) // assuming all the right headers are included
{
EXPECT_NEAR(2.0, std::sqrt(4.0), 0.000001);
}
This avoids creating an unnecessary library for the sole purpose of testing, although you can still do it if it is correct structure-wise. The downside is that the testing code goes into the executable you are going to release. If you don't want that I guess you need an additional configuration which specifies a pre-processor directive to disable the tests.
Debugging the tests or running them automatically at post build is easy, simply by specifying "--tests" as debug args or at post build command line respectively.
If you want to test a console app you can run a test that opens a console window and run the exe file of the first app.
Then in your googletest catch the standard output from the exe you just ran.
[For more control over the first app you might need to have the first app parse arguments sent to it, e.g. some flags like -x or what ever you need.]
I have prepared a github repo including Visual Studio 2015 solution in parralel of Mike's "drill-sergeant" suggestion. You can use it directly without any additional requirement or dependency.
https://github.com/fuatcoskun/GoogleTestVS2015
I hope it helps...
I have a basic question regarding Googletest in Eclipse.
I am using the test-runner plug in to run the Googletests.
But I need to specify a binary which runs my unit tests (of course that makes sense.)
The problem is that in my project I now have two main functions, one to run the actual program and one
int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
to run the google tests.
Each time I want to run one I comment the other out, which of course is stupid.
But what practice do you use to handle this situation?
Googletest C++ is a unit-testing framework. That means it is intended for testing
implementations of C++ APIs. It isn't intended for testing programs.
For practical purposes a C++ API is what you get in a C++ header file. The
implementation of such an an API might be:
Just the header file itself. (The implementation is entirely inline)
The header file plus a single C++ source file
The header file plus a bunch of C++ source files
To generalize, the implementation of a C++ API is a header file plus
0 or more source files.
Say your program my_prog invokes an API that you or your team have developed
for managing gizmos. The implementation is something like:
gizmo.h
[gizmo_0.cpp,...gizmo_N.cpp]
where [...] means optionally ...
Maybe my_prog relies on other APIs for which you or your team are responsible,
but we'll stick with just one. my_prog uses the gizmo API by:-
Using #include "gizmo.h" in some source files.
Compiling the [gizmo_0.cpp,...gizmo_N.cpp] source files, if any.
Linking the [gizmo_0.o,...gizmo_N.o] object files, if any.
(gizmo_0.obj, etc. if you're on Windows)
Testing your implementation of the gizmo API with Googletest is supposed
to confirm that this implementation is correct, independently of my_prog
or any other program that relies on it to manage gizmos. So incorporating
the unit-testing of the implementation in the implementation of my_prog is misguided:-
Maybe your colleague writes another program that also needs to manage gizmos
with this implementation. Maybe you write another one. Is whoever writes this
other program supposed to repeat the process of incorporating gizmo unit-tests
into it - The same ones? Different ones? - and making the program conditionally
compile as either a gizmo test-harness or as whatever it's supposed to be in real life?
And how do you know that the gizmo implementation isn't somehow entangled with
functionality that's unique to my_prog, or with the implementation of some
other API that my_prog uses in the same way - so that when you or somebody
else tries to reuse it in another program, it breaks or behaves wrongly?
No program that relies on this gizmo implementation is the place to put
its unit-testing. Making my_prog conditionally compile different main functions so it can
double as a unit-test harness for your gizmo library is similar to cutting a hole in the
crotch of your jeans for your head to fit through.
The way you're supposed to unit-test the gizmo library is to write a program that is the
test-harness for this library, and nothing else. This program, say gizmo_test, will
use the gizmo API in just the same way as any other program would use it, but
for the sole purpose of testing the gizmo library. All that gizmo_test will do is execute tests
of the gizmo library, by invoking its API.
As a first approximation, the GoogleTest recipe for gizmo_test is:
Write a header file, gizmo_test.h
#include "gizmo.h" in it
#include <gtest/gtest.h> in it
Then write your Googletest test cases in it
Write the following source file gizmo_test.cpp
#include "gizmo_test.h"
int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
Create a project gizmo_test - in Eclipse or whatever development environment or build system you use -
that builds the gizmo_test executable by:
Compiling the source files gizmo_test.cpp + [gizmo_0.cpp,...gizmo_N.cpp]
Linking the resulting object files gizmo_test.o + [gizmo_0.o,...gizmo_N.o], plus libgtest and any other libraries on which your gizmo library depends
You have two projects. The one that makes my_prog and the one that makes gizmo_test. In your development environment or
build system, make the build of my_prog depend on the build of gizmo_test, so that when you change anything that affects
the gizmo library and rebuild my_prog, gizmo_test gets rebuilt first.
That's a first approximation. Did you notice a while back that I started talking about your gizmo library? That's what
you've got (or should have). In C++ and programming generally, the implementation of an an API is called a library.
And maybe you also noticed some fragility, inconvenience and wastage in the recipe for gizmo_test. You have the same set of gizmo source files
[gizmo_0.cpp,...gizmo_N.cpp] in both projects. So you might edit, compile and link them differently in two projects. And they'll get compiled in both projects, either differently,
which is wrong, or identically, which is pointless.
Of course if this set of source files is empty - the gizmo library is nothing but gizmo.h - there's no such problem. But if it's not empty,
there is.
As you know, in C++ we don't use a library by building its source files in every program that uses it - not unless it's a header-only library.
A library is built by itself into an object library (either static or dynamic), and to use it a program just includes the library's
header file(s) and links the object library.
That's how a program should use your gizmo library too. So to the final approximation:-
Make a project libgizmo that builds a gizmo object library (static or dynamic, as you see fit).
Make a project gizmo_test as above, except that instead of compiling and linking [gizmo_0.cpp,...gizmo_N.cpp], it just links libgizmo, and make this project
depend on the libgizmo project.
Make a project my_prog as you have it now, but instead of compiling and linking [gizmo_0.cpp,...gizmo_N.cpp], just link libgizmo, and make this project
depend on the gizmo_test project.
So you have three projects by the time you build the first program that uses the gizmo library. Each subsequent program that uses the gizmo library needs one
more project, like the my_prog project.
Googletest is designed for testing C++ libraries, and this is how you're supposed to use it.
Now I know nothing about your program or how you are currently deploying Googletest test cases in your project. Maybe there aren't any well-defined API implementations in it
that those test cases are supposed to exercise, that you can factor out into free-standing libraries. Possibly that could be because your program is so very simple that
unit-testing its "components" is non-applicable, and you'd be wiser just to write blackbox tests of the program. More likely it would be because you've so far failed
to design a program architecture that is capable of being unit-tested. If that's what you find, you need to fix it, and then apply Googletest the right way. It will be worth the
effort.
And in case it needs pointed out, unit-tests are not program tests, so as well as unit-testing any libraries your program relies on, if they are your responsibility, you also need blackbox tests of your program.
Indeed a valid question. I believe that Mike Kinghan's detailed response explains how you should use google-test but nothing is set in stone and in the end, you can use google-test in any way you want.
So, since I actually encountered a similar situation, what I did was rather simple.
I modified the production file source code so that the main function will change from this:
void main(void)
to this:
#ifndef TDD_ENABLED // TDD_ENABLED is defined only in the unit test project
void main(void)
#else
void main_app(void)
#endif
this way when you are running unit tests, the original 'main' function is converted to 'main_app' and you avoid conflicts with the 'main' of google test. When you are running the original production code project, the 'main_app' function is named again 'main' as it should. So, technically, from your production code perspective, nothing has changed.
Lately I have been trying to use gcc/gcov to undertake code-coverage testing for a C++ project. The project consists of its main module and several .so libraries, which should all be taken into measurement.
I have compiled all the modules with --coverage parameter each with gcc and kept them where they were generated, together with corresponding .gcno tag files. After a normal execution and graceful exit the .gcda files could be properly generated.
The problem is, the program is supposed to be a services without interruption or termination, and it's not allowed to insert any customized code (like a signal handler) into the main module. As the solutions from the web suggest, I wrote a signal handler function in a standalone .so library, which invokes __gcov_flush on receiving SIGUSR1 signal to flush the runtime coverage counters to files.
However, it was observed that, while the __gcov_flush function is guaranteed to be properly invoked, only the .gcda file of the .so library is generated during runtime. It seems to me that __gcov_flush is in charge of flushing the data of the wrapper module but not the others. I wonder if this is how it is supposed to work, or is there some tricks I need to take care of to generate complete results?
I see two problems here.
If your executable loads several shared libraries it is quite hard
to get the functionality needed. The linker adds profiling code from
libgcov.a for every shared library and it is hard to call every
library's __gcov_flush() from one central place like signal handler
defined in yet another shared library.
The function __gcov_flush() function from libgcov.a is declared
with __attribute__ ((__visibility__ ("hidden")))
If you extract _gcov.o from the archive and run
objdump -t _gcov.o
you see something like
_gcov.o: file format elf32-littlearm
SYMBOL TABLE:
000014b4 g F .text 00000016 .hidden __gcov_flush
Linker doesn't export hidden symbols even if you ask it to do that
as it mentioned here or here
So I see two solutions for the second problem.
You may try to edit symbol table for _gcov.o in libgcov.a and
set visibility to "default" instead of "hidden". I didn't take
that way because I didn't find any good elf editor.
If you managed to do that make sure to update your link command so
it links this patched _gcov.o instead of default one from
libgcov.a. Basically, you need to remove --coverage option from
your linker flags for that.
You may create a wrapper for __gcov_flush() declare and export it
as it suggested in the links above. Call this wrapper from your
signal handler in the shared library.
I suggest you not to add profiling for the small signal handler
library - it is really unnecessary.
I have a static library (lib.a) and a program that links to it. The library doesn't have any entry point that would always be called before using it, but I need to execute a piece of code very early in the program (preferably before main() starts). Therefore I thought I would use static variable of my own class. I added new source file that contains something like:
#include <MyClass.h>
static MyClass myVar;
The constructor of MyClass would then execute my code. When I link lib.a and try executing "nm" on it I get information that myVar is there. However, when I link my program and try "nm" on it I do not see myVar. When I put this piece of code into an existing file then the symbol is visible in the final executable. Why is that? Can linker omit object file from lib.a library in this case? I know that the variable is not referenced from outside (it cannot be as it is static) but it should execute code on it's own and therefore I don't get why should it be removed.
In case it makes a difference I'm using some old SunPro compiler.
Technically speaking, the linker should be forced to include that object file while compiling your program. However, support for this is buggy in many compilers, such as MSVC++. Adding an external reference somewhere in your main program should force that object file to be included.
Also note that in the case of nm, it's possible that your static initializer was inlined, and therefore the symbol need not exist in your final binary. Try something with side effects (such as a std::cout statement) in your static, and make sure it doesn't run before blaming the compiler :)
It turns out that what the linker does is pretty standard (I don't mean C++ standard, just generally observer behaviour) and you can work around it. In GNU ld it is --whole-archive option, in my case of Sun tools it is -z allextract. Which didn't actually work as expected for my project, so I used some magic with weak symbols an -z weakextract to achieve what I wanted.
I am writing a hello world c++ application, in the instruction #include help the compiler or linker to import the c++ library. My " cout << "hello world"; " use a cout in the library. The question is after compile and generated exe is about 96k in size, so what instructions are actually contained in this exe file, does this file also contains the iostream library?
Thanks
In the general case, the linker will only bring in what it needs. Once the compiler phase has turned your source code into an object file, it's treated much the same as all other object files. You have:
the C start-up code which prepares the execution environment (sets up argv, argv and so on) then calls your main or equivalent.
your code itself.
whatever object files need to be dragged in from libraries (dynamic linking is a special case of linking that happens at runtime and I won't cover that here since you asked specifically about static linking).
The linker will include all the object files you explicitly specify (unless it's a particularly smart linker and can tell you're not using the object file).
With libraries, it's a little different. Basically, you start with a list of unresolved symbols (like cout). The linker will search all the object files in all the libraries you specify and, when it finds an object file that satisfies that symbol, it will drag it in and fix up the symbol references.
This may, of course, add even more unresolved symbols if, for example, there was something in the object file that relies on the C printf function (unlikely but possible).
The linker continues like this until all symbols are satisfied (when it gives you an executable) or one cannot be satisfied (when it complains to you bitterly about your coding practices).
So as to what is in your executable, it may be the entire iostream library or it may just be the minimum required to do what you asked. It will usually depend on how many object files the iostream library was built into.
I've seen code where an entire subsystem went into one object file so, that if you wanted to just use one tiny bit, you still got the lot. Alternatively, you can put every single function into its own object file and the linker will probably create an executable as small as possible.
There are options to the linker which can produce a link map which will show you how things are organised. You probably won't generally see it if you're using the IDE but it'll be buried deep within the compile-time options dialogs under MSVC.
And, in terms of your added comment, the code:
cout << "hello";
will quite possibly bring in sizeable chunks of both the iostream and string processing code.
Use cl /EHsc hello.cpp -link /MAP. The .map file generated will give you a rough idea which pieces of the static library are present in the .exe.
Some of the space is used by C++ startup code, and the portions of the static library that you use.
In windows, the library or part of the libraries (which are used) are also usually included in the .exe, the case is different in case of Linux. However, there are optimization options.
I guess this Wiki link will be useful : Static Libraries