I'm currently updating a C++ library for Arduino (Specifically 8-bit AVR processors compiled using avr-gcc).
Typically the authors of the default Arduino libraries like to include an extern variable for the class inside the header, which is defined in the class .cpp file also. This I assume is basically to have everything provided ready to go for newbies as built-in objects.
The scenario I have is: The library I have updated no longer requires the .cpp file and I have removed it from the library. It wasn't until I went on a final pass checking for bugs that I realized, no linker error was produced despite the fact a definition wasn't provided for the extern variable in a .cpp file.
This is as simple as I can get it (header file):
struct Foo{
void method() {}
};
extern Foo foo;
Including this code and using it in one or many source files does not cause any linker error. I have tried it in both versions of GCC which Arduino uses (4.3.7, 4.8.1) and with C++11 enabled/disabled.
In my attempt to cause an error, I found it was only possible when doing something like taking the address of the object or modifying the contents of a dummy variable I added.
After discovering this I find its important to note:
The class functions only return other objects, as in, nothing like operators returning references to itself, or even a copy.
It only modifies external objects (registers which are effectively volatile uint8_t references in code), and returns temporaries of other classes.
All of the class functions in this header are so basic that they cost less than or equal to the cost of a function call, therefore they are (in my tests) completely in-lined into the caller. A typical statement may create many temporary objects in the call chain, however the compiler sees through these and outputs efficient code modifying registers directly, rather than a set of nested function calls.
I also recall reading in n3797 7.1.1 - 8 that extern can be used on incomplete types, however the class is fully defined whereas the declaration is not (this is probably irrelevant).
I'm led to believe that this may be a result of optimizations at play. I have seen the effect that taking the address has on objects which would otherwise be considered constant and compiled without RAM usage. By adding any layer of indirection to an object in which the compiler cannot guarantee state will cause this RAM consuming behavior.
So, maybe I've answered my question by simply asking it, however I'm still making assumptions and it bothers me. After quite some time hobby-coding C++, literally the only thing on my list of do-not's is making assumptions.
Really, what I want to know is:
With respect to the working solution I have, is it a simple case of documenting the inability to take the address (cause indirection) of the class?
Is it just an edge case behavior caused by optimizations eliminating the need for something to be linked?
Or is plain and simple undefined behavior. As in GCC may have a bug and is permitting code that might fail if optimizations were lowered or disabled?
Or one of you may be lucky enough to be in possession of a decoder ring that can find a suitable paragraph in the standard outlining the specifics.
This is my first question here, so let me know if you would like to know certain details, I can also provide GitHub links to the code if needed.
Edit: As the library needs to be compatible with existing code I need to maintain the ability to use the dot syntax, otherwise I'd simply have a class of static functions.
To remove assumptions for now, I see two options:
Add a .cpp just for the variable declaration.
Use a define in the header like #define foo (Foo()) allowing dot syntax via a temporary.
I prefer the method using a define, what does the community think?
Cheers.
Declaring something extern just informs the assembler and the linker that whenever you use that label/symbol, it should refer to entry in the symbol table, instead of a locally allocated symbol.
The role of the linker is to replace symbol table entries with an actual reference to the address space whenever possible.
If you don't use the symbol at all in your C file, it will not show up in the assembly code, and thus will not cause any linker error when your module is linked with others, since there is no undefined reference.
It is either an edge case behaviour caused by optimization, or you never use the foo variable in your code. I'm not 100% sure it is formally not an undefined behavior, but i'm quite sure it isn't undefined from practical point of view.
extern variables are implemented in such way, that code compiled with them produces so-called relocations - empty places where addres of variable should be placed - which are then filled by linker. Apparently foo is never used in your code in such a way that would need getting it's address and therefore linker doesn't even try to find that symbol. If you turn optimization off (-O0) you will probably get linker error.
Update: If you want to keep "dot notation" but remove the problem with undefined extern, you may replace extern with static (in header file), creating separate "instance" of variable for each TU. As this variable is going to be optimized out anyway, this will not change the real code at all, but will also work for unoptimized build.
Related
If I declare a large function inline, and the compiler chooses to compile it as a standalone function anyway, is there guaranteed to be only a single copy of that non-inlined inline function in the program, or might the compiler end up creating multiple non-inlined copies of this function in different translation units?
Or, to rephrase that in a way that's almost certain to cause much gnashing of teeth: in cases where preprocessing time is not a concern, is defining a large method inline in a header file an effective way of being able to reuse that code without having to compile/link in a separate translation unit?
You need to be extremely specific here, as you are asking for a guarantee. The C++ standard makes guarantees about program behaviour, not about implementation, and even then is not written to protect against malicious compilers. Many implementation details are effectively constrained by behaviour. But not completely.
Compilers are free to embed in your executable 2^17 different copies of any function, inline or not. This would be pointless, but the standard does not ban it! The value and address of static variables would have to be shared, as that is observable, and all the function pointers to each function would have to compare equal if C++ code could get at them (they could have distinct binary representation of the address, just change what it means to do a ==!).
Does this happen? No, but you asked for a guarantee. The C++ standard leaves huge latitude for implementors. Quality of implementation means that modern compilers don't do stupid things that often.
In practice, one inline function is created in each .o file that uses it. It is marked as special ("weak"). When linking statically, all but one of these copies are discarded. Which one is kept will depend on linking order, and can vary from build to build (esp. partial builds). Only one copy of the static locals are kept in a simular way. All pointers to the function or static locals within must compare equal at runtime.
Dynamical linking results in some compilers discarding it when being built into dlls. Other compilers when the .so is loaded look to see if the symbol was already loaded, and if so do not load the copy in the .so. Dynamic linking is the case where it is most likely for multiple copies of the function to continue to exist and be accessed. It this case is not occurring, you are clear, and if it is, test it. C++ standard does not describe dynamic linking.
If I declare a large function inline, and the compiler chooses to compile it as a standalone function anyway, is there guaranteed to be only a single copy of that non-inlined inline function in the program,
No. There is no such guarantee. What is guaranteed, is that there will only be a single copy of any static variables declared in the function, and that if you take the address of the function, you will always get the same value.
or might the compiler end up creating multiple non-inlined copies of this function in different translation units?
It might. They might not even be identical: If it can see the argument is always null in one translation unit, it can omit the code for if (arg != nullptr)...`.
I would recommend you to go through below question:
"inline" keyword vs "inlining" concept
For optimization purpose, if the inline function is:
large enough, then compiler may not do "inlining"
short enough, then compiler might do "inlining"
However, in both the cases One Definition Rule (ODR) is guaranteed, since you have used inline keyword.
First I'll try to describe the current situation:
I am adapting an existing code base for our uses and in some instances a .h/.cpp file contains multiple class definitions. We cannot change the existing public interface of the API without major modifications to other portions of the code that we would rather avoid at this time.
I have found the need for constant values that are used by more than one class (in the same .cpp file and nowhere else) and therefore cannot be defined as a class-specific constant. The only way I know to do this is to define the constants externally from any class (but still in the .cpp file) and reference them as needed.
I have done this and the code compiles and links, but when I go to run a test program on the code it fails with an error that appears to be related to the constant value I have defined. I get the impression that when the code executes, the constant has not actually been defined and the code blows up because of that.
I don't have a lot of experience writing C++ code and wonder if I'm doing this the wrong way. I will include code snippets below to try to illustrate what I'm doing.
In DateTime.cpp (currently nothing defined in DateTime.h for DATE_FORMAT_REGEX):
...
#include <boost/regex.hpp>
static const boost::regex DATE_FORMAT_REGEX("[0-9]{4}-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])(Z|([+|-]([01][0-9]|2[0-4]):[0-5][0-9]))?");
// Other class method implementations ...
bool our_ns::Date::parse_string(const std::string& s)
{
// Validate string against regex.
bool valid = boost::regex_match(s, DATE_FORMAT_REGEX);
if (valid) {
...
}
}
...
The call to regex_match is what fails. All of the classes, by the way are defined within a namespace in the header file.
It would appear the constant is not being initialized. What do I need to do to initialize these values? Based upon what I've described, is there a better way to do this?
[Update: 6/9/15 12:52 EDT]
Actually, I am relaying information witnessed by another developer in the organization. He verified in a debugger that the regex item was null when he reached the line where it blows up. He also mentioned that when he did some experimentation and moved the definitions from the .cpp to the .h file, the error did not recur. Beyond that, I don't know if things were executing properly.
The correct way is to not defined these as static const but rather as const. These are constants, they do not need static linkage. If you're doing it to avoid the global namespace, const variables have implicit compilation unit scoping anyway.
If Date::parse_string is called from a Date constructor and the Date object is static and initialized in a different compilation unit (C++ file) then it could be an issue related to static variable initialization order. In this case the order of initialization of variables is not defined so the Date object might be initialized before DATE_FORMAT_REGEX and as a result parse_string is called before DATE_FORMAT_REGEX is initialized. You can fix it by moving DATE_FORMAT_REGEX to the Date class definition.
This is likely a result of initialization order, a well-known issue that one could run into.
It has been described in books, such as C++ FAQs in detail for a very long time. Here is a quick reference:
https://isocpp.org/wiki/faq/ctors#static-init-order
Basically, there is no guaranteed order in which static variables must initialize. This creates situations where you could run into an error, when the "second" object makes use of the "first", but, "first" hasn't been initialized. Please note that "first" and "second" are interchangeable, therein lies the crux of the problem.
In fact, the more dangerous situation is a latent bug, where things continue to work until one day when they don't - usually, induced by a new compiler version or some such change.
It is better to get out of this dependency altogether. You don't seem to need to static here. If the goal is to limit visibility of the variable, then use anonymous namespace and place it before any use.
On my project I often see people defining global functions in .cpp files, i.e functions that are not restricted to file scope, class scope or any particular namespace.
These are clearly local helper functions that the author only wanted to be able to access in that file.
I know this is bad practice and the solution is to restrict them to file scope either by using the static keyword or better yet use an anonymous namespace.
But my question is, if these functions are not declared in the header file, what can actually go wrong?
I would like to advise these people against this practice but I feel my argument would have more weight if I could clearly describe what could go wrong. Or even what what might already be going wrong that we are not aware of!
Thanks.
One, you are cluttering the namespace. The result can be multiple definitions, i.e. linker errors, and programmers choosing awkward function names to circumvent this. Imagine one source file defining its helper() function, the next one a my_helper() because helper() resulted in an error, then a third a other_helper() and so on... in any case, the cleaner the namespace, the easier it becomes to understand what is actually going on.
Two, and this is an extension of the above, imagine a helper( int x ) and a helper( long y ), and you can imagine the kind of ambiguity that could arise from this. If you are lucky (and using appropriate warning options), the compiler will warn you about these conditions, but you might end up calling a different function than what you expected.
Three, and this is from a maintainer's point of view, if you see a function that is static or declared in an anonymous namespace, you know that you only have to check the current source file for calls to this function. This makes refactorings that much easier. ("Does anyone actually use this exotic but buggy feature, or can I optimize it away?")
Ulrich Drepper's paper on shared ELF libraries is relevant for you if you produce dynamically shared objects, usually shared libraries. I assume that some considerations also apply to applications which just are dynamically linked. The paper discusses the GNU tools but similar concerns will likely apply to other tool chains.
In short there may be build-time, load-time and run-time penalties for global objects and functions.
Build- and load-time penalties are rooted in the number of (string) comparisons needed to resolve dependencies which are not necessary for locally-defined symbols like file static functions and variables. Drepper discusses this at page 8 using the example of OpenOffice.
The reason for run-time penalties is ELF specifying that even locally-defined but global symbols could be replaced at run time with definitions in other objects. Therefore function code cannot be inlined and further optimized, even though it is visible at compile time; and the function call proper is more complicated than necessary, involving more indirections. See Drepper's paper, pp. 17 and 18.
I have a function in legacy code, that is no longer being called.
My question is:
would the compiler optimize for a function which is not being called, or would the executable file include the code of that function?
possibly. it's implementation, toolset, and build parameter-defined.
altering your optimizations settings, linker flags, and the visibility (static/private/extern/internal/anonymous namespace) can increase the probability that it will be omitted from the final executable.
If it is compiled to object file, than compiler does not know if your function will be used or not. Unless you are using link time optimization (lto) or whole program optimization options. If function is in header - you can make it static, so that compiler can optimize it away.
There's a good chance it's in the file because of the possibility of run-time access through dynamic means. Such as a concatenated string yielding a plethora of different function names and being used to access them.
Although this type of implementation is rare, it is still a possibility and thus the code must remain available.
Dead code removal is usually done by the linker (as the compiler does not have a clue as to which functions are used or not). However, sometimes the compiler itself can remove functions that have static linkage.
This is because by default all functions have external linkage. The reserved term "extern" which is used while declaring external linkage variables could (and in fact is) omitting while declaring functions. So if those are not declared static those could be used elsewhere and compiler don't know nothing about it.
Also, GCC (if that's what you're using) has SSA Aggressive Dead Code Elimination (the -fssa-dce flag) which can help with removal of unneeded code.
If you're looking for something to remove dead functions or sections then you can use gcov http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html#Gcov-Intro
In C++, declaration and definition of functions, variables and constants can be separated like so:
function someFunc();
function someFunc()
{
//Implementation.
}
In fact, in the definition of classes, this is often the case. A class is usually declared with it's members in a .h file, and these are then defined in a corresponding .C file.
What are the advantages & disadvantages of this approach?
Historically this was to help the compiler. You had to give it the list of names before it used them - whether this was the actual usage, or a forward declaration (C's default funcion prototype aside).
Modern compilers for modern languages show that this is no longer a necessity, so C & C++'s (as well as Objective-C, and probably others) syntax here is histotical baggage. In fact one this is one of the big problems with C++ that even the addition of a proper module system will not solve.
Disadvantages are: lots of heavily nested include files (I've traced include trees before, they are surprisingly huge) and redundancy between declaration and definition - all leading to longer coding times and longer compile times (ever compared the compile times between comparable C++ and C# projects? This is one of the reasons for the difference). Header files must be provided for users of any components you provide. Chances of ODR violations. Reliance on the pre-processor (many modern languages do not need a pre-processor step), which makes your code more fragile and harder for tools to parse.
Advantages: no much. You could argue that you get a list of function names grouped together in one place for documentation purposes - but most IDEs have some sort of code folding ability these days, and projects of any size should be using doc generators (such as doxygen) anyway. With a cleaner, pre-processor-less, module based syntax it is easier for tools to follow your code and provide this and more, so I think this "advantage" is just about moot.
It's an artefact of how C/C++ compilers work.
As a source file gets compiled, the preprocessor substitutes each #include-statement with the contents of the included file. Only afterwards does the compiler try to interpret the result of this concatenation.
The compiler then goes over that result from beginning to end, trying to validate each statement. If a line of code invokes a function that hasn't been defined previously, it'll give up.
There's a problem with that, though, when it comes to mutually recursive function calls:
void foo()
{
bar();
}
void bar()
{
foo();
}
Here, foo won't compile as bar is unknown. If you switch the two functions around, bar won't compile as foo is unknown.
If you separate declaration and definition, though, you can order the functions as you wish:
void foo();
void bar();
void foo()
{
bar();
}
void bar()
{
foo();
}
Here, when the compiler processes foo it already knows the signature of a function called bar, and is happy.
Of course compilers could work in a different way, but that's how they work in C, C++ and to some degree Objective-C.
Disadvantages:
None directly. If you're using C/C++ anyway, it's the best way to do things. If you've got a choice of language/compiler, then maybe you can pick one where this is not an issue. The only thing to consider with splitting declarations into header files is to avoid mutually recursive #include-statements - but that's what include guards are for.
Advantages:
Compilation speed: As all included files are concatenated and then parsed, reducing the amount and complexity of code in included files will improve compilation time.
Avoid code duplication/inlining: If you fully define a function in a header file, each object file that includes this header and references this function will contain it's own version of that function. As a side-note, if you want inlining, you need to put the full definition into the header file (on most compilers).
Encapsulation/clarity: A well defined class/set of functions plus some documentation should be enough for other developers to use your code. There is (ideally) no need for them to understand how the code works - so why require them to sift through it? (The counter-argument that it's may be useful for them to access the implementation when required still stands, of course).
And of course, if you're not interested in exposing a function at all, you can usually still choose to define it fully in the implementation file rather than the header.
The standard requires that when using a function, a declaration must be in scope. This means, that the compiler should be able to verify against a prototype (the declaration in a header file) what you are passing to it. Except of course, for functions that are variadic - such functions do not validate arguments.
Think of C, when this was not required. At that time, compilers treated no return type specification to be defaulted to int. Now, assume you had a function foo() which returned a pointer to void. However, since you did not have a declaration, the compiler will think that it has to return an integer. On some Motorola systems for example, integeres and pointers would be be returned in different registers. Now, the compiler will no longer use the correct register and instead return your pointer cast to an integer in the other register. The moment you try to work with this pointer -- all hell breaks loose.
Declaring functions within the header is fine. But remember if you declare and define in the header make sure they are inline. One way to achieve this is to put the definition inside the class definition. Otherwise prepend the inline keyword. You will run into ODR violation otherwise when the header is included in multiple implementation files.
There are two main advantages to separating declaration and definition into C++ header and source files. The first is that you avoid problems with the One Definition Rule when your class/functions/whatever are #included in more than one place. Secondly, by doing things this way, you separate interface and implementation. Users of your class or library need only to see your header file in order to write code that uses it. You can also take this one step farther with the Pimpl Idiom and make it so that user code doesn't have to recompile every time the library implementation changes.
You've already mentioned the disadvantage of code repetition between the .h and .cpp files. Maybe I've written C++ code for too long, but I don't think it's that bad. You have to change all user code every time you change a function signature anyway, so what's one more file? It's only annoying when you're first writing a class and you have to copy-and-paste from the header to the new source file.
The other disadvantage in practice is that in order to write (and debug!) good code that uses a third-party library, you usually have to see inside it. That means access to the source code even if you can't change it. If all you have is a header file and a compiled object file, it can be very difficult to decide if the bug is your fault or theirs. Also, looking at the source gives you insight into how to properly use and extend a library that the documentation might not cover. Not everyone ships an MSDN with their library. And great software engineers have a nasty habit of doing things with your code that you never dreamed possible. ;-)
Advantage
Classes can be referenced from other files by just including the declaration. Definitions can then be linked later on in the compilation process.
You basically have 2 views on the class/function/whatever:
The declaration, where you declare the name, the parameters and the members (in the case of a struct/class), and the definition where you define what the functions does.
Amongst the disadvantages are repetition, yet one big advantage is that you can declare your function as int foo(float f) and leave the details in the implementation(=definition), so anyone who wants to use your function foo just includes your header file and links to your library/objectfile, so library users as well as compilers just have to care for the defined interface, which helps understanding the interfaces and speeds up compile times.
One advantage that I haven't seen yet: API
Any library or 3rd party code that is NOT open source (i.e. proprietary) will not have their implementation along with the distribution. Most companies are just plain not comfortable with giving away source code. The easy solution, just distribute the class declarations and function signatures that allow use of the DLL.
Disclaimer: I'm not saying whether it's right, wrong, or justified, I'm just saying I've seen it a lot.
One big advantage of forward declarations is that when used carefully you can cut down the compile time dependencies between modules.
If ClassA.h needs to refer to a data element in ClassB.h, you can often use just a forward references in ClassA.h and include ClassB.h in ClassA.cc rather than in ClassA.h, thus cutting down a compile time dependency.
For big systems this can be a huge time saver on a build.
Disadvantage
This leads to a lot of repetition. Most of the function signature needs to be put in two or more (as Paulious noted) places.
Separation gives clean, uncluttered view of program elements.
Possibility to create and link to binary modules/libraries without disclosing sources.
Link binaries without recompiling sources.
When done correctly, this separation reduces compile times when only the implementation has changed.