As I understand function-level linking builds (explicitly or not) a graph of all possible calls and only includes the reachable functions' code into the produced binary. But how does it deal with variables declared at file level?
Say I have
MyClass GlobalVariable;
static MyClass StaticGlobalVariable;
in some file that contains only these two variables and a set of functions not actually called from any of the remaining code.
Will the code for these variables allocation/initialization be included into the output?
From experience (rather than quoting the standard):
If the initilaization has visible side effects like calls into external libraries or file I/O, the initialization will always happen.
boost::singleton_default provides an interesting solution that enforces the initialization to be done only when the object is referenced elsewhere, i.e. when all other references to the object are removed by the linker, the initialization is removed, too.
Edit: Yes. g++ optimize flags try to figure out function calls and prune away .o files resulting in linker errors. I'm not sure if this happens only with certain optimize flags, but it does happen.
A bad habit in our company is the presence of a lot of 'extern g_GlobalFunction()' definitions in different files. As their calls depended on conditional code, the .o files were often thrown away, resulting in link errors.
We fixed that with g_InitModule() and g_InitFileName() calls that are called hierarchically starting from main(). Mostly, these are empty functions just meant to dissuade g++ from discarding the .o file.
Related
I'm currently updating a C++ library for Arduino (Specifically 8-bit AVR processors compiled using avr-gcc).
Typically the authors of the default Arduino libraries like to include an extern variable for the class inside the header, which is defined in the class .cpp file also. This I assume is basically to have everything provided ready to go for newbies as built-in objects.
The scenario I have is: The library I have updated no longer requires the .cpp file and I have removed it from the library. It wasn't until I went on a final pass checking for bugs that I realized, no linker error was produced despite the fact a definition wasn't provided for the extern variable in a .cpp file.
This is as simple as I can get it (header file):
struct Foo{
void method() {}
};
extern Foo foo;
Including this code and using it in one or many source files does not cause any linker error. I have tried it in both versions of GCC which Arduino uses (4.3.7, 4.8.1) and with C++11 enabled/disabled.
In my attempt to cause an error, I found it was only possible when doing something like taking the address of the object or modifying the contents of a dummy variable I added.
After discovering this I find its important to note:
The class functions only return other objects, as in, nothing like operators returning references to itself, or even a copy.
It only modifies external objects (registers which are effectively volatile uint8_t references in code), and returns temporaries of other classes.
All of the class functions in this header are so basic that they cost less than or equal to the cost of a function call, therefore they are (in my tests) completely in-lined into the caller. A typical statement may create many temporary objects in the call chain, however the compiler sees through these and outputs efficient code modifying registers directly, rather than a set of nested function calls.
I also recall reading in n3797 7.1.1 - 8 that extern can be used on incomplete types, however the class is fully defined whereas the declaration is not (this is probably irrelevant).
I'm led to believe that this may be a result of optimizations at play. I have seen the effect that taking the address has on objects which would otherwise be considered constant and compiled without RAM usage. By adding any layer of indirection to an object in which the compiler cannot guarantee state will cause this RAM consuming behavior.
So, maybe I've answered my question by simply asking it, however I'm still making assumptions and it bothers me. After quite some time hobby-coding C++, literally the only thing on my list of do-not's is making assumptions.
Really, what I want to know is:
With respect to the working solution I have, is it a simple case of documenting the inability to take the address (cause indirection) of the class?
Is it just an edge case behavior caused by optimizations eliminating the need for something to be linked?
Or is plain and simple undefined behavior. As in GCC may have a bug and is permitting code that might fail if optimizations were lowered or disabled?
Or one of you may be lucky enough to be in possession of a decoder ring that can find a suitable paragraph in the standard outlining the specifics.
This is my first question here, so let me know if you would like to know certain details, I can also provide GitHub links to the code if needed.
Edit: As the library needs to be compatible with existing code I need to maintain the ability to use the dot syntax, otherwise I'd simply have a class of static functions.
To remove assumptions for now, I see two options:
Add a .cpp just for the variable declaration.
Use a define in the header like #define foo (Foo()) allowing dot syntax via a temporary.
I prefer the method using a define, what does the community think?
Cheers.
Declaring something extern just informs the assembler and the linker that whenever you use that label/symbol, it should refer to entry in the symbol table, instead of a locally allocated symbol.
The role of the linker is to replace symbol table entries with an actual reference to the address space whenever possible.
If you don't use the symbol at all in your C file, it will not show up in the assembly code, and thus will not cause any linker error when your module is linked with others, since there is no undefined reference.
It is either an edge case behaviour caused by optimization, or you never use the foo variable in your code. I'm not 100% sure it is formally not an undefined behavior, but i'm quite sure it isn't undefined from practical point of view.
extern variables are implemented in such way, that code compiled with them produces so-called relocations - empty places where addres of variable should be placed - which are then filled by linker. Apparently foo is never used in your code in such a way that would need getting it's address and therefore linker doesn't even try to find that symbol. If you turn optimization off (-O0) you will probably get linker error.
Update: If you want to keep "dot notation" but remove the problem with undefined extern, you may replace extern with static (in header file), creating separate "instance" of variable for each TU. As this variable is going to be optimized out anyway, this will not change the real code at all, but will also work for unoptimized build.
I've created a simple static library, contained in a .a file. I might use it in a variety of projects, some of which simply will not need 90% of it. For example, if I want to use neural networks, which are a part of my library, on an AVR microcomputer, I probably wont need a tonne of other stuff, but will that be linked in my code potentially generating a rather large file?
I intend to compile programs like this:
g++ myProg.cpp myLib.a -o prog
G++ will pull in only the object files it needs from your library, but this means that if one symbol from a single object file is used, everything in that object file gets added to your executable.
One source file becomes one object file, so it makes sense to logically group things together only when they are sure to be needed together.
This practice varies by compiler (actually by linker). For example, the Microsoft linker will pick object files apart and only include those parts that actually are needed.
You could also try to break your library into independent smaller parts and only link the parts you are really going to need.
When you link to a static library the linker pulls in things that resolve names used in other parts of the code. In general, if the name isn't used it doesn't get linked in.
The GNU linker will pull in the stuff it needs from the libraries you have specified on an object file by object file basis. Object files are atomic units as far as the GNU linker is concerned. It doesn't split them apart. The linker will bring in an object file if that object file defines one or more unresolved external references. That object file may have external references. The linker will try to resolve these, but if it can't, the linker adds those to the set of references that need to be resolved.
There are a couple of gotchas that can make for a much larger than needed executable. By larger than needed, I mean an executable that contains functions that will never be called, global objects that will never be examined or modified, during the execution of the program. You will have binary code that is unreachable.
One of these gotchas results when an object file contains a large number of functions or global objects. Your program might only need one of these, but your executable gets all of them because object files are atomic units to the linker. Those extra functions will be unreachable because there's no call path from your main to these functions, but they're still in your executable. The only way to ensure that this doesn't happen is to use the "one function per source file" rule. I don't follow that rule myself, but I do understand the logic of it.
Another set of gotchas occur when you use polymorphic classes. A constructor contains auto-generated code as well as the body of the constructor itself. That auto-generated code calls the constructors for parent classes, inserts a pointer to the vtable for the class in the object, and initializes data members per the initializer list. These parent class constructors, the vtable, and the mechanisms to process the initializer list might be external references that the linker needs to resolve. If the parent class constructor is in a larger header file, you've just dragged all that stuff into your executable.
What about the vtable? The GNU compiler picks a key member function as the place to store the vtable. That key function is the first member function in the class that does not have a an inline definition. Even if you don't call that member function, you get the object file that contains it in your executable -- and you get everything that that object file drags in.
Keeping your source files down to a small size once again helps with this "look what the cat dragged in!" problem. It's a good idea to pay special attention to the file that contains that key member function. Keep that source file small, at least in terms of stuff the cat will drag in. I tend to put small, self-contained member functions in that source file. Functions that will inevitably drag in a bunch of other stuff shouldn't go there.
Another issue with the vtable is that it contains pointers to all of the virtual functions for a class. Those pointers need to point to something real. Your executable will contain the object files that define each and every virtual function defined for a class, including the ones you never call. And you're going to get everything that those virtual functions drag in as well.
One solution to this problem is to avoid making big huge classes. They tend to drag in everything. God classes in particular are problematic in this regard. Another solution is to think hard about whether a function really does need to be virtual. Don't just make a function virtual because you think someday someone will need to overload it. That's speculative generality, and with virtual functions, speculative generality comes with a high cost.
I have a function in legacy code, that is no longer being called.
My question is:
would the compiler optimize for a function which is not being called, or would the executable file include the code of that function?
possibly. it's implementation, toolset, and build parameter-defined.
altering your optimizations settings, linker flags, and the visibility (static/private/extern/internal/anonymous namespace) can increase the probability that it will be omitted from the final executable.
If it is compiled to object file, than compiler does not know if your function will be used or not. Unless you are using link time optimization (lto) or whole program optimization options. If function is in header - you can make it static, so that compiler can optimize it away.
There's a good chance it's in the file because of the possibility of run-time access through dynamic means. Such as a concatenated string yielding a plethora of different function names and being used to access them.
Although this type of implementation is rare, it is still a possibility and thus the code must remain available.
Dead code removal is usually done by the linker (as the compiler does not have a clue as to which functions are used or not). However, sometimes the compiler itself can remove functions that have static linkage.
This is because by default all functions have external linkage. The reserved term "extern" which is used while declaring external linkage variables could (and in fact is) omitting while declaring functions. So if those are not declared static those could be used elsewhere and compiler don't know nothing about it.
Also, GCC (if that's what you're using) has SSA Aggressive Dead Code Elimination (the -fssa-dce flag) which can help with removal of unneeded code.
If you're looking for something to remove dead functions or sections then you can use gcov http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html#Gcov-Intro
Lets say I have two .cpp files, file1.cpp and file2.cpp, which use std::vector<int>. Suppose that file1.cpp has a int main(void). If I compiled both into file1.o and file2.o, and linked the two object files into an elf binary which I can execute. I am compiling on a 32-bit Ubuntu Linux machine.
My question regards how the compiler and linker put together the symbols for the std::vector:
When the linker makes my final binary, is there code duplication? Does the linker have one set of "templated" code for the code in f1.o that uses std::vector and another set of std::vector code for the code that comprises f2.o?
I tried this for myself (I used g++ -g) and I looked at my final executable disassembly, and I found the labels generated for the vector constructor and other methods were apparently random, although the code from f1.o appeared to have called the same constructor as the code from f2.o. I could not be sure, however.
If the linker does prevent the code duplication, how does it do it? Must it "know" what templates are? Does it always prevent code duplication regarding multiple uses of the same templated code across multiple object files?
It knows what the templates are through name mangling. The type of the object is encoded by the compiler in its name, and that allows the linker to filter out the duplicate implementations of the same template.
This is done during linking, and not compilation, because each .o file can be linked with anything thus cannot be stripped of something that may later be needed. Only the linker can decide which code is unused, which template is duplicate, etc. This is done by using "Weak Symbols" in the object's symbol list: Symbols that the linker can remove if they appear multiple times (as opposed to other symbols, like user-defined functions, that cannot be removed if duplicate and cause a linking error).
Your question is stated verbatim in the opening section of this documentation:
http://gcc.gnu.org/onlinedocs/gcc/Template-Instantiation.html
Technically due to the "one definition rule" there is only one std::vector<int> and therefore the code should be linked together. What may happen is that some code is inlined which would speed up execution time but could produce more code.
If you had one file using std::vector<int> and another using std::vector<unsigned int> then you would have 2 classes and potentially lots of duplicate code.
Of course the writers of vector might use some common code for certain situations eg POD types that removes the duplication.
I'm using static initialisation to ease the process of registering some classes with a factory in C++. Unfortunately, I think the compiler is optimising out the 'unused' objects which are meant to do the useful work in their constructors. Is there any way to tell the compiler not to optimise out a global variable?
class SomeClass {
public:
SomeClass() {
/* do something useful */
}
};
SomeClass instance;
My breakpoint in SomeClass's constructor doesn't get hit. In my actual code, SomeClass is in a header file and instance is in a source file, more or less alone.
EDIT: As guessed by KJAWolf, this code is actually compiled into a static lib, not the executable. Its purpose is to register some types also provided by the static lib with a static list of types and their creators, for a factory to then read from on construction. Since these types are provided with the lib, adding this code to the executable is undesirable.
Also I discovered that by moving the code to another source file that contains other existing code, it works fine. It seems that having a file purely consisting of these global objects is what's causing the problem. It's as if that translation unit was entirely ignored.
The compiler is not allowed to optimiza away global objects.
Even if they are never used.
Somthing else is happening in your code.
Now if you built a static library with your global object and that global object is not referenced from the executable it will not be pulled into the executable by the linker.
The compiler should never optimize away such globals - if it does that, it is simply broken.
You can force that one object (your list of types) pulls some other objects with it by partially linking them before building the complete static lib.
With GNU linker:
ld -Ur -o TypeBundle.o type1.o type2.o type3.o static_list.o
ld -static -o MyStaticLib.a type_bundle.o other_object.o another_object.o ...
Thus, whenever the static list is referenced by code using the library, the complete "TypeBundle.o" object will get linked into the resulting binary, including type1.o, type2.o, and type3.o.
While at it, do check out the meaning of "-Ur" in the manual.
To build off of Arthur Ulfeldt, volatile tells the compiler that this variable can change outside of the knowledge of the compiler. I've used it for put a statement to allow the debugger to set a breakpoint. It's also useful for hardware registers that can change based on the environment or that need a special sequence. i.e. Serial Port receive register and certain watchdog registers.
you could use
#pragma optimize off
int globalVar
#pragma optimize on
but I dunno if that only works in Visual Studio ( http://msdn.microsoft.com/en-us/library/chh3fb0k(VS.80).aspx ).
You could also tell the compiler to not optimize at all, especially if you're debugging...
Are you using gcc with gdb? There was a problem in the past where gdb could not accurately set breakpoints in constructors.
Also, are you using an optimization level which allows the compiler to inline methods in the class definition.
You need to use -whole-archive when linking. See the answer here:
ld linker question: the --whole-archive option
I have same setup & problem on VS2008.
I found that if you declare you class with dllexport it will not optimize.
class __declspec( dllexport ) Cxxx
{
.
}
However this generates a lot of warnings in my case because I must declare all classes used in this class also as dllexport.
All optimizations are off (in debug mode), still this is optimized. Also volatile/pragma optimize off. On global variable created of this class (in same cpp file) etc does not work.
Just found that dllexport does require at least to include header files of these classes in some other cpp file from exe to work! So only option is to add a file with calls to some static members for each class, and add this file to all projects used these classes.
It would not be a compiler, but the library linker (or the tool that pools object files into the .lib), who can decide that the whole file is not used, so discard it.
A workaround would be to add an empty function to that file and call it from a file that contains stuff that is called directly.
How about using the keyword volatile? It will prevent the compiler from too much optimization.