I have a function in legacy code, that is no longer being called.
My question is:
would the compiler optimize for a function which is not being called, or would the executable file include the code of that function?
possibly. it's implementation, toolset, and build parameter-defined.
altering your optimizations settings, linker flags, and the visibility (static/private/extern/internal/anonymous namespace) can increase the probability that it will be omitted from the final executable.
If it is compiled to object file, than compiler does not know if your function will be used or not. Unless you are using link time optimization (lto) or whole program optimization options. If function is in header - you can make it static, so that compiler can optimize it away.
There's a good chance it's in the file because of the possibility of run-time access through dynamic means. Such as a concatenated string yielding a plethora of different function names and being used to access them.
Although this type of implementation is rare, it is still a possibility and thus the code must remain available.
Dead code removal is usually done by the linker (as the compiler does not have a clue as to which functions are used or not). However, sometimes the compiler itself can remove functions that have static linkage.
This is because by default all functions have external linkage. The reserved term "extern" which is used while declaring external linkage variables could (and in fact is) omitting while declaring functions. So if those are not declared static those could be used elsewhere and compiler don't know nothing about it.
Also, GCC (if that's what you're using) has SSA Aggressive Dead Code Elimination (the -fssa-dce flag) which can help with removal of unneeded code.
If you're looking for something to remove dead functions or sections then you can use gcov http://gcc.gnu.org/onlinedocs/gcc/Gcov-Intro.html#Gcov-Intro
Related
If I declare a large function inline, and the compiler chooses to compile it as a standalone function anyway, is there guaranteed to be only a single copy of that non-inlined inline function in the program, or might the compiler end up creating multiple non-inlined copies of this function in different translation units?
Or, to rephrase that in a way that's almost certain to cause much gnashing of teeth: in cases where preprocessing time is not a concern, is defining a large method inline in a header file an effective way of being able to reuse that code without having to compile/link in a separate translation unit?
You need to be extremely specific here, as you are asking for a guarantee. The C++ standard makes guarantees about program behaviour, not about implementation, and even then is not written to protect against malicious compilers. Many implementation details are effectively constrained by behaviour. But not completely.
Compilers are free to embed in your executable 2^17 different copies of any function, inline or not. This would be pointless, but the standard does not ban it! The value and address of static variables would have to be shared, as that is observable, and all the function pointers to each function would have to compare equal if C++ code could get at them (they could have distinct binary representation of the address, just change what it means to do a ==!).
Does this happen? No, but you asked for a guarantee. The C++ standard leaves huge latitude for implementors. Quality of implementation means that modern compilers don't do stupid things that often.
In practice, one inline function is created in each .o file that uses it. It is marked as special ("weak"). When linking statically, all but one of these copies are discarded. Which one is kept will depend on linking order, and can vary from build to build (esp. partial builds). Only one copy of the static locals are kept in a simular way. All pointers to the function or static locals within must compare equal at runtime.
Dynamical linking results in some compilers discarding it when being built into dlls. Other compilers when the .so is loaded look to see if the symbol was already loaded, and if so do not load the copy in the .so. Dynamic linking is the case where it is most likely for multiple copies of the function to continue to exist and be accessed. It this case is not occurring, you are clear, and if it is, test it. C++ standard does not describe dynamic linking.
If I declare a large function inline, and the compiler chooses to compile it as a standalone function anyway, is there guaranteed to be only a single copy of that non-inlined inline function in the program,
No. There is no such guarantee. What is guaranteed, is that there will only be a single copy of any static variables declared in the function, and that if you take the address of the function, you will always get the same value.
or might the compiler end up creating multiple non-inlined copies of this function in different translation units?
It might. They might not even be identical: If it can see the argument is always null in one translation unit, it can omit the code for if (arg != nullptr)...`.
I would recommend you to go through below question:
"inline" keyword vs "inlining" concept
For optimization purpose, if the inline function is:
large enough, then compiler may not do "inlining"
short enough, then compiler might do "inlining"
However, in both the cases One Definition Rule (ODR) is guaranteed, since you have used inline keyword.
The reason we have to define inline functions in the header is that each compilation unit where that function is called must have the entire definition in order to replace the call, or substitute it. My question is why are we forced to put a definition in a header file if the compiler can and does do its own optimisations of inlining, which would require it to dig into the cpp files where the functions are defined anyway.
In other words, the compiler seems to me to have the ability to see the function "declaration" in a header file, go to the corresponding cpp file and pull the definition from it and paste it in the appropriate spot in the other cpp. Given that this is the case, why the insistence of defining the function in the header, implying as if the compiler can't "see" into other cpp files.
The MSDN says about Ob2/ optimisation setting:
Ob2/ The default value. Allows expansion of functions marked as inline, __inline, or __forceinline, and any other function that the compiler chooses (My emphasis).
The reason we're forced to provide definitions of inline function in header files (or at least, in some form that is visible to the implementation when inlining a function in a given compilation unit) is requirements of the C++ standard.
However, the standard does not go out of its way to prevent implementations (e.g. the toolchain or parts of it, such as the preprocessor, compiler proper, linker, etc) from doing things a little smarter.
Some particular implementations do things a little smarter, so can actually inline functions even in circumstances where they are not visible to the compiler. For example, in a basic "compile all the source files then link" toolchain, a smart linker may realise that a function is small and only called a few times, and elect to (in effect) inline it, even if the points where inlining occurs were not visible to the compiler (e.g. because the statements that called the functions were in separate compilation units, the function itself is in another compilation unit) so the compiler would not do inlining.
The thing is, the standard does not prevent an implementation from doing that. It simply states the minimum set of requirements for behaviour of ALL implementations.
Essentially, the requirement that the compiler have visibility of a function to be inlined is the minimum requirement from the standard. If a program is written in that way (e.g. all functions to be inlined are defined in their header file) then the standard guarantees that it will work with every (standard compliant) implementation.
But what does this mean for our smarter tool-chain? The smarter tool-chain must produce correct results from a program that is well-formed - including one that defines inlined functions in every compilation unit which uses those functions. Our toolchain is permitted to do things smarter (e.g. peeking between compilation units) but, if code is written in a way that REQUIRES such smarter behaviour (e.g. that a compiler peek between compilation units) that code may be rejected by another toolchain.
In the end, every C++ implementation (the toolchain, standard library, etc) is required to comply with requirements of the C++ standard. The reverse is not true - one implementation may do things smarter than the standard requires, but that doesn't generate a requirement that some other implementation do things in a compatible way.
Technically, inlining is not limited to being a function of the compiler. It may happen in the compiler or the linker. It may also happen at run time - for example "Just In Time" technology can, in effect, restructure executable code after it has been run a few times in order to enhance subsequent performance [this typically occurs in a virtual machine environment, which permits the benefits of such techniques while avoiding problems associated with self-modifying executables].
The inline keyword isn't just about expanding the implementation at the point it was called, but in fact primarily about declaring that multiple definitions of a function may exist in a given translation unit.
This has been covered in other questions before, which can it explain much better than I :)
Why are class member functions inlined?
Is "inline" implicit in C++ member functions defined in class definition
No, compilers traditionally can't do this. In classic model, compiler 'sees' only one cpp file at a time, and can't go to any other cpp files. Out of this cpp file compiler so-called object file in platofirm native format, which is than linked using effectively linker from 1970s, which is as dumb as a hammer.
This model is slowly evolving. With more and more effective link-time optimizations (LTO) linkers become aware of what cpp code is, and can perform their own inlining. However, even with link-time optimization model compiler-done inlining and optimization are still way more efficient than link-time - a lot of important context is lost when cpp code is converted to intermediate format suitable for linking.
It's much easier for the compiler to expand a function inline if it has seen the definition of that function. The easiest way to let the compiler see the definition of a function in every translation unit that uses that function is to put the definition in a header and #include that header wherever the function will be used. When you do that you have to mark the definition as inline so that the compiler (actually the linker) won't complain about seeing the definition of that function in more than one translation unit.
I'm currently updating a C++ library for Arduino (Specifically 8-bit AVR processors compiled using avr-gcc).
Typically the authors of the default Arduino libraries like to include an extern variable for the class inside the header, which is defined in the class .cpp file also. This I assume is basically to have everything provided ready to go for newbies as built-in objects.
The scenario I have is: The library I have updated no longer requires the .cpp file and I have removed it from the library. It wasn't until I went on a final pass checking for bugs that I realized, no linker error was produced despite the fact a definition wasn't provided for the extern variable in a .cpp file.
This is as simple as I can get it (header file):
struct Foo{
void method() {}
};
extern Foo foo;
Including this code and using it in one or many source files does not cause any linker error. I have tried it in both versions of GCC which Arduino uses (4.3.7, 4.8.1) and with C++11 enabled/disabled.
In my attempt to cause an error, I found it was only possible when doing something like taking the address of the object or modifying the contents of a dummy variable I added.
After discovering this I find its important to note:
The class functions only return other objects, as in, nothing like operators returning references to itself, or even a copy.
It only modifies external objects (registers which are effectively volatile uint8_t references in code), and returns temporaries of other classes.
All of the class functions in this header are so basic that they cost less than or equal to the cost of a function call, therefore they are (in my tests) completely in-lined into the caller. A typical statement may create many temporary objects in the call chain, however the compiler sees through these and outputs efficient code modifying registers directly, rather than a set of nested function calls.
I also recall reading in n3797 7.1.1 - 8 that extern can be used on incomplete types, however the class is fully defined whereas the declaration is not (this is probably irrelevant).
I'm led to believe that this may be a result of optimizations at play. I have seen the effect that taking the address has on objects which would otherwise be considered constant and compiled without RAM usage. By adding any layer of indirection to an object in which the compiler cannot guarantee state will cause this RAM consuming behavior.
So, maybe I've answered my question by simply asking it, however I'm still making assumptions and it bothers me. After quite some time hobby-coding C++, literally the only thing on my list of do-not's is making assumptions.
Really, what I want to know is:
With respect to the working solution I have, is it a simple case of documenting the inability to take the address (cause indirection) of the class?
Is it just an edge case behavior caused by optimizations eliminating the need for something to be linked?
Or is plain and simple undefined behavior. As in GCC may have a bug and is permitting code that might fail if optimizations were lowered or disabled?
Or one of you may be lucky enough to be in possession of a decoder ring that can find a suitable paragraph in the standard outlining the specifics.
This is my first question here, so let me know if you would like to know certain details, I can also provide GitHub links to the code if needed.
Edit: As the library needs to be compatible with existing code I need to maintain the ability to use the dot syntax, otherwise I'd simply have a class of static functions.
To remove assumptions for now, I see two options:
Add a .cpp just for the variable declaration.
Use a define in the header like #define foo (Foo()) allowing dot syntax via a temporary.
I prefer the method using a define, what does the community think?
Cheers.
Declaring something extern just informs the assembler and the linker that whenever you use that label/symbol, it should refer to entry in the symbol table, instead of a locally allocated symbol.
The role of the linker is to replace symbol table entries with an actual reference to the address space whenever possible.
If you don't use the symbol at all in your C file, it will not show up in the assembly code, and thus will not cause any linker error when your module is linked with others, since there is no undefined reference.
It is either an edge case behaviour caused by optimization, or you never use the foo variable in your code. I'm not 100% sure it is formally not an undefined behavior, but i'm quite sure it isn't undefined from practical point of view.
extern variables are implemented in such way, that code compiled with them produces so-called relocations - empty places where addres of variable should be placed - which are then filled by linker. Apparently foo is never used in your code in such a way that would need getting it's address and therefore linker doesn't even try to find that symbol. If you turn optimization off (-O0) you will probably get linker error.
Update: If you want to keep "dot notation" but remove the problem with undefined extern, you may replace extern with static (in header file), creating separate "instance" of variable for each TU. As this variable is going to be optimized out anyway, this will not change the real code at all, but will also work for unoptimized build.
Compilers these days tend to do a significant amount of optimizations. Do they also remove unused functions from the final output?
It depends on the compiler. Visual C++ 9 can do that - unused static functions are removed at compilation phase (there's even a C4505 warning for that), unused functions with external linkage can be removed at link phase depending on linker settings.
MSVC (the Visual Studio compiler/linker) can do this if you compile with /Gy and link with /OPT:REF.
GCC/binutils can do this if you compile with -ffunction-sections -fdata-sections and link with --gc-sections.
Don't know about other compilers.
As a general rule, the answer is:
Yes: for unused static functions.
No: for unused globally available functions.
The compiler doesn't know if some other compilation unit references it. Also, most object module types do not allow functions to be removed after compilation and also do not provide a way for the linker to tell if there exist internal references. (The linker can tell if there are external ones.) Some linkers can do it but many things work against this.
Of course, a function in its own module won't be loaded unnecessarily by any linker, unless it is part of a shared library. (Because it might be referenced in the future at runtime, obviously.)
Many compilers do, but it depends on the particular implementation. Debug builds will often include all functions, to allow them to be invoked or examined from within the debugger. Many embedded systems compilers, for reasons I don't totally understand(*), will include all of the functions in an object file if they include any, but will omit entirely any object files that aren't used at all.
Note that in languages which support reflection (e.g., Java, C#, VB.NET, etc.) it's possible, given the name of a function, to create a reference to it at runtime even if no references exist in the code. For example, a routine could accept a string from the console, munge it in some fashion, and generate a call to a function by that name. There wouldn't be any way for a compiler or linker to know what names might be so generated, and thus no way to know what functions may be safely omitted from the code.
No such difficulty exists in C or C++, however, since there is no defined way for code to create a reference to a function, variable, or constant without an explicit reference existing in the code. Some implementations may arrange things so that consecutively-declared constants or variables will be stored consecutively, and one might thus create a reference to a later-declared one by adding an offset to an earlier-declared one, but the behavior of such tricks is explicitly not guaranteed by the C or C++ standards.
(*)I understand that it makes compiling and linking easier, but today's computers should have no trouble running more sophisticated compiling and linking algorithms than would have been practical in decades past. If nothing else, a two-pass pre-compile/pre-link/compile/link method could on the pre-compile/link phase produce a list of things that are used, and then on the "real" compile/link phase omit those that are not.
GCC, if you turn on the optimizations, can remove unused functions and dead code.
More on GCC optimizations can be found here.
Quite a lot of the time, yes. It’s often called linker stripping.
When it comes to Microsoft, it's the linker that takes care of this during the link phase and the compiler might warn you about unused static functions (file scope).
If you want the linker to remove unused functions, you use the /OPT:REF option.
Under MSVC and with global functions or variable you can use __declspec( selectany ).
It will remove the function or variable if it has not being referenced in the code if the linker option /OPT:REF (Optimizations) is selected.
As I understand function-level linking builds (explicitly or not) a graph of all possible calls and only includes the reachable functions' code into the produced binary. But how does it deal with variables declared at file level?
Say I have
MyClass GlobalVariable;
static MyClass StaticGlobalVariable;
in some file that contains only these two variables and a set of functions not actually called from any of the remaining code.
Will the code for these variables allocation/initialization be included into the output?
From experience (rather than quoting the standard):
If the initilaization has visible side effects like calls into external libraries or file I/O, the initialization will always happen.
boost::singleton_default provides an interesting solution that enforces the initialization to be done only when the object is referenced elsewhere, i.e. when all other references to the object are removed by the linker, the initialization is removed, too.
Edit: Yes. g++ optimize flags try to figure out function calls and prune away .o files resulting in linker errors. I'm not sure if this happens only with certain optimize flags, but it does happen.
A bad habit in our company is the presence of a lot of 'extern g_GlobalFunction()' definitions in different files. As their calls depended on conditional code, the .o files were often thrown away, resulting in link errors.
We fixed that with g_InitModule() and g_InitFileName() calls that are called hierarchically starting from main(). Mostly, these are empty functions just meant to dissuade g++ from discarding the .o file.