Consider the following code (this is not pthread specific; other examples, such as those involving the realtime library, exhibit similar behavior):
#define _GNU_SOURCE
#include <pthread.h>
inline void foo() {
static cpu_set_t cpuset;
pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), &cpuset);
}
int main(int argc, char *argv[]) { }
This is a valid program in C and in C++. So I save the contents of this to testc.c and testcpp.cpp and try to build.
When I build in C++ I get no error. When I build in C I get an undefined reference error. Now, this error occurs in -O1 and in -O3. Is there anyway to instruct gcc to do the right thing (see that foo is unused and skip the requirement for a definition of pthread_setaffinity_np)?
EDIT: I thought it was obvious from context, but the error message is:
/tmp/ccgARGVJ.o: In function `foo':
testc.c:(.text+0x17): undefined reference to `pthread_setaffinity_np'
Note that since foo isn't being referenced in the main path, g++ correctly ignores the function entirely but gcc does not.
EDIT 2: Let me try this one more time. The function foo, and the subsequent call to pthread_setaffinity_np, is unused. The main function is empty. Just look at it! Somehow, g++ figured out that foo did not need to be included, and subsequently the build process did not trip up when we intentionally omitted -lpthread (and checking the exported symbols with nm confirms that neither foo nor reference to pthread_setaffinity_np were needed). The resultant output from gcc didn't pick up on that fact.
I am asking this question because the C++ and the C frontends seem to give different results on the same input. This doesn't seem to be an ld issue prima facie because I would expect both paths to give the same linking error, which is why I emphasized that it seems to be a compiler issue. If both C++ and C gave problems, then yes I would agree that its a linking issue.
Well, apparently your program contains an error: you declare and call function pthread_setaffinity_np, but you never define it. Apparently you forgot to supply the library that contains the definition. This is an error in both C and C++.
In other words, this is not a valid program in C and in C++. It violates the One Definition Rule of C++ (and whatever the similar rule is called in C).
The rest depends on whether the compiler will catch this error and issue a diagnostic message for it. While formally the compiler is supposed to catch it, in reality linking errors are not always caught by the compilation process (in extended sense of the term, i.e. including linking as well).
Whether they are caught or not might depend on many factors. In this particular case the factor that matters is apparently the difference between the properties of inline functions of C and C++ languages. (And yes, they are really different between C and C++). I would guess that in C++ mode the compiler decided that this inline function does not need the actual body, while in C mode it decided to generate the body anyway.
So, again, if this program, somehow successfully compiles in some circumstances, it is only because you got lucky. You seem to believe that a function that is not called is supposed to be "ignored entirely". Neither C nor C++ make such guarantees. Assuming that the definition of pthread_setaffinity_np is indeed missing, your program is invalid in both C and C++. For this reason, the compiler that refused to compile it is actually the one with the correct behavior.
Taking the above into account, you might want to ask yourself whether you really care about why you got different error reports in C and C++ modes. If you do, it will require some research into the internal mechanics of that specific implementation and won't have much to do with the languages themselves.
In C, the inline keyword does not affect the linkage of the function. Thus foo has external linkage, and cannot be optimized out because it might be called from another translation unit. If the compiler/assembler put functions in their own individual sections and the linker is able to discard unneeded function sections at link time, it might be able to avoid a linking error, but to be correct, since this program references pthread_setaffinity_np, it must contain a definition for that function somewhere anyway, i.e. you must use -lpthread or equivalent.
In C++, inline functions have internal some weird pseudo-external linkage by default, so gcc optimized it out. See the comments for details.
In short, the lack of an error in certain configurations is a failure of gcc to diagnose an invalid program. It's not the behavior you should expect.
The other lesson you should take away from this is that C and C++ are nowhere near the same thing. Choose which one you're writing and stick to it! Don't try to write code that's "interchangeable" between the two or you're likely to make it subtly incorrect in both...
inline is only a suggestion, not something a compiler is obligated to listen to, so it can't assume that foo is not used in another compilation unit.
But, yeah, it would be nice to know exactly which is the undefined reference, given that you didn't post the error, and odd that it's shows up in C and not C++ compilation.
foo might not be used in your source code, but it's almost certainly referenced elsewhere in the build process and consequently it needs to be compiled.
Especially since a lot of optimization occur in the linking process, because the linker can determine that a function is "dead" and can be discarded.
If, internally, the linker decides to assemble the entire program as one pass, and then optimization in another, I would expect you to see this error (how can it assemble the whole program?)
Further, if the function is to be exported then it most certainly has to be compiled, linked, and end up in the output.
It sounds like you're relying on compiler/linker specific behavior.
Related
In my code (either C or C++; let's say it's C++) I have a one-liner inline function foo() which gets called from many places in the code. I'm using a profiling tool which gathers statistics by line in the object code, which it translates into statistics by using the source-code-line information (which we get with -g in clang or GCC). Thus the profiler can't distinguish between calls to foo() from different places.
I would like the stats to be counted separately for the different places foo() get called. For this to happen, I need the compiler to "fully" inline foo() - including forgetting about it when it comes to the source location information.
Now, I know I can achieve this by using a macro - that way, there is no function, and the code is just pasted where I use it. But that wont work for operators, for example; and it may be a problem with templates. So, can I tell the compiler to do what I described?
Notes:
Compiler-specific answers are relevant; I'm mainly interested in GCC and clang.
I'm not compiling a debug build, i.e. optimizations are turned on.
So I went through this video - https://youtu.be/e4ax90XmUBc
Now, my doubt is that if C++ is compiled language, that is, it goes through the entire code and translates it, then if I do something like
void main() {
int a;
cout<<"This is a number = "<<a; //This will give an error (Why?)
a = 10;
}
Now, answer for this would be that I have not defined the value for a, which I learned in school. But if a compiler goes through the entire code and then translates it then I think it shouldn't give any error.
But by giving an error like this, it looks to me as if C++ is a interpreted language.
Can anyone put some light on this and help me solve my dilemma here?
Technically, the C++ standard doesn't mandate that the compiler has to compile C++ into machine code. As an example LLVM Clang first compiles it to IR (Intermediate Representation) and only then to machine code.
Similarly, a compiler could embed a copy of itself in a program that it compiles and then, when the program is executed compile the program, immediately invoke it and delete the executable afterwards which in practice would be very similar to the program being interpreted. In practice, all widely used C++ compilers parse and assemble programs beforehand.
Regarding your example, the statement "This will give an error" is a bit ambiguous. I'm not sure if you're saying that you're getting a compile-time error or a runtime error. As such, I will discuss both possibilities.
If you're getting a compile time error, then your compiler has noticed that your program has undefined behaviour. This is something that you always want to avoid (in some cases, such as when your application operates outside the scope of the C++ Standard, such as when interfacing with certain hardware, UB occurs by definition, as certain behaviour is not defined by the Standard). This is a simple form of static analysis. The Standard doesn't mandate the your compiler informs you of this error and it would usually be a runtime error, but your compiler informed you anyway because it noticed that you probably made a mistake. For example on g++ such behaviour could be achieved by using the -Wall -Werror flags.
In the case of the error being a runtime error then you're most likely seeing a message like "Memory Access Violation" (on Windows) or "Signal 11" (on Linux). This is due to the fact that your program accessed uninitialized memory which is Undefined Behaviour.
In practice, you wouldn't most likely get any error at all at runtime. Unless the compiler has embedded dynamic checks in your program, it would just silently print a (seemingly) random value and continue. The value comes from uninitialized memory.
Side note: main returns int rather than void. Also using namespace std; considered harmful.
When linking to pthreads I get unexpected behavior during program execution, the ncurses interface becomes unresponsive, but the program does not crash. This is without using any pthread functionality, no threading headers or anything related to threads, just linking.
Are there any reasons that linking to a library can change program behavior without issuing warnings?
Pertinent information:
ncurses is also linked to the executable.
Everything is in a namespace.
Tried on current gcc and clang.
No compiler/linker errors or warnings using -Wall.
Arch Linux.
Linking ncurses and pthread together works fine on a smaller test program I made. So what I am looking for are more places to look, it's a library project and the codebase is a good size, it is all plain C++14 except for a small component that uses the ncurses library.
If you didn't changed anything in your code, didn't even include headers, but behavior changed, then your project contains UB. Possible UBs defined in standard, that may cause this:
Defining ANY function or global variables that match by name ones contained in standard headers
Defining something in std namespace.
Usage old C standard headers mixed with new C++ headers
Usage of variables that were initialized with undetermined value.
Array bound breaches, that do not lean to page fault or other kind of catastrophic result, may cause program change behavior.
Illegal use of side-effects of operators, e.g.: a ^=b ^= a ^=b;
and so on.
I recommend to check warnings, not only errors. There are certain warnings that are actually signs of semantic error or UB. AT least for GCC. Many novice programmers just ignore warnings as non-essential. E.g. this kind sneaks in quite often:
const char a = 167;
if(a == 167) // compiler makes warning here, a ==-89 in this example
{
// code will never be executed or even compiled
}
The code compiles but doesn't execute in expected way.
I have developed a cross-platform library which makes fair use of type-punning in socket communications. This library is already being used in a number of projects, some of which I may not be aware of.
Using this library incorrectly can result in dangerously Undefined Behavior. I would like to ensure to the best of my ability that this library is being used properly.
Aside from documentation of course, under G++ the best way I'm aware of to do that is to use the -fstrict_aliasing and -Wstrict-aliasing options.
Is there a way under GCC to apply these options at a source file level?
In other words, I'd like to write something like the following:
MyFancyLib.h
#ifndef MY_FANCY_LIB_H
#define MY_FANCY_LIB_H
#pragma (something that pushes the current compiler options)
#pragma (something to set -fstrict_aliasing and -Wstrict-aliasing)
// ... my stuff ...
#pragma (something to pop the compiler options)
#endif
Is there a way?
I rather dislike nay-sayers. You can see an excellent post at this page: https://www.codingame.com/playgrounds/58302/using-pragma-for-compile-optimization
All the other answers clearly have nothing to do with the question so here is the actual documentation for GCC:
https://gcc.gnu.org/onlinedocs/gcc/Pragmas.html
Other compilers will have their own methods so you will need to look those up and create some macros to handle this.
Best of luck. Sorry that it took you 10 years to get any relevant answer.
Let's start with what I think is a false premise:
Using this library incorrectly can result in dangerously Undefined Behavior. I would like to ensure to the best of my ability that this library is being used properly.
If your library does type punning in a way that -fstrict-aliasing breaks, then it has undefined behavior according to the C++ standard regardless of what compiler flags are passed. The fact that the program seems to work on certain compilers when compiled with certain flags (in particular, -fno-strict-aliasing) does not change that.
Therefore, the best solution is to do what Florian said: change the code so it conforms to the C++ language specification. Until you do that, you're perpetually on thin ice.
"Yes, yes", you say, "but until then, what can I do to mitigate the problem?"
I recommend including a run-time check, used during library initialization, to detect the condition of having been compiled in a way that will cause it to misbehave. For example:
// Given two pointers to the *same* address, return 1 if the compiler
// is behaving as if -fstrict-aliasing is specified, and 0 if not.
//
// Based on https://blog.regehr.org/archives/959 .
static int sae_helper(int *h, long *k)
{
// Write a 1.
*h = 1;
// Overwrite it with all zeroes using a pointer with a different type.
// With naive semantics, '*h' is now 0. But when -fstrict-aliasing is
// enabled, the compiler will think 'h' and 'k' point to different
// memory locations ...
*k = 0;
// ... and therefore will optimize this read as 1.
return *h;
}
int strict_aliasing_enabled()
{
long k = 0;
// Undefined behavior! But we're only doing this because other
// code in the library also has undefined behavior, and we want
// to predict how that code will behave.
return sae_helper((int*)&k, &k);
}
(The above is C rather than C++ just to ease use in both languages.)
Now in your initialization routine, call strict_aliasing_enabled(), and if it returns 1, bail out immediately with an error message saying the library has been compiled incorrectly. This will help protect end users from misbehavior and alert the developers of the client programs that they need to fix their build.
I have tested this code with gcc-5.4.0 and clang-8.0.1. When -O2 is passed, strict_aliasing_enabled() returns 1. When -O2 -fno-strict-aliasing is passed, that function returns 0.
But let me emphasize again: my code has undefined behavior! There is (can be) no guarantee it will work. A standard-conforming C++ compiler could compile it into code that returns 0, crashes, or that initiates Global Thermonuclear War! Which is also true of the code you're presumably already using elsewhere in the library if you need -fno-strict-aliasing for it to behave as intended.
You can try the Diagnostic pragmas and change the level in error for your warnings. More details here:
http://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Pragmas.html
If your library is a header-only library, I think the only way to deal with this is to fix the strict aliasing violations. If the violations occur between types you define, you can use the usual tricks involving unions, or the may_alias type attribute. If your library uses the predefined sockaddr types, this could be difficult.
Just some opitmization considerations:
Does anyone know it for sure whether intel C++ compiler (such as ICC 13.0, and of cause, compiled with some optimzation options like /O3 etc) will automatically optimize out any unused/uncalled struct/class/functions/variables in the codes like examplefun() below:
//...defining examplefunc()....//
const int a=0;
if (a>0)
int b=examplefunc();
The compiler does not usually optimize out unused functions unless they are static and, therefore, only accessible within a specific module. However, the linker might dead strip the function if linking is done at the function level and not the module level.
You can check the assembly output, linker map, or use something like objdump to check if the function was included in the linked binary.
I don't think the question is correctly stated. While the question literally asks whether the compiler will optimize away a function that is not used, but that is something that only the linker can do.
So what can the compiler do? The compiler can optimize away dead code, so for example in your code above, and because a is known to be 0, the compiler can remove the if statement altogether. For most uses, that is good enough (whether a function makes it to the executable or not won't affect performance much, whether a branch is avoided or not will affect the performance of the function --in particular with branch mispredictions).
Additionally, if the compiler optimizes the branch above, there will be one less reference to the exampleFunc function in the program, and when the linker processes the generated binaries, if there is no reference to a function in the whole program, it can drop the symbol altogether. Note that this can only be done as part of the program linkage, for libraries, even if the function is not called now, a program linked with the library at a later time might use it.
So getting back to the original question, the compiler will optimize away the branch, and the linker might or not remove the function from the binary, but that should not matter as much.
Regarding the other constructs, for struct and class, the only thing that makes it to the binary are the member functions, and the same thing applies there: if you are linking the program and none of the functions is used, the linker can drop the symbols.