I'm experimenting with shared libraries to build a modularized program.
There are two cpp files to compile:
Shared library, compile with
g++ -fPIC -shared module.cpp -o module.so
//module.cpp
#include <iostream>
File using the shared library, compile with
g++ src/main.cpp -ldl -o binary
or
g++ -DFIX src/main.cpp -ldl -o binary
//main.cpp
#include <dlfcn.h>
#ifdef FIX
# include <iostream>
#endif
int main()
{
void* h = dlopen("./module.so", RTLD_LAZY);
if ( h )
{
dlclose(h);
}
}
With FIX undefined, valgrind reports lot's of still reachable memory (5,373bytes), with FIX defined, no memory is leaked.
What's the problem with using iostream in shared libraries?
This problem occurs with g++-4.6, g++-4.7 and g++-4.8. g++-4.4 does not show this behaviour. I don't have other compilers to test with, sadly (and I don't want to switch to g++-4.4 because of this).
Update:
Compiling the shared library file with the additional flags -static-libstdc++ -static-libgcc reduces the number of leaked blocks, but not completely. -static-libgcc alone has no effect, -static-libstdc++ has some effect, but not as much as both.
1. Why or how does this code snippet "fix" the issue?
I'm not sure why without digging into the libstdc++ code, but I assume that the memory allocated by the iostreams library, which is kept allocated for the duration of the whole program, is reported as a problem by valgrind when it is allocated in a shared library, but not when allocated in the main program.
2. What equivalent, independent (from the standard library) code snippet provides the same bug fix?
Firstly, I don't know why you want something "independent from the standard library" when the still reachable memory is probably being allocated by the standard library. The fix is either don't use the standard library at all, anywhere, or use it differently.
Secondly, that "fix" is undefined behaviour, because you violate the One Definition Rule by redefining std::ios_base differently to the proper definition in the std lib.
The correct way to get the same behaviour is to #include <iostream> in your main.cpp file, including <iostream> defines a static std::ios_base::Init object. Alternatively, just #include <ios> and then define the static variable (but don't redefine the std::ios_base type) but that's basically what <iostream> does, so you might as well use that.
Related
I have this simple code:
max = (int) sqrt (number);
and in the header I have:
#include <math.h>
But application still says undefined reference to sqrt. Do you see any problem here? It looks like everything should be okay.
You may find that you have to link with the math libraries on whatever system you're using, something like:
gcc -o myprog myprog.c -L/path/to/libs -lm
^^^ - this bit here.
Including headers lets a compiler know about function declarations but it does not necessarily automatically link to the code required to perform that function.
Failing that, you'll need to show us your code, your compile command and the platform you're running on (operating system, compiler, etc).
The following code compiles and links fine:
#include <math.h>
int main (void) {
int max = sqrt (9);
return 0;
}
Just be aware that some compilation systems depend on the order in which libraries are given on the command line. By that, I mean they may process the libraries in sequence and only use them to satisfy unresolved symbols at that point in the sequence.
So, for example, given the commands:
gcc -o plugh plugh.o -lxyzzy
gcc -o plugh -lxyzzy plugh.o
and plugh.o requires something from the xyzzy library, the second may not work as you expect. At the point where you list the library, there are no unresolved symbols to satisfy.
And when the unresolved symbols from plugh.o do appear, it's too late.
I suppose you have imported math.h with #include <math.h>
So the only other reason I can see is a missing linking information. You must link your code with the -lm option.
If you're simply trying to compile one file with gcc, just add -lm to your command line, otherwise, give some informations about your building process.
Just adding the #include <math.h> in c source file and -lm in Makefile at the end will work for me.
gcc -pthread -o p3 p3.c -lm
Here are my observation, firstly you need to include the header math.h as sqrt() function declared in math.h header file. For e.g
#include <math.h>
secondly, if you read manual page of sqrt you will notice this line Link with -lm.
#include <math.h> /* header file you need to include */
double sqrt(double x); /* prototype of sqrt() function */
Link with -lm. /* Library linking instruction */
But application still says undefined reference to sqrt. Do you see any
problem here?
Compiler error is correct as you haven't linked your program with library lm & linker is unable to find reference of sqrt(), you need to link it explicitly. For e.g
gcc -Wall -Wextra -Werror -pedantic test.c -lm
I had the same issue, but I simply solved it by adding -lm after the command that runs my code.
Example.
gcc code.c -lm
I'm observing unexpected behaviour (at least I cant find explanation for it) with GCC flag -flto and jemalloc/tcmalloc. Once -flto is used and I link with above libraries malloc/calloc and friends are not replaced by je/tc malloc implementation, the glibc implementation is called. Once I remove -flto flag, everything works as expected. I tried to use -fno-builtin/-fno-builtin-* with -flto but still, it doesnt pick the je/tc malloc implementation.
How the -flto machinery works? Why the binary doesnt pick new implementation? How it even links with -fno-builtin when it should fail on unresolved external for, say, printf?
EDIT001:
GCC 7.3
Sample code
int main()
{
auto p = malloc(1024);
free(p);
return 0;
}
Compilation:
/usr/bin/c++ -O2 -g -DNDEBUG -flto -std=gnu++14 -o
CMakeFiles/flto.dir/main.cpp.o -c
/home/user/Development/CPPJunk/flto/main.cpp
Linkage:
/usr/bin/c++ -O2 -g -DNDEBUG -flto CMakeFiles/flto.dir/main.cpp.o
-o flto -L/home/user/Development/jemalloc -Wl,-rpath,/home/user/Development/jemalloc -ljemalloc
EDIT002:
More suitable sample code
#include <cstdlib>
int main()
{
auto p = malloc(1024);
if (p) {
free(p);
}
auto p1 = new int;
if (p1) {
delete p1;
}
auto p2 = new int[32];
if (p2) {
delete[] p2;
}
return 0;
}
First, your sample code is wrong. Read carefully the C11 standard n1570. When you want to use the standard malloc, you should #include <stdlib.h>.
In C++11 (read n3337) malloc is frowned upon and should not be used (prefer new). If you still want to use std::malloc in C++ you should #include <cstdlib> (which, in GCC, is internally including <stdlib.h>)
Then your sample code is almost C code (once you replace auto with void*), not C++. It could be optimized (once you include <stdlib.h>), even without -flto but with just -O3, according to the as-if rule, to an empty main. (I've even wrote a public report, bismon-chariot-doc.pdf, which has a section §1.4.2 explaining in several pages how that optimization happens).
To optimize around malloc and free, GCC uses some __attribute__(malloc) function attribute in the declaration (inside <stdlib.h>) of malloc.
How the -flto machinery works?
LTO is explained in GCC internals §25.
It works by using some internal (GIMPLE-like and/or SSA-like) representation of the code both at "compile" and at "link" time (actually, the linking step becomes another compilation with whole-program optimization, so your code gets "compiled" twice in practice).
LTO always should (in practice) be used with some optimization flag (e.g. -O2 or even -O3) both at compile and at link time. So you should compile and link with g++ -flto -O2 (it has no practical sense to use -flto without at least -O2 and the exact same optimization flags should be used at compile and at link time).
More precisely -flto also embeds in the object files some internal (GIMPLE-like) representation of the source code, and that is also used "at link time" (notably for optimization and inlining happening again when "linking" your entire program, re-using its GIMPLE). Actually GCC contains some LTO front-end and compiler called lto1 (in addition of the C++ front-end and compiler called cc1plus) and lto1 is (when you link with g++ -flto -O2) used at link time to reprocess these GIMPLE representations.
Probably, libjemalloc has its own headers, and might have inline (or inlinable) functions. Then you also need to use -flto -O2 when compiling that library from its source code (so that its Gimple is stored in the library)
At last, the fact that the usual malloc gets called is independent of -flto. It is a linker issue, not a compiler one. You could try to link -ljemalloc statically (and then you'll better build that library also with gcc -flto -O2; if you don't build it like that you won't get LTO optimizations across malloc calls).
You could pass also -v to your compilation and linking commands to understand what g++ is doing. You could even pass -Wl,--verbose to ask the ld (started by g++) to be verbose.
Notice that LTO (and the internal representations that it is using) is compiler and version specific. The internal (Gimple & SSA) representation is slightly different between GCC 7 & GCC 8 (and in Clang it is very different, so of course incompatible). The dynamic linker ld-linux(8) does not know about LTO.
PS. You could install the libjemalloc-dev package and add #include <jemalloc/jemalloc.h> in your code. See also jemalloc(3) man page. Probably libjemalloc could be configured or patched to define some je_malloc symbol as a replacement for malloc. Then it would be simpler (for LTO) to use je_malloc in your code (to avoid conflict between several malloc ELF symbols). To learn more about symbols in shared libraries, read Drepper's How to Write Shared Libraries paper. And of course you should expect LTO to change the behavior of linking!
Background:
I have application which part is used as library for other independent application. They link to that library (lets say lib.so) in linking time. Problem with such approach is that we have to use same external libraries like boost, ace etc or we will have duplicated symbols which in the end will cause crash. We want to resolve this problem.
I know two techniques - one is hiding all symbols (not sure about orders of scopes global/local for shared library) and other is to use dynamic linking. We chose 2nd options (dynamic linking) as it gave client opportunity to do easy testing with stubbed lib.so. and we have very simple api.
I wrote below small example of application which load example shared library and it crash (I want to understand why it crashed and how it should be written).
Crash is in dlopen, exactly in initialization of global variable on assignment to std::string (constructor of Aclass type). From our testing it looks that any access to std library while ongoing initialization of library will result in crash.
We managed to remove crash by adding -fPIC flag to EXECUTABLE (why this resolved our problem, I thought it should be set for shared library, may anyone explain me that more precisely)? Unnecessary to my understanding this flag is problematic as it slow down application and in my case (low latency applications) it is quite problematic.
To summary:
1. Why this crash occur?
2. Why -fPIC flag is enough to resolve this crash?
3. Why it is enough to set -fPIC flag to executable?
4. Is it possible to resolve my problem in other way so shared library and client application could use different versions of libraries (like boost, ace etc, compiler, linux version and std libraries are guarantee to be same)?
5. Removing flag RTLD_DEEPBIND will fix crash too but from gcc man it looks that I should use this flag as it will change symbol scope order for shared library - first it will search symbols in local scope then in global - looks as must have for me as shared library will use different external libraries than executable (and dynamic loading will protect executable with polluting its symbol scope). Why removing this flag fix crash in this simple case?
Shared library dynLib.cpp:
#include <string>
class Aclass
{
std::string s;
s = "123";
}
Aclass a;
Exacutable main.cpp:
#include <stdlib.h>
#include <dlfcn.h>
#include <string>
#include <unistd.h>
#include <iostream>
int main()
{
std::string dummyCrasher;
dlerror();
void* handle = dlopen("./libdynLib.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);
if(!handle)
{
std::cout << "handle is null" << dlerror();
}
usleep(1000 * 1000 * 10);
}
Makefile: makefile
CXXFLAGS=-m32 -march=x86-64 -Wl,v -g -O3 -Wformat -Werror=format -c
CLINKFLAGS=-Wl,-Bstatic -Wl,Bdynamic -ldl -m32 -march=x86-64
all: dynLib.so dynamiclinking
dynLib.so: dynLib.o
g++44 $(CLINKFLAGS) -shared -o libdynLib.so dynLib.o
dynLib.o: dynLib.cpp
g++44 $(CXXFLAGS) dynLib.cpp
dynamiclinking: main.o
g++44 $(CLINKFLAGS) -o dynamiclinking main.o -ldl
main.o: main.cpp
g++44 (CXXFLAGS) main.cpp
.PHONY: clean
clean:
rm dynLib.o main.o dynamiclinking libdynLib.so
PS. I write that code by hand (could did some spell errors)
PS 2. with -fPIC flag it will work:
main.o: main.cpp
g++44 (CXXFLAGS) main.cpp -fPIC
UPDATE
It is possible to resolve this problem by static linkage of libstdc++. But still my questions are not answered :( Maybe someone have some time to look at it?
UPDATE2 Same problem occur on GCC 4.4.6 and 4.8.1.
I think you are facing the same problem as in When we are supposed to use RTLD_DEEPBIND?, where the executable gets a copy of global variables:
well that's a wonderful feature of you building the main application without the -fPIC option.
[...]
This means that when the symbol is found in libdep.so, it gets copied into the initial data segment of the main executable at that address. Then the reference to duplicate in libdep.so is looked up and it points to the copy of the symbol that's in the main executable.
Due to RTLD_DEEPBIND, dynLib.so is seeing the wrong set of global variables from original libstdc++ when initializing std::string and thus crashes.
As for why the linker has such behavior, this article has a detailed explanation (emphasis mine):
Recall that the program/executable is not relocatable, and thus its data addresses have to bound at link time. Therefore, the linker has to create a copy of the variable in the program's address space, and the dynamic loader will use that as the relocation address. This is similar to the discussion in the previous section - in a sense, myglob in the main program overrides the one in the shared library, and according to the global symbol lookup rules, it's being used instead.
One final note: this behavior is platform-specific, at least on PowerPC there is no such additional copy of global variables in main executable.
I have wrapped my C++ header file in an extern "C" block to allow calling C++ functions from C. I have included the proper header files in the proper places. It works if I do something like the following:
g++ -c cpp_src.cc
gcc -c c_src.c
gcc -o c_exe c_src.o cpp_src.o -lstdc++
But if I remove the -lstdc++ from the last line, I get a bunch of errors. Because of reasons beyond my control, I cannot add the -lstdc++ flag. Is there a way to get the above to work without -lstdc++? What exactly is lstdc++, as in how does the gcc use it while linking with g++ compiled code?
Also, cpp_src.cc uses STL, if that makes a difference.
If you really need to have an object file that you can link with gcc without using -lstdc++, you can do an incremental link of your C++ object file with -lstdc++ and then use the result of that on your gcc link line. Something like:
ld -i -static cpp_src.o -L/usr/lib/gcc/x86_64-linux-gnu/4.8 -lstdc++ -o cpp_withstdlib.o
This will link your C++ object file with the standard C++ library and produce a new object file (cpp_withstdlib.o) that contains all the standard C++ library stuff you need, so can be linked on the gcc command line without needing -lstdc++
The tricky part is the -L option -- you need to figure out where g++ has stashed its standard library, as it generally doesn't put it in the standard /usr/lib location. The above is for an Ubuntu 14.04 machine. On most Linux machines, you can find it with locate libstdc++.a
You're generally just better off using g++ to link C++ code, as it knows all the right paths and won't make little mistakes that result in a binary that superficially appears to work, but is actually incorrect.
Using Josh's suggestion, you can do:
ld -i -static cpp_src.o `g++ -print-file-name=libstdc++.a` -o cpp_withstdlib.o
which is a bit cleaner, and could readily be done in a Makefile.
-lstdc++ causes the linker to link to libstdc++, which is gcc's implementation of the C++ standard library. If your code uses any symbols from the standard library (it uses the standard library, and not all code is inlined from headers), it must link libstdc++ (or whichever standard library implementation you use).
You can avoid the standard library, or you can link against the system standard library, or you can statically link a copy of the standard library into your program.
I know there are many questions related to shared libraries on Linux but maybe because I'm tired of having a hard day trying to create a simple dynamic library on Linux (on Windows it would have taken less than 10 minutes) I can't find what happens in this case.
So, I am trying to create a library to be linked at build-time and used at run-time (not a static library, not a library to be embedded into the executable, in other words). For now it contains a simple function. These are my files:
1.
// gugulibrary.cpp
// This is where my function is doing its job
#include "gugulibrary.h"
namespace GuGu {
void SayHello() {
puts("Hello!");
}
}
2.
// gugulibrary.h
// This is where I declare my shared functions
#include <stdio.h>
namespace Gugu {
void SayHello();
}
3.
// guguapp.cpp
// This is the executable using the library
#include "gugulibrary.h"
int main() {
GuGu::SayHello();
return 0;
}
This is how I try to build my project (and I think this is what is wrong):
gcc -Wall -s -O2 -fPIC -c gugulibrary.cpp -o gugulibrary.o
ld -shared -o bin/libGugu.so gugulibrary.o
gcc -Wall -s -O2 guguapp.cpp -o bin/GuGu -ldl
export LD_LIBRARY_PATH=bin
This is saved as a .sh file which I click and execute in a terminal. The error I get when trying to link the library is this:
/tmp/ccG05CQD.o: In function `main':
guguapp.cpp:(.text.startup+0x7): undefined reference to `SayHello'
collect2: ld returned 1 exit status
And this is where I am lost. I want the library to sit in the same folder as the executable for now and maybe I need some symbols/definitions file or something, which I don't know how to create.
Thanks for your help!
In your C++ file, GuGu::SayHello is declared as a C++ symbol. In your header, you are wrapping it in an extern "C" block. This is actually undefined, as you aren't allowed to use C++ syntax (namespace) in that context. But my guess is that what the compiler is doing is ignoring the namespace and generating a C symbol name of "SayHello". Obviously such a function was never defined by your library. Take out the extern "C" bits, because your API as defined cannot be used from C anyway.
You are inconsistent with your GuGu, there are also Gugu's running around, this needs to be made consistent, then it works (At least on my computer are some Gugu's now)