Internal exceptions in shared library terminate end user application - c++

I am building a shared library which uses Boost.thread internally. As a result, Boost.system is also used since Boost.thread depends on that. My shared library exports a C interface, so I want to hide all my internal exception handling and thread usage etc from the end user. It is supposed to be a black box so to speak. However, when I link with a client application, while the program runs fine - as soon as it is time to stop the processing by invoking a library function I get:
terminate called after throwing an instance of 'boost::thread_interrupted'
I catch this exception internally in the library, so I have no idea why it is not actually being caught. The end user's program is not meant to know about or handle Boost exceptions in any way. When building the shared library, I use static linking for both Boost.thread and Boost.system so the outside world is never meant to see them. I am on GCC 4.7 on Ubuntu 12. On Windows, I have no such problems (neither with MSVC or MinGw).
(EDIT)
I am editing the question to show a minimalistic example that reproduces the problem, as per the requests in the comments.
Here first is the code for testlib.cpp and testlib.h.
testlib.cpp:
#include <boost/thread/thread.hpp>
void thread_func()
{
while(1)
{
boost::this_thread::interruption_point();
}
}
void do_processing()
{
// Start a thread that will execute the function above.
boost::thread worker(thread_func);
// We assume the thread started properly for the purposes of this example.
// Now let's interrupt the thread.
worker.interrupt();
// And now let's wait for it to finish.
worker.join();
}
And now testlib.h:
#ifndef TESTLIB_H
#define TESTLIB_H
void do_processing();
#endif
I build this into a shared library with the following command:
g++ -static-libgcc -static -s -DNDEBUG -I /usr/boost_1_54_0 -L /usr/boost_1_54_0/stage/lib -Wall -shared -fPIC -o libtestlib.so testlib.cpp -lboost_thread -lboost_system -lpthread -O3
Then, I have the code for a trivial client program which looks as follows:
#include "testlib.h"
#include <cstdio>
int main()
{
do_processing();
printf("Execution completed properly.\n");
return 0;
}
I build the client as follows:
g++ -DNDEBUG -I /usr/boost_1_54_0 -L ./ -Wall -o client client.cpp -ltestlib -O3
When I run the client, I get:
terminate called after throwing an instance of 'boost::thread_interrupted'
Aborted (core dumped)
I am not explicitly catching the thread interruption exception, but according to the Boost documentation Boost.thread does that and terminates the given thread only. I tried explicitly catching the exception from within the thread_func function, but that made no difference.
(End OF EDIT)
(EDIT 2)
It is worth noting that even with -fexceptions turned on, the problem still persists. Also, I tried to throw and catch an exception that is defined in the same translation unit as the code that catches and throws it, with no improvement. In short, all exceptions appear to remain uncaught in the shared library even though I definitely have catch handlers for them. When I compile the client file and the testlib file as part of a single program, that is to say without making testlib into a shared library, everything works as expected.
(End OF EDIT 2)
Any tips?

I finally figured it out. The -static flag should never be specified when -shared is specified. My belief was that it merely told the linker to prefer static versions of libraries that it links, but instead it makes the generated dynamic library unsuitable for dynamic linking which is a bit ironic. But there it is. Removing -static solved all my problems, and I am able to link Boost statically just fine inside my dynamic library which handles exceptions perfectly.

Maybe this?
If you have a library L which throws E, then both L and the
application A MUST be linked against X, the library containing the
definition of E.
Try to link executable against boost, too.

A shared library that itself includes statically linked libraries is not such a good idea, and I don't think that this scenario is well supported in the GNU toolchain.
I think that your particular problem arises from the option -static-libgcc, but I've been unable to compile it in my machine with your options. Not that linking statically-dinamically to libpthread.so sounds as such a good idea either... What will happen if the main executable wants to create its own threads? Will it be compiled with -pthread? If it is, then you will link twice the thread functions; if it isn't, it will have the functions but not the precompiler macros nor the thread-safe library functions.
My advice is simply not to compile your library statically, that's just not the Linux way.
Actually that should not be a real problem, even if you don't want to rely on the distribution version of boost: compile your program against the shared boost libraries and deploy all these files (libboost_thread.so.1.54.0, libboost_system.so.1.54.0 and libtestlib.so) to the same directory. Then run your program with LD_LIBRARY_PATH=<path-to-so-files>. Since the client is not intended to use boost directly, it doesn't need the boost headers, nor link them in the compiler command. You still have your black box, but now it is formed by 3 *so files, instead of just 1.

Related

C++ threading reported as disabled when pthread is linked to shared lib, but not to main executable

This question is very similar to Using C++11 multithreading in shared library loaded by program without thread support
I have a shared library which uses OpenMP and a main program, which calls a function from that.
testlib.cpp
#include <memory>
void foo(std::shared_ptr<int> f)
{
#pragma omp parallel for
for (size_t g = 0; g < 100; g++) {
auto other = f;
}
}
tester.cpp
#include <memory>
void foo(std::shared_ptr<int> f);
int main() {
foo(std::make_shared<int>(4));
}
I compile and link with:
g++ -fPIC -g -shared -o libtestlibd.so testlib.cpp -openmp -pthread
g++ -g tester.cpp -o tester -Wl,-rpath,`pwd` libtestlibd.so
However when running this on a machine with a custom (automatically) build compiler, this crashes with "pure virtual method called", "SIGSEV" and similar which I traced to __gthread_active_p returning false even though it should not, after all the program is (transitively) linked to libpthread
With gdb I set breakpoints inside the __gthread_active_p function with condition to trigger when the contained pointer is NULL or not, i.e. the function would return true or false.
I then compared the backtrace and the address of the variable:
In both cases the call to __gthread_active_p originates from foo, so from the shared library
The address of __gthread_active_ptr varies and is either set or not
However the situation where __gthread_active_p is, according to the addresses of the functions, when the shared_ptr destructor from the main program is called, i.e. I guess the dynamic linker found 2 definitions of it and chose to use the one from the main program
When doing the same on my Linux Mint 20 system the program works and debugging it shows that the destructor from the shared lib is used.
It also works, if I link the main program explicitly with -lpthread, although I already link it with the shared lib, which links pthread.
The __gthread_active_p function and the referenced variables are all static, i.e. local to the translation unit. Hence only my 2 cpp files matter.
So my questions are:
Why is libpthread not loaded before the static variables of the main program are initialized?
The function even contains a static void *const which to my understanding should only be initialized on the first call to the function. This doesn't seem to be true here, the debugger shows the first call from inside the shared lib, so after pthread is loaded. Why?
Is this a bug in libstdc++? This sounds like a massive footgun that you need to know, if any of your shared libs uses threading and if so link to pthread too.
What could I check to find out why it happens in one environment but not the other, given the GCC and libstdc++ version are the same?

C++ force unloading shared library

I'm trying to create an application which reloads a shared library multiple times. But at some point in time, dlmopen fails with error
/usr/lib/libc.so.6: cannot allocate memory in static TLS block
Here is the minimal code reproducing this issue:
#include <dlfcn.h>
#include <cstdio>
#include <vector>
int main() {
for (int i = 0; i < 100; ++i) {
void *lib_so = dlmopen(LM_ID_NEWLM, "lib.so", RTLD_LAZY | RTLD_LOCAL);
if (lib_so == NULL) {
printf("Iteration %i loading failed: %s\n", i, dlerror());
return 1;
}
dlclose(lib_so);
}
return 0;
}
And empty lib.cpp, compiled with
g++ -rdynamic -ldl -Wl,-R . -o test main.cpp
g++ -fPIC -shared lib.cpp -o lib.so
Update
It seems that it crashes even with one thread. The question is: how can I force a library unload or a destruction of unused namespaces created with LM_ID_NEWLM?
There is a built-in limit to the number of link map namespaces available to a process. This is rather poorly documented in the comment:
The glibc implementation supports a maximum of 16 namespaces
in the man page.
Once you create a link map namespace, there is no support for 'erasing' it via any APIs. This is just the way it's designed, and there's no real way to get around that without editing the glibc source and adding some hooks.
Using namespaces for reloading of a library is not actually reloading the library - you're simply loading a new copy of the library. This is one of the use cases of the namespaces - if you tried to dlopen the same library multiple times, you would get the same handle to the same library; however if you load the second instance in a different namespace, you won't get the same handle. If you want to accomplish reloading, you need to unload the library using dlclose, which will unload the library once the last remaining reference to the library has been released.
If you want to attempt to 'force unload' a library, then you could try issuing multiple dlclose calls until it unloads; however if you don't know what the library has done (e.g. spawned threads) there may be no way of preventing a crash in that case.
Older glibc versions might have some bugs related to this:
https://bugzilla.redhat.com/show_bug.cgi?id=89692
https://sourceware.org/bugzilla/show_bug.cgi?id=14898
What version are you using? Try using a newer glibc version, your code works pretty fine on my computer (glibc 2.23).

C++ runtime errors in CodeBlocks when printing strings with cout <<

I recently started using CodeBlocks and began encountering odd runtime errors which I have traced back to printing strings using cout <<. For example, even the following..
#include <string>
#include <iostream>
int main()
{
std::string str;
str = "Hi!";
std::cout << str << std::endl;
return 0;
}
results in an error. It will compile fine (using Borland) but when I run it I get a pop up window saying 'test.exe has stopped working' and in the console I get the message:
Process returned -1073741819 (0xC0000005) execution time : 1.526 s
Press any key to continue.
It compiles and runs fine in MS Visual C++ and with G++ in Ubuntu.. any thoughts would be greatly appreciated!
Cheers,
Weatherwax
My one-off comment ended up helping solve the problem so here it is packaged up as an answer for future users:
This guy had a similar issue and it ended up being a linker issue which he
fixed. The fix is the last post in the thread, although reading the
whole thread could be useful for you.
Long Story short: Borland compiler is a bit dated and annoying to use. Ended up being a linker issue within borland. Better off using a different compiler like GCC/G++ or Visual Studio compiler.
This answer is here to elaborate on the root cause of the issue.
The reason for your crashing program is because the wrong runtime library is being linked. Specifically, your example is compiled as a single threaded object file(the default) but the linking step is using the multithreaded cw32mt.lib runtime -- the "mt" suffix at the end means multithreaded.
The solution is to make sure the runtime your program is compiled to use matches with the runtime you're linking against. A few ways to do this.
Important bcc32 compile switches:
-tW Windows GUI program. WinMain() is expected
-tWC Windows Console program. main() is expected. default.
-tWR Use dynamically linked runtime. Absence implies static runtime linkage.
-tWM Use multithreaded runtime. Absence implies single thread.
Compiling your example program as single threaded like this works:
bcc32 -oexample.obj -c example.cpp
ilink32 -ap example.obj c0x32, example.exe,, cw32.lib import32.lib,,
or you can compile it as multithreaded like this(note the -tWM switch matching up with cw32mt.lib):
bcc32 -tWM -oexample.obj -c example.cpp
ilink32 -ap example.obj c0x32, example.exe,, cw32mt.lib import32.lib,,
A third approach that is easier and less error prone is to not call the linker yourself. Instead, let the compiler drive the linker indirectly(similar to gcc):
bcc32 -tWM -oexample.obj -c example.cpp
bcc32 -tWM example.obj -eexample.exe
For your simple example, it can even be shortened to:
bcc32 -eexample.exe example.cpp
Lastly, you can pass the -tW switch multiple times. For example, this command compiles your example as a console program with multithread support and dynamic runtime linkage:
bcc32 -tWM -tWR -tWC -eexample.exe example.cpp
The produced example.exe executable is much smaller and its import table has an entry for CC3250MT.DLL confirming that the borland runtime is dynamically linked.
We should not assume that a non-functioning program is caused by nonconformity to the standard or a bug in the tool we're using without first investigating user error as a potential cause (even though in this case it's tempting to do so). In the OP's case, the code::block IDE didn't have the right commands setup for the toolchain being used.

in gcc how to force symbol resolution at runtime

My first post on this site with huge hope::
I am trying to understand static linking,dynamic linking,shared libraries,static libraries etc, with gcc. Everytime I try to delve into this topic, I have something which I don't quite understand.
Some hands-on work:
bash$ cat main.c
#include "printhello.h"
#include "printbye.h"
void main()
{
PrintHello();
PrintBye();
}
bash$ cat printhello.h
void PrintHello();
bash$ cat printbye.h
void PrintBye();
bash$ cat printbye.c
#include <stdio.h>
void PrintBye()
{
printf("Bye bye\n");
}
bash$ cat printhello.c
#include <stdio.h>
void PrintHello()
{
printf("Hello World\n");
}
gcc -Wall -fPIC -c *.c -I.
gcc -shared -Wl,-soname,libcgreet.so.1 -o libcgreet.so.1.0 *.o
ln -sf libcgreet.so.1.0 libcgreet.so
ln -sf libcgreet.so.1.0 libcgreet.so.1
So I have created a shared library.
Now I want to link this shared library with my main program to create an executable.
gcc -Wall -L. main.c -lcgreet -o greet
It very well works and if I set the LD_LIBRARY_PATH before running greet( or link it with rpath option) I can make it work.
My question is however different:
Since I am anyway using shared library, is it not possible to force symbol resolution at runtime (not sure about the terminology but perhaps called dynamic linking as per the book "Linkers and Loaders"). I understand that we may not want to do it because this makes the program run slow and has overhead everytime we want to run the program, but I am trying to understand this to clear my concepts.
Does gcc linker provide any option to delay symbol resolution at runtime? (to do it with the library we are actually going to run the program with)(as library available at compile time may be different than the one available at runtime if any changes in the library)
I want to be able to do sth like:
bash$ gcc main.c -I.
(what option needed here?)
so that I don't have to give the library name, and just tell it that I want to do symbol resolution at runtime, so headers are good enough for now, actual library names are not needed.
Thanks,
Learner For Ever.
Any linker (gcc, ld or any other) only resolves links at compile-time. That is because the ELF standard (as most others) do not define 'run-time' linkage as you describe. They either link statically (i.e. lib.a) or at start-up time (lib.so, which must be present when the ELF is loaded). However, if you use a dynamic link, the linker will only put in the ELF the name of the file and the symbols it must find, it does not link the file directly. So, if you want to upgrade the lib to a newer version later, you can do so, as long as system can find the same filename (the path can actually be different) and the same symbol names.
The other option, to get symbols at run-time, is to use dlopen, which has nothing to do with gcc or ld. dlopen simply put, opens a dynamic link library, just like fopen might, and returns you a handle, which then you pass to dlsym with the name of the symbol you want, which might be a function name for example. dlsym will then pass you a pointer to that symbol, which you can then use to call the function or use as a variable. This is how plugins are implemented.
I think you are looking for ld option '--unresolved-symbols=ignore-all', yes it can actually do it (ignore prev answer). Imagine scenario where a shared library loaded late (when program is already running), it can use all symbols that are already resolved/loaded by the main process, no need to bother to do it again . btw it does not nervelessly makes it slow , at least on Linux

Boost program will not working on Linux

I have this program which uses Boost::Asio for sockets. I pretty much altered some code from the Boost examples. The program compiles and runs just like it should on Windows in VS. However, when I compile the program on Linux and run it, I get a Segmentation fault.
I posted the code here
The command I use to compile it is this:
c++ -I/appl/htopopt/Linux_x86_64/NTL-5.4.2/include
-I/appl/htopopt/Linux_x86_64/boost_1_43_0/include
mpqs.cpp mpqs_polynomial.cpp mpqs_host.cpp -o mpqs_host
-L/appl/htopopt/Linux_x86_64/NTL-5.4.2/lib -lntl
-L/appl/htopopt/Linux_x86_64/gmp-4.2.1/lib -lgmp -lm
-L/appl/htopopt/Linux_x86_64/boost_1_43_0/lib -lboost_system
-lboost_thread -static -lpthread
By commenting out code, I have found out that I get the Segmentation fault due to the following line:
boost::asio::io_service io_service;
Can anyone provide any assistance, as to what may be the problem (and the solution)?
Thanks!
Edit: I tried changing the program to a minimal example, using no other libraries or headers, just boost/asio.hpp:
#define DEBUG 0
#include <boost/asio.hpp>
int main(int argc, char* argv[]) {
boost::asio::io_service io_service;
return 0;
}
I also removed other library inclusions and linking on compilation, however this minimal example still gives me a segmentation fault.
From the GCC online documentation of the -static option:
On systems that support dynamic linking, this prevents linking with the shared libraries.
Boost can support static-only linkage but only if it was configured that way when the OS Package maintainer built it. Are you absolutely certain you should be using this flag? If not, try recompiling without the flag and see if that doesn't take care of the problem.