c++ & OpenMP : undefined reference to GOMP_loop_dynamic_start - c++

I'm stuck in the following problem : at first I compile the following file cancme.cpp :
void funct()
{
int i,j,k,N;
double s;
#pragma omp parallel for default(none) schedule(dynamic,10) private(i,k,s) shared(j,N)
for(i=j+1;i<N;i++) {}
}
by:
mingw32-g++.exe -O3 -std=c++11 -mavx -fopenmp -c C:\pathtofile\cancme.cpp -o C:\pathtofile\cancme.o
Next I build a second file, test.cpp to simply link cancme.o with :
int main()
{
return(0);
}
by:
mingw32-g++.exe -O3 -std=c++11 -mavx -fopenmp -c C:\pathtofile\test.cpp -o C:\pathtofile\test.o
When linking it with cancme.o, by :
mingw32-g++.exe -o C:\pathtofile\test.exe C:\pathtofile\test.o -lgomp C:\pathtofile\cancme.o
I get the following error messages :
C:\pathtofile\cancme.o:cancme.cpp:(.text+0x39): undefined reference to `GOMP_loop_dynamic_start'
C:\pathtofile\cancme.o:cancme.cpp:(.text+0x49): undefined reference to `GOMP_loop_dynamic_next'
C:\pathtofile\cancme.o:cancme.cpp:(.text+0x52): undefined reference to `GOMP_loop_end_nowait'
C:\pathtofile\cancme.o:cancme.cpp:(.text+0x92): undefined reference to `GOMP_parallel_start'
C:\pathtofile\cancme.o:cancme.cpp:(.text+0x9f): undefined reference to `GOMP_parallel_end'
Does anyone have an idea about what's going wrong there??? The OpenMP library is correctly linked by the -lgomp flag but it is like it was not recognized.
note : I use MingW 4.8.1 c++ compiler under windows 7 64 bit:
thank you
renato

The GNU linker is a single pass linker. It means that it only resolves symbols that it has seen before it has reached the object file that defines the corresponding symbol. That means, if object file foo.o refers symbols from library libbar.a, then having ... foo.o -lbar ... will result in successful link since the undefined references seen in foo.o are satisfied during the processing of libbar.a (as long as no other object listed after -lbar refers symbols from the library). The opposite, i.e. ... -lbar foo.o ... won't work since once the linker has processed libbar.a, it will no longer search it while trying to resolve the references in foo.o.
On Unix systems that support dynamic link libraries (e.g. Linux, FreeBSD, Solaris, etc.), this is often not the case since -lbar will first look for the dynamic version of the library, e.g. libbar.so, and only if not found would try to link against the static libbar.a. When linking with dynamic libraries, the order doesn't matter since the unresolved references are handled later by the runtime link editor.
On Windows, linking against dynamic link libraries (DLLs) require the so-called import libraries to be statically linked into the executable. Therefore, even if the external dependencies are handled by the runtime linker, one still needs to properly order the static import libraries. libgomp.a is one such library.
Note that this behaviour is specific to the GNU linker that is part of MinGW. The Microsoft linker behaves differently: when resolving a reference, it first searches the libraries listed after the object file and then the libraries listed before it.

You made a mistake in compile command:
mingw32-g++.exe -o C:\pathtofile\test.exe C:\pathtofile\test.o -lgomp C:\pathtofile\cancme.o
If you'd use some makefile generators such as CMake for example you not get any linking error.
Don't mess compiler parameters like this: object_file_1 linker_flag_1 object_file_2.
Correct command to compile would be this:
mingw32-g++.exe -o C:\pathtofile\test.exe C:\pathtofile\test.o C:\pathtofile\cancme.o -lgomp

Related

why dependency error while creating shared library in c++ in macos but not in ubuntu

(Note: This question had been closed, citing that this had an answer. However, my question is not generic, I am asking why this works in ubuntu/redhat, but not in macos/cygwin. So I have edited this question, by modifying the title, mentioning the words macos and ubuntu.)
I have the following c++ code:
// main.cpp
#include<iostream>
#include<cstdio>
#include "defs.h" // has the function headers only
int func0(int a0) {
printf("func0-%d\n", a0);
return a0+1;
}
int func1(int a1) {
int x;
x=func0(a1);
printf("func1-%d\n", x);
return a1+1;
}
int func2(int a2) {
int x;
x=func1(a2);
printf("func2-%d\n", x);
return x+5;
}
int main() {
func1(5);
func2(8);
}
I can compile and run this code as:
g++ main.cpp; ./a.out
Now I would like to move the functions to different files (func1 to f1.cpp,
fun0 and func2 to f2.cpp, and main to main.cpp), and create shared libraries
like this:
g++ -c -pipe -std=c++11 -fPIC main.cpp
g++ -c -pipe -std=c++11 -fPIC f1.cpp
g++ -c -pipe -std=c++11 -fPIC f2.cpp
g++ -shared -o libx1.so f1.o
g++ -shared -o libx2.so f2.o
g++ main.o -L. -lx1 -lx2 -o exe
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
./exe
The above instructions work in redhat linux and ubuntu. But when I run the same commands in other variants of linux (eg macos or cygwin) I get errors during creation of the shared library like this:
g++ -shared -o libx1.so f1.o
undefined reference to func0(int)
g++ -shared -o libx2.so f2.o
undefined reference to func1(int)
Why is this error happening only in some OS versions, and not happening in redhat/ubuntu? Is it due to the gcc versions, or something to do with the OS?
(The above instructions work with g++ in redhat(gcc version 8.3.1) and ubuntu (9.4.0). It does not work with g++ in cygwin(11.3.0) and in macos(11.2.0).)
The problem is caused by cyclic dependencies between the two libraries. Before doing anything else, you should ask yourself whether it is acceptable to have cyclic dependencies for your project. It is never a good idea, but if the alternative involves massive refactoring, it could be the lesser of two evils. Still, refactoring should probably be the default answer in most cases. If you cannot refactor, the rest of this answer is for you.
How are cyclic dependencies handled on different OSes?
On both Linux and Mac OS X (and on FreeBSD and on most commercial Unixes of old), references are resolved at load time. The loader uses the first suitable symbol definition it encounters, be it it in the main executable, in the shared object itself, or in a different shared object. It is not known until load time where that definition will be found.
So when the executable from the question is loaded, the dynamic loader finds the definition of func1 in libx1 and the definitions of func0 and func2 in libx2, and all is well.
The difference between Linux and Mac OS X lies in the linker (ld) behaviour. Both GNU ld and LLVM ld by default allow unresolved references when building a shared library. Mac OS X ld appears to be of a different breed and unresolved references are not allowed by default. One can either list the dependencies on the link line, or explicitly allow unresolved references using the Mac-specific ld option -undefined dynamic_lookup. But of course when the dependencies are cyclic, the first option is problematic. For code in question:
g++ -shared -o libx1.so f1.o -Wl,-undefined,dynamic_lookup
g++ -shared -o libx2.so f2.o -Wl,-undefined,dynamic_lookup
Windows DLLs work very differently. Each symbol must be resolved at link time. Unlike the Unix-y loaders, the loader must know exactly which DLL to search for each imported symbol. There is no option to allow unresolved references in DLLs because there is no mechanism to resolve them at load time from an unknown source.
Windows still allows cyclic dependencies between DLLs, but the mechanism is a bit different. The linker must use separate import libraries in this case (they are usually optional when using GNU or LLVM toolchains). The linking is done in two phases. First, the .lib files are generated for each future .dll, and then .dll themselves are produced using the .lib files from the first stage. For code in question:
# first stage
g++ -shared -Wl,--out-implib=x1.lib -o x1.dll f1.o
g++ -shared -Wl,--out-implib=x2.lib -o x2.dll f2.o
# second stage
g++ -shared -o x1.dll f1.o x2.lib
g++ -shared -o x2.dll f2.o x1.lib
The first stage will report undefined symbols but will still produce the .lib file needed for the second stage.

Linker error undefined reference even if functions are defined in shared library

I´m trying to modify a piece of C++ code (developed under Linux with gcc toolchain) using in it new functions that are defined in two libraries, one shared library (called libsio4_api.so) and a static library (called sio4_main.a). The original code is built with a makefile that first creates a shared library linking toghether various object files and then links this library with the main object file.
This is the line that links the shared library:
g++ -s -shared main.o RTLinkResolution.o IORefresh.o UserProgramImpl.o UserProgramDataManager.o UserProgramCode.o -L./genericFiles/lib -lUserProgramEnvironment -lm -o libUProg.so
I modified it in this way to add to the linkage my libraries (-pthread is needed by one of them):
g++ -s -shared -pthread libsio4_api.so sio4_main.a main.o RTLinkResolution.o IORefresh.o UserProgramImpl.o UserProgramDataManager.o UserProgramCode.o -L./genericFiles/lib -lUserProgramEnvironment -lm -o libUProg.so
This linking terminate without errors and in the resulting lib I can see the functions I want to use (in this example I show just one but also the others are present):
nm -D libUProg.so | grep sio4_async_init
000000000006d581 T sio4_async_init
000000000006dcae T sio4_async_init_data
U _Z15sio4_async_initv
The problem is that when the final linking is done the functions are not found:
g++ -o test main.o libUProg.so -pthread sio4_main.a libsio4_api.so
libUProg.so: undefined reference to `sio4_async_close(int)'
libUProg.so: undefined reference to `sio4_async_init()'
libUProg.so: undefined reference to `sio4_async_open(int, int, int*)'
I searched already for similar problems in other topics and I found out that the order in which libraries are fed to the linker is important, but even if I change the order in the final linking command those functions are not found.
Does someone have any clue about how I can proceed?

Link only needed symbols when compiling an executable with a Shared Library

I'm working on a heavy project that has a lot of static libraries that are interdependent. Furthermore some symbols are redundant between some libraries, with different implementations. My goal is to make the project work with shared libraries.
I tried to compile an executable with one of my shared libs, and I get undefined symbols errors on functions that my executable isn't using. After some research I understood that the dynamic linker works in very different ways than the static linker. If I understood right, when linking a shared library, all symbols need to be resolved as the whole library is loaded in the memory.
A simple workaround would be to add all the dependencies of my libraries when compiling the executable. But they're so full of dependencies that this sometimes means adding 10+ libraries to the command line, and this would be for something like a hundred executable.
So far I tried using -Wl,--as-needed, -Wl,--unresolved-symbols=ignore-in-shared-libs, and opening the shared object with dlopen to get the function I want with dlsym. But all of these methods fail at one point or another.
My question is: Are you forced to resolve every undefined symbol of a dynamic library when linking it against an executable ?
Details of dynamic linking and the kinds of objects involved vary across environments and toolchains. On Linux, where you say you are, and on Solaris, and several other UNIX-y platforms, you are looking at ELF objects and semantics.
So far I tried using -Wl,--as-needed,
-Wl,--unresolved-symbols=ignore-in-shared-libs,
These both have their full effect at (static) link time. The first tells the linker that the libraries following it on the command line should be linked in only if they resolve at least one as-yet undefined symbol. The latter tells the linker to not worry about resolving symbols in shared libraries included in the link. That has nothing to do with the behavior of the dynamic linker when you run the program.
and opening the shared object with dlopen to get the function I want with dlsym.
dlopen instructs the dynamic linker to link in a shared object at runtime that was not specified in the binary as a required shared library. Its behavior at that point can be modulated by the flags passed to dlopen, but the options available are not more than can be specified at link time. There is little reason to use dlopen when you actually know at link time what libraries you need.
Are you forced to resolve every undefined symbol of a dynamic library
when linking it against an executable ?
Focusing on ELF and the GNU toolchain, no. -Wl,--unresolved-symbols=ignore-in-shared-libs serves precisely the purpose of avoiding that. But as you've discovered, that comes with caveats.
In the first place, in every shared object, every symbol referring to data needs to be resolved at runtime by the dynamic linker, no matter how you linked the various shared objects, including the main program. This is primarily an operational consideration -- the dynamic linker has no way to defer resolving symbols referring to objects because it has no good way to trap attempts to access them.
On the other hand, it is possible to defer resolution of symbols referring to functions until their first use. In fact, this is the GNU linker's default, but you can reaffirm this by passing -Wl,-z,lazy to gcc when linking. Note well, however, that this sets a property of the object being linked, so you should ensure that every shared object is built with that link option (but ordinarily they are because, again, that's the default).
Additionally, you should be aware that the dynamic linker's behavior can be influenced by environment variables. In particular, lazy binding will be disabled if the dynamic linker finds LD_BIND_NOW set to a nonempty string in the runtime environment.
A simple workaround would be to add all the dependencies of my
libraries when compiling the executable. But they're so full of
dependencies that this sometimes means adding 10+ libraries to the
command line, and this would be for something like a hundred
executable.
And what's the big deal with that, really? Surely you have a well-factored Makefile (or several) to help you, so it shouldn't be a big deal to ensure that all the libraries are linked. Right?
But you should also consider refactoring your libraries, especially if "interdependent" means there are loops in the dependency graph. Dynamic linking is different from static linking, as you've discovered, and the differences are sometimes more subtle than those you're presently struggling with. Although it is not a hard rule, I urge you to avoid creating situations where the shared objects used by one process contain among them multiple definitions of the same external symbol, especially if that symbol is actually used.
Update
The above discussion focuses on linking shared libraries to an executable, but there is another important consideration: how the libraries themselves are linked. Each ELF object, whether executable or shared library, carries its own list of needed shared libraries. The dynamic linker will recursively include all of these in the list of shared libraries to be loaded (immediately) at program startup, notwithstanding its behavior with respect to lazy binding of symbols referring to functions.
Therefore, if you want an executable not to require a given shared library X, then not only that executable itself but also every shared library it does rely upon must avoid expressing a dependency on X. If some of the shared libs require X when used in conjunction with other programs, then that puts the onus on you to link in all the needed libraries when building those programs (otherwise, you can arrange to link only direct dependencies). You can tell the GNU linker to build shared libraries this way by passing it the --allow-shlib-undefined flag.
Here is a complete proof of concept:
main.c
int mul(int, int);
int main(void) {
return mul(2, 3);
}
mul.c
int add(int, int);
int mul(int x, int y) {
return x * y;
}
int mul2(int x, int y) {
return add(x, y) * add(x, -y);
}
Makefile
CC = gcc
LD = gcc
CFLAGS = -g -O2 -fPIC -DPIC
LDFLAGS = -Wl,--unresolved-symbols=ignore-in-shared-libs
SHLIB_LDFLAGS = -shared -Wl,--allow-shlib-undefined
all: main
main: main.o libmul.so
$(LD) $(CFLAGS) $(LDFLAGS) -o $# $^
libmul.so: mul.o
$(LD) $(CFLAGS) $(SHLIB_LDFLAGS) -o $# $^
clean:
rm -f main main.o libmul.so mul.o
Demo
$ make
gcc -g -O2 -fPIC -DPIC -c -o main.o main.c
gcc -g -O2 -fPIC -DPIC -c -o mul.o mul.c
gcc -g -O2 -fPIC -DPIC -shared -Wl,--allow-shlib-undefined -o libmul.so mul.o
gcc -g -O2 -fPIC -DPIC -Wl,--unresolved-symbols=ignore-in-shared-libs -o main main.o libmul.so
$ LD_LIBRARY_PATH=$(pwd) ./main
$ echo $?
6
$
Note that the -zlazy linker option discussed in comments is omitted, as it's the default.

Why does the library linker flag sometimes have to go at the end using GCC?

I'm writing a small C program that uses librt. I'm quite surprised that the program won't compile if I place the link flag at the start instead of at the end:
At the moment, to compile the program I do:
gcc -o prog prog.c -lrt -std=gnu99
If I were to do the following, it will fail to find the functions in librt:
gcc -std=gnu99 -lrt -o prog prog.c
Yet, this works with other libraries. I found the issue when attempting to use a simple Makefile. make actually compiled prog.c without liking first (using -c flag) and then did the linking.
This is the Makefile:
CC = gcc
CFLAGS = -std=gnu99
LIBS= -lrt
LDFLAGS := -lrt
prog: prog.o
$(CC) -o prog prog.c -lrt -std=gnu99
The output I would get when typing make would be:
gcc -std=gnu99 -c -o prog.o prog.c
gcc -lrt prog.o -o prog
prog.o: In function `main':
prog.c:(.text+0xe6): undefined reference to `clock_gettime'
prog.c:(.text+0x2fc): undefined reference to `clock_gettime'
collect2: ld returned 1 exit status
make: *** [buff] Error 1
I have now crafted a Makefile that puts the linking at the end of the gcc line, however I'm puzzled why it doesn't work if the linking flag is at the start.
I would appreciate if anybody can explain this to me. Thanks.
As the linker processes each module (be it a library or a object file), it attempts to resolve each undefined symbol while potentially adding to its list of undefined symbols. When it gets to the end of the list of modules, it either has resolved all undefined symbols and is successful or it reports undefined symbols.
In your case, when it processed librt, it had no undefined symbols. Processing proc resulted in clock_gettime being an undefined symbol. gcc will not go back and look in librt for the undefined symbols.
For that reason, you should always have your code first, followed by your libraries, followed by platform provided libraries.
Hope this helps.
From the ld (the GNU linker) documentation (http://sourceware.org/binutils/docs/ld/Options.html#Options):
The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again.
So if you specify the library too early, the linker will scan it, but not find anything of interest. Then the linker moves on to the object file produced by the compiler and finds references that need to be resolved, but it has already scanned the library and won't bother looking there again.

Undefined symbol when trying to load a library with dlopen

I'm trying to load a shared library (plugin) I was provided (closed source) with dlopen under a Linux ARM platform. I'm trying to load this way:
void* handle = dlopen(<library_path>/<library_name>, RTLD_NOW);
The result is a failure with this message:
Failed to load <library_path>/<library_name>: undefined symbol: <symbol_name>.
I tried to look inside the library with nm, but it seems the lib was stripped, no symbol could be found. I also tried using readelf -s, and, in fact, I got this result:
12663: 00000000 0 NOTYPE GLOBAL DEFAULT UND <symbol_name>
By reading around, I get that readelf -s returns all the symbols, including those symbols defined in libraries referenced by it.
The answers to this question are not completely clear to me: is this a symbol which is supposed to be in the library and which is not there because it was compiled the wrong way or is this a symbol I'm supposed find somewhere else? The output of readelf -d seems to suggest I'm providing all the needed shared libraries. May this error be related to a mistake in the way I'm compiling my executable or is this something not related to the loader?
Also, I read about the meaning of each column, but those values are quite strange. How do you interpret that symbol description? Why is address 0? Why is type NOTYPE?
undefined symbol: X means always that X should be exported from one of loaded libraries, but it's not. You should find out in which library requested symbol is and link to it.
You should know that this message is always result of problem with library, it's not fault. Library should know how to get all it's symbols. If it doesn't you can link your executable to required library so when you load your plugin, requested symbol is already known.
This error might have more complex reason. In case when both plugin and main app are linking to library, then attempts to link it might end with undefined symbols anyway. This might happen if main app and plugin are using different version of library (namely plugin uses newer one). Then at the point of loading plugin older version is already loaded, so loader assumes everything is ok, but newer version might contain new symbols. If plugin uses them, you will get undefined symbol errors.
This problem appears also if the order of the static libraries in the linking command is wrong for the app. The Unix ld linker requires that the library which implements a function is specified after the library which refers the function.
I got this trouble when I was trying to build libtesseract shared library taking libz library from a custom location (not a standard libz from the host, but manually built from source as well). I have put an example below:
Wrong linking order (-lz before -llept):
$ g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginS.o -Wl,--whole-archive ....(some libs) -Wl,--no-whole-archive -L/home/build/jenkins/workspace/tesseract/zlib/bin/lib -L/home/build/jenkins/workspace/tesseract/leptonica/bin/lib -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/x86_64-linux-gnu -lz -llept -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/5/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o -g -O2 -Wl,-soname -Wl,libtesseract.so.4 -o .libs/libtesseract.so.4.0.1
Check with "nm -D":
$ nm -D .libs/libtesseract.so.4.0.1 | grep deflateInit
U deflateInit_
Check with "dlopen":
Cannot load ./tesseract/src/api/.libs/libtesseract.so.4.0.1 (./tesseract/src/api/.libs/libtesseract.so.4.0.1: undefined symbol: deflateInit_)
It happens because the linker is processing in the loop all static libraries passed in the command line and skipping those which are not used by any of the preceeding ones. Since on the moment of checking of libz.a the linker sees that all of already checked libraries do not use any function from libz.a the linker just "forgets" libz.a.
Proper linking order (-lz after -llept):
$ g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginS.o -Wl,--whole-archive ....(some libs) -Wl,--no-whole-archive -L/home/build/jenkins/workspace/tesseract/zlib/bin/lib -L/home/build/jenkins/workspace/tesseract/leptonica/bin/lib -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/x86_64-linux-gnu -llept -lz -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/5/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o -g -O2 -Wl,-soname -Wl,libtesseract.so.4 -o .libs/libtesseract.so.4.0.1
Check with "nm -D":
$ nm -D .libs/libtesseract.so.4.0.1 | grep deflateInit
000000000041fb5b T deflateInit_
000000000041fba3 T deflateInit2_
"dlopen" did not show this error this time.