Easy check for unresolved symbols in shared libraries? - c++

I am writing a fairly large C++ shared-object library, and have run into a small issue that makes debugging a pain:
If I define a function/method in a header file, and forget to create a stub for it (during development), since I am building as a shared object library rather than an executable, no errors appear at compile-time telling me I have forgotten to implement that function. The only way I find out something is wrong is at runtime, when eventually an application linking against this library falls over with an 'undefined symbol' error.
I am looking for an easy way to check if I have all the symbols I need at compile time, perhaps something I can add to my Makefile.
One solution I did come up with is to run the compiled library through nm -C -U to get a demangled list of all undefined references. The problem is this also comes up with the list of all references that are in other libraries, such as GLibC, which of course will be linked against along with this library when the final application is put together. It would be possible to use the output of nm to grep through all my header files and see if any of the names corresponding.. but this seems insane. Surely this is not an uncommon issue and there is a better way of solving it?

Check out the linker option -z defs / --no-undefined. When creating a shared object, it will cause the link to fail if there are unresolved symbols.
If you are using gcc to invoke the linker, you'll use the compiler -Wl option to pass the option to the linker:
gcc -shared ... -Wl,-z,defs
As an example, consider the following file:
#include <stdio.h>
void forgot_to_define(FILE *fp);
void doit(const char *filename)
{
FILE *fp = fopen(filename, "r");
if (fp != NULL)
{
forgot_to_define(fp);
fclose(fp);
}
}
Now, if you build that into a shared object, it will succeed:
> gcc -shared -fPIC -o libsilly.so silly.c && echo succeeded || echo failed
succeeded
But if you add -z defs, the link will fail and tell you about your missing symbol:
> gcc -shared -fPIC -o libsilly.so silly.c -Wl,-z,defs && echo succeeded || echo failed
/tmp/cccIwwbn.o: In function `doit':
silly.c:(.text+0x2c): undefined reference to `forgot_to_define'
collect2: ld returned 1 exit status
failed

On Linux (which you appear to be using) ldd -r a.out should give you exactly the answer you are looking for.
UPDATE: a trivial way to create a.out against which to check:
echo "int main() { return 0; }" | g++ -xc++ - ./libMySharedLib.so
ldd -r ./a.out

What about a testsuite ? You create mock executables that link to the symbols you need. If the linking fails, it means that your library interface is incomplete.

I had the same problem once. I was developing a component model in C++, and, of course, components should load at runtime dynamically. Three solutions come to mind, that were the ones I applied:
Take some time to define a build system that is able to compile statically. You'll lose some time engineering it, but it will save you much time catching these annoying runtime errors.
Group your functions in well-known and well-understood sections, so that you can group of functions/stubs to be sure that each corresponding function has its stub. If you take the time on documenting it well, you can write perhaps a script that checks the definitions (via, for example, its doxygen comments) and check the corresponding .cpp file for it.
Do several test executables that load the same set of libraries and specify the RTLD_NOW flag to dlopen (if you're under *NIX). They will signal the missing symbols.
Hope that helps.

Related

Yet another linking issue with unresolved symbols

I am trying to build my program in OpenCL for ARM GPU - Mali.
I have a library libMali.so, which contains necessary symbols:
arm-v7a15v4r3-linux-gnueabi-nm *root_to_lib*/libMali.so
returns lines such as
002525b4 t clCreateKernel
and many others with all the expected OpenCL symbols.
However, compiling with
arm-v7a15v4r3-linux-gnueabi-g++ -c -Wall mandelbrot.cpp -o mandelbrot.o
arm-v7a15v4r3-linux-gnueabi-g++ mandelbrot.o -o mandelbrot -L*root_to_lib* -lMali
gives me errors like
mandelbrot.cpp:(.text+0x2e4): undefined reference toclCreateKernel'`
and others with all the symbols, which are actually present in libMali.so!
So, I kept the correct order of librabies in linking command, library is on the specified path (it is indeed) and it has the symbols.
Mangling is not the issue in this case as well: extern C specifiers were used in place and you can see that the raw entries of both lib and object file are not mangled.
Trying to accomplish the same thing using the arm-v7a15v4r3-linux-gnueabi-gcc didn't bring any change apart from necessity to link more c++ libs by hand (with -L*path* -llib).
libMali.so was built with arm-v7a15v4r3-linux-gnueabi-g++/gcc/ld, so this is not the matter of toolchain version.
I've ran out of ideas. May be someone here knows more tricky parts of linking process?
EDIT:
In fact, mandelbrot.cpp is a sample code from Mali-SDK. I'm just showing my linker problem on this example, since there obviously are no problems in the code. You can see the code here:
http://malideveloper.arm.com/downloads/deved/tutorial/SDK/opencl/mandelbrot_8cpp_source.html
http://malideveloper.arm.com/downloads/deved/tutorial/SDK/opencl/mandelbrot_8cl_source.html
If you look closely at the nm output:
002525b4 t clCreateKernel
you'll notice that the symbol is marked with a lowercase 't' which indicates that the symbol has a local binding (for example a static function) so it's not considered for binding to a undefined symbol in another object file. You can find an explanation of most of the cryptic "symbol type" letters used by nm here: https://sourceware.org/binutils/docs/binutils/nm.html
The readelf utility's output is more clear about symbols types.
Maybe the library was built incorrectly?

"undefined reference to" many (all?) of the functions in shared library (newbie)

I'm a novice at C++. Be patient if this is incoherent. I'm called upon to build a large system on linux that was originally built on OS X, where it works fine. The original authors are no longer with the company. The build system makes use of autotools, but there are also some hand made Makefiles which walk through the system calling the auto-made Makefiles. I've managed to get all of the c++ code compiled. The build system also uses libtools, and shared libraries are produced and deposited in /usr/local/lib.
So now I'd like to use these libraries. I've written a short program that simply instantiates an object of class ds_dictionary and calls one of its methods. Here it is:
#include <iostream>
#include <DSUtils/DSUtils.h>
int main (int argc, char * const argv[]) {
int32_t integer_data=123;
char key_alice_integer[] = "alice_integer";
ds_dictionary my_dict;
my_dict.add_int(key_alice_integer, integer_data);
return 0;
}
I compile this with
g++ -lDSUtils -o main my_test_code.cpp
With the result:
//usr/local/lib/libDSUtils.so: undefined reference to `ds_breakdown_from_time_interval'
//usr/local/lib/libDSUtils.so: undefined reference to `ds_date_breakdown_with_string'
//usr/local/lib/libDSUtils.so: undefined reference to `ds_seconds_duration_of_interval'
... (about 25 lines like these)
collect2: ld returned 1 exit status
Let's look inside the library:
garyp#VM:/usr/local/lib$ nm libDSUtils.so | grep ds_breakdown_from_time
U ds_breakdown_from_time_interval
The "U" in the line above ... does that mean that the library wasn't built correctly?
Am I calling g++ correctly?
Do I have to put something in the code to tell it that I'm using functions found in that library?
What are possible errors? Where should I start poking around?
EDIT:
Aha. The library DSUtils is built from several c++ sources. There is one c program in the source, and it contains all of the problem functions. The Makefile system doesn't deal at all with that one c file. That c program compiles. Ideally I suppose I'd figure out how to modify the Makefile to compile that file and add it to the library, but I'm not to the point where I can figure out how to do that.
Can I add the .o file to the existing library? How? Create a library with one file? etc?
EDIT_2: I simply did
g++ -o main -lDSUtils main.o my_new_objectfile.o
and the thing compiles, links, and runs without error. Should that work? After fixing a logic bug, it does work.
This
U ds_breakdown_from_time_interval
tells me that ds_breakdown_from_time_interval will be resolved by another library during runtime. So I am guessing you need to link to the library that defines ds_breakdown_from_time_interval.

in gcc how to force symbol resolution at runtime

My first post on this site with huge hope::
I am trying to understand static linking,dynamic linking,shared libraries,static libraries etc, with gcc. Everytime I try to delve into this topic, I have something which I don't quite understand.
Some hands-on work:
bash$ cat main.c
#include "printhello.h"
#include "printbye.h"
void main()
{
PrintHello();
PrintBye();
}
bash$ cat printhello.h
void PrintHello();
bash$ cat printbye.h
void PrintBye();
bash$ cat printbye.c
#include <stdio.h>
void PrintBye()
{
printf("Bye bye\n");
}
bash$ cat printhello.c
#include <stdio.h>
void PrintHello()
{
printf("Hello World\n");
}
gcc -Wall -fPIC -c *.c -I.
gcc -shared -Wl,-soname,libcgreet.so.1 -o libcgreet.so.1.0 *.o
ln -sf libcgreet.so.1.0 libcgreet.so
ln -sf libcgreet.so.1.0 libcgreet.so.1
So I have created a shared library.
Now I want to link this shared library with my main program to create an executable.
gcc -Wall -L. main.c -lcgreet -o greet
It very well works and if I set the LD_LIBRARY_PATH before running greet( or link it with rpath option) I can make it work.
My question is however different:
Since I am anyway using shared library, is it not possible to force symbol resolution at runtime (not sure about the terminology but perhaps called dynamic linking as per the book "Linkers and Loaders"). I understand that we may not want to do it because this makes the program run slow and has overhead everytime we want to run the program, but I am trying to understand this to clear my concepts.
Does gcc linker provide any option to delay symbol resolution at runtime? (to do it with the library we are actually going to run the program with)(as library available at compile time may be different than the one available at runtime if any changes in the library)
I want to be able to do sth like:
bash$ gcc main.c -I.
(what option needed here?)
so that I don't have to give the library name, and just tell it that I want to do symbol resolution at runtime, so headers are good enough for now, actual library names are not needed.
Thanks,
Learner For Ever.
Any linker (gcc, ld or any other) only resolves links at compile-time. That is because the ELF standard (as most others) do not define 'run-time' linkage as you describe. They either link statically (i.e. lib.a) or at start-up time (lib.so, which must be present when the ELF is loaded). However, if you use a dynamic link, the linker will only put in the ELF the name of the file and the symbols it must find, it does not link the file directly. So, if you want to upgrade the lib to a newer version later, you can do so, as long as system can find the same filename (the path can actually be different) and the same symbol names.
The other option, to get symbols at run-time, is to use dlopen, which has nothing to do with gcc or ld. dlopen simply put, opens a dynamic link library, just like fopen might, and returns you a handle, which then you pass to dlsym with the name of the symbol you want, which might be a function name for example. dlsym will then pass you a pointer to that symbol, which you can then use to call the function or use as a variable. This is how plugins are implemented.
I think you are looking for ld option '--unresolved-symbols=ignore-all', yes it can actually do it (ignore prev answer). Imagine scenario where a shared library loaded late (when program is already running), it can use all symbols that are already resolved/loaded by the main process, no need to bother to do it again . btw it does not nervelessly makes it slow , at least on Linux

Linking with .so files (webkit)

I'm trying to create a program that uses some of the code from WebKit/GTK+. Specifically, I want to load a string, use WebKit's parser to construct a DOM tree and then iterate over that tree.
I'm trying to use a class called HTMLDocument. WebKit/GTK+ doesn't expose this as part of its API and I'm running into some trouble linking against it.
I'm able to build WebKit/GTK+ normally, which gives me a file called: libwebkit-1.0.so. My program is:
#include <iostream>
#include <WebCore/config.h>
#include <WebCore/html/HTMLDocument.h>
using namespace WebCore;
int main() {
String title = "test";
RefPtr<HTMLDocument> d = HTMLDocument::create(0);
d->open();
d->write("<!doctype html><html><head><title>" + title + "</title></head><body></body></html>");
}
This compiles fine (I'm using the same include directives used by webkit to build), but results in linking errors.
...test_doc.cpp:18: undefined reference to `WebCore::String::String(char const*)'
...test_doc.cpp:21: undefined reference to WebCore::Document::open(WebCore::Document*)'
...(similar for every function I use)
If I run:
nm -C .libs/libwebkit-1.0.so | grep 'WebCore::Document::open'
I see:
003b1830 T WebCore::Document::open(WebCore::Document*)
which seems to indicate that the function is available. I have a reasonable amount of C++ experience, but not much experience with linking files under Linux.
I'm not expecting this exact problem to be solved, but I'm hoping someone can correct me if I have conceptual problems. My main question is why I see "undefined reference" errors when I'm linking with an .so file that lists that function as being defined. Is another file or build step needed?
Thank you very much.
Using:
Ubuntu 9.10
g++ 4.4.1
g++ is invoked with:
g++ --debug -DHAVE_CONFIG_H -I. `pkg-config --cflags libsoup-2.4` \
-DBUILDING_CAIRO__=1 -DBUILDING_GTK__=1 -DWTF_CHANGES -DWTF_USE_ICU_UNICODE=1 \
-DNDEBUG -I./WebCore -I./WebCore/accessibility -I./WebCore/bindings/js \
-I./WebCore/bridge -I./WebCore/bridge/c -I./WebCore/css -I./WebCore/dom \
...many more webkit include directories...
-DDATA_DIR=\"/usr/local/share\" \
test_doc.cpp -o test_doc.out \
./webkit-1.1.15.3/.libs/libwebkit-1.0.so
(I get the same result with -L/path/to/lib -lwebkit-1.0)
I think you might be running into an ordering problem: man g++ specifies that the order of the -l option is significant, and from memory the linker will only look for symbols in objects which have preceeded the current file on the command line.
I suspect what is happening is that the linker is trying to link test_doc before it's seen libwebkit-1.0.so, so it hasn't seen any of those symbols yet and bails.
You should use the -L/path/to/web and -lwebkit-1.0.
Also, I would compile your .cpp file in to a .o and then build your executable separately to make sure things are isolated.
Anyway, you may need to set your $LD_LIBRARY_PATH environment variable to include the path where that .so is stored. If you link to a shared library, you will need that library at run-time. Therefore, you do not want to have your webkit SO stored in its build directory (build/.libs). You want to install it. If you are not root, then you should ./configure with a --prefix=/some/path to install it to some local directory. Alternatively, you can link against the static library. One way to do this is to use the -bstatic (or similar) flag before your -lwebkit-1.0.
This is a good resource for Linux library creation and use.
I think you're issue is that the symbols you need are not exported. You can do objdump --dynamic-syms libwebkit-1.0.so to see which symbols are available. In the WebKit GTK build files there is use of the -fvisibility=hidden flag to restrict the symbols. Check your generated GNUMakefile and you'll see SYMBOL_VISIBILITY = -fvisibility=hidden. You should be able to modify the build files to get what you need.

linker error: undefined reference c++

I've tried looking at similar problems, but could not easily find one that helped my problem.
I've created a project in C++ and am working on UNIX to compile, link, and run it. My specific problem is an undefined reference to a method I declare in a separate file.
In the file SharedCache.cpp, I have the following method:
int SharedCache::replaceLine(ullong address){
int evictPID = -1;
int cacheSet = calcCacheSet( address );
//random uniformly-distributed value for cache line
float numLines = static_cast<float>(CACHE_LINES_PER_SET);
uint cacheLine = static_cast<uint>(uniformDistr( numLines ));
if(cache[cacheSet][cacheLine] != NULL){
evictPID = cache[cacheSet][cacheLine]->PID;
}
uint PID= calcPID(address);
uint tag= calcTag(address);
cache[cacheSet][cacheLine]->setLine(PID, tag); //mutex method
return PID;
}
The line uint cacheLine = static_cast<uint>( uniformDistr( numLines )); makes a call to the function I want to use from another file. The linker error is an undefined reference to this method.
uniformDistr( float ); is declared in the header distributions.h and defined in distributions.cpp. In my project options I set the linker flag -distributions and I also compiled the distributions.cpp to make sure a distributions.o file exists to link with.
From here, I don't know where to go, because this has not solved the problem.
Without more precise information on which compiler/linker commands were invoked and the exact error outputs, it is difficult to provide a good answer.
However, from your description of what you did, it seems that you are not passing distributions.o to the linker. Unlike other languages where the compiler/linker search for object files to link in automatically, C++ linkers require an explicit specification of the objects to link together.
Your use of -ldistributions here is incorrect: the -l flag is used to link to a static or dynamic library (respectively .a and .so files on Linux), whereas you want to specify another object file that the linker should consider. Specifying -ldistributions makes the linker look for distributions.a or distributions.so in the standard library locations.
Basically, your linker invocation now looks something like this (probably with many more flags and libraries):
gcc -o my_program SharedCache.o -ldistributions
To correctly link the distributions code in, you need to make it look something like (again, many flags and libraries probably missing compared to the real thing):
gcc -o my_program SharedCache.o distributions.o
This should resolve the missing symbols issue and produce a working binary (or at the very least a different error ;-) ). How to do this in KDevelop however I do not know.
I've not used KDevelop, however, on the command line you would just add distributions.o as an input file to the linking process. No need for dashes or leaving off the .o extension.
Alternatively, can you just add distributions.cpp to your KDevelop project? That way it should get compiled and linked automatically (this is how it works in things like Visual Studio or Eclipse).
Did you add the distributions.cpp to your makefile? Also I believe the required linker flag is -ldistributions.