A shared object library required by my program is present in two paths under LD_LIBRARY_PATH, but only one of the two should be loaded by my program because of version compatibility issue. For example, if LD_LIBRARY_PATH is path1:path2 and the shared library libxxx.so is in both path1 and path2, will path1/libxxx.so be loaded or path2/libxxx.so be loaded when I run the program?
Also I would appreciate it if anyone has a better solution than reordering the paths in LD_LIBRARY_PATH. After searching on the internet I saw some solution using rpath but didn't quite understand how that works.
The directories get searched in the order they appear in LD_LIBRARY_PATH.
Keep in mind that the loader also looks elsewhere, not just LD_LIBRARY_PATH, which may or may not be a factor.
Since you asked for a "better solution", here are two:
Get this broken shared library built correctly
There is a well-established mechanism and convention for versioning shared libraries so that different versions of the same shared library get loaded. A program that needs a particular version of a shared library will load the one that it needs.
In fact, it's a safe bet that you already have a bunch of different versions of multiple shared libraries installed, due to applications in your Linux distribution that need different versions of the same shared library.
This is nothing more complicated than not using the same name for incompatible versions of the same shared library. There is a well-defined convention for naming actual shared library filenames, that work together with the linker in order to make this happen (the -soname link option, see your linker documentation for more information).
Don't use LD_LIBRARY_PATH
It is possible to link an executable and embed in the executable itself a pathname to search, first and foremost, for any shared libraries, either before or after LD_LIBRARY_PATH.
Remove all directories from LD_LIBRARY_PATH.
Use the -rpath, with/without --enable-new-dtags or --disable-new-dtags option when linking your executable. The correct set of options depends on your specific details, and specific versions of your linker. See your linker documentation for more information. You mentioned you tried to find information on this in Google, but all that Google will do is, perhaps, refer you to the same documentation that you already have: the manual pages for the linker. That's the best source for complete information on using this or any other linker option.
The best solution depends on your specific circumstances; whether you're building the executable, and/or shared libraries, how easy/hard it is to change whatever you're building; or whether you're not building anything at all, just black boxes that need to be executed.
Related
I am building an application using OpenGL. I have multiple OpenGL installed on a server.
I noticed that even after specifying link path for OpenGL libraries at runtime in Makefile, when running the application it still looks for library in different places, resulting error.
The correct openGL path is /usr/lib/nvidia-410/
yuqiong#sturfee-dnn:~/sturgRender/assets$ ls /usr/lib/nvidia-410/ | grep GL
libEGL_nvidia.so.0
libEGL_nvidia.so.410.129
libEGL.so
libEGL.so.1
libEGL.so.410.129
libGLdispatch.so.0
libGL.so
libGL.so.1
libGL.so.410.129
libGLX_indirect.so.0
libGLX_nvidia.so.0
libGLX_nvidia.so.410.129
libGLX.so
libGLX.so.0
libOpenGL.so
libOpenGL.so.0
However the LD_LIBRARY_PATH points to :
yuqiong#sturfee-dnn:~/sturgRender/assets$ echo $LD_LIBRARY_PATH
/usr/local/torch/lib:/usr/local/tensorrt/lib:/usr/local/caffe/lib/:/usr/local/lib;//usr/local/cuda/lib64:/home/yuqiong/TensorRT-7.0.0.11/lib
This will cause the application to result in eglDisplayError. However after changing LD_LIBRARY_PATH to /usr/lib/nvidia-410/, this error is gone.
I suspect this is because libEGL and libGLX and libOpenGL is dynamically loaded.
However, on another machine, I build the application using CMake, and even though LD_LIBRARY_PATH is empty the application still links the correct libraries.
Why do I need to specify LD_LIBRARY_PATH on one machine but not the other?
Is the information about where to load dynamic libraries stored in system variables like LD_LIBRARY_PATH, or in the application itself?
You need to understand what is rpath, what is library search path and the rules for searching libraries.
For rpath and library search path, please check this one:
What's the difference between -rpath and -L?
For the rules for searching libraries, please check this one:
https://unix.stackexchange.com/questions/22926/where-do-executables-look-for-shared-objects-at-runtime
The makefile based build apparently does not uses rpath therefore based on the search order, loader finds the libraries in some other folder and it causes the issue. The cmake based build system either uses rpath or library is installed in default folders that loader checks regardless of settings.
I do not want to repeat what is already explained in other answers. I am merely trying to direct you to correct path to read more and understand these settings, then it will be obvious why you experience the issue and how to solve it.
I have some C++ code in place, which internally uses multiple third party libraries. The code compiles fine but during execution it is failing to load some shared library(libintbasic.so). Given the condition, I can not install any library into the system, the only way out is to find which function is calling that library. How could I find that out who is calling that library (my code does not call that directly).
I can not install any library into the system,
That appears to be a bogus claim: clearly you can copy your binary onto the system. Installing an additional library into the same directory is not much harder.
How could I find that out who is calling that library
There are two cases to consider:
your binary or one of the libraries it is directly linked with links to libintbasic.so. Your binary will not run at all, or
your binary, or one of the libraries it is directly linked with calls dlopen("libintbasic.so",...) and fails when that dlopen fails.
Debugging the first case is often easiest by setting LD_DEBUG=files,libs. The dynamic loader will then tell you which libraries are being loaded, and why they are required.
Debugging the second case is easy with gdb: set a breakpoint on dlopen, and execute where and info shared commands every time the breakpoint is hit.
Remove the linking option -lintbasic, you will see all the functions that needs this library in the error messages. This is not clean but it should work fine.
You may use ldd utility recursively for find all dependencies between shared libraries.
The following link also may be useful: Does ldd also show dependencies of dependencies?
I am using OpenMP in my C++ code.
The libgomp.so.1 exists in my lib folder. I also added its path to the LD_LIBRARY_PATH
Still at run-time I get the error message: libgomp.so.1: cannot open shared object file
At Compile time I compile my code with -fopenmp option.
Any idea what can cause the problem?
Thanks
Use static linking for your program. In your case, that means using -fopenmp -static, and if necessary specifying the full paths to the relevant librt.a and libgomp.a libraries.
This solves your problem as static linking just packages all code necessary to run you program together with your binary. Therefore, your target system does not need to look up any dynamic libraries, it doesn't even matter if they are present on the target system.
Note that static linking is not a miracle cure. For your particular problem with the weird hardware emulator, it should be a good approach. In general, however, there are (at least) two downsides to static linking:
binary size. Imagine if you linked all your KDE programs statically, so you would essentially have hundreds of copies of all the KDE/QT libs on your system when you could have just one if you used shared libraries
update paths. Suppose that people find a security problem in a library x. With shared libraries, it's enough if you simply update the library once a patch is available. If all your applications were statically linked, you would have to wait for all of these developers to re-link and re-release their applications.
For example, I need to generate a shared library, libXXX.so.
What's the common way to generate it? Only use -fPIC -shared ?
Do I need to create a soft link to this library? I mean using ln -s.
What if I update the shared libray, ie. from libXXX.so.1.2.3 to libXXX.so.1.2.4, how my project knows that the shared library is updated? Do I need to re-compile the whole project again?
What's the common way to manage the shared library in Makefile?
I totally do not understand these issues. Please do me a favor :-)
The topic on which you have raised the question is vast. Most of your questions depend upon your environment and also whether or not to create a softlink depends on in which folder you keep your library and which processes may need to access it etc.
Please check this link & this link also which gives a detailed explanation on creating and using shared libraries.
Some short answers to your questions:
What's the common way to generate it? Only use -fPIC -shared ?
The options used depends on the compiler. fPIC helps in generating position independent code and shared makes the library a shared library. But, apart from, there are also options which can expose only those symbols which you decide to export from your library and options which create a strong link between the global symbols of your library (-symbolic) etc.
Do I need to create a soft link to this library? I mean using ln -s.
As I mentioned above, this depends on where you decide to place your library. The common mechanism is keep your library in some folder of your choice and create a link to it in commonly "known" or accessible folder like /usr/lib etc.
What if I update the shared libray, ie. from libXXX.so.1.2.3 to
libXXX.so.1.2.4, how my project knows that the shared library is
updated? Do I need to re-compile the whole project again?
One of the main advantages of shared library is that you can update latest versions without having to recompile the entire code. Again achieving this depends on the platform & compiler.
What's the common way to manage the shared library in Makefile?
There are umpteen links available in google and my link also provides some basic detail.
I am working on a project with SDL. I feel like a lot of my code could/should be reusable for any other projects I might want to take on.
I was going to start a library project, but I don't know how to go about linking to the SDL libraries.
Usually, I go to Project Settings and edited Mingw32 Linker to include the libraries (ming32, SDLmain, and SDL). But the linker does not show up in the settings.
So my questions:
Is there no linker settings because building a library is only a compiling operation?
And in general, is it possible to build a library on top of the existing libraries? That is, write my library using SDL functions and stucts. Or would I have to get the source code and rebuild entirely with my code included?
Also, advice on shared vs. static in this instance?
(Also, any advice for where to go about learning more about compilers and linkers and such. I got through data structures in school but no farther)
As an introduction, you have to distinguish very well static library from dynamic ones, they are completely different beasts... said that, to your questions:
Is there no linker settings because building a library is only a compiling operation?
I guess you are creating a static library in this case. A static library is simply the collection in one single object file of all the individual object files (i.e., the .o files produced by the compiler) that make up your source tree. No more, no less.
With a static library you don't need to specify which are the dependencies, since it is understood that it is when compiling the final executable that your library will be linked with all the other libraries that it depends upon. Therefore it is only at that time (final executable build) that any missing symbol will be detected, and all other libraries must be available.
A shared library (also dynamic library), is an executable file that embeds all the static libraries that it depends upon. It can also have external dependencies with other shared library, which would not be embedded.
And in general, is it possible to build a library on top of the existing libraries? That is, write my library using SDL functions and stucts. Or would I have to get the source code and rebuild entirely with my code included?
It is perfectly possible, both for static and dynamic libraries.
Also, advice on shared vs. static in this instance?
In this instance it is not possible to advice, because you don't specify enough information.
Look at this: When to use dynamic vs. static libraries, and this, to have a sort of guideline.
(Also, any advice for where to go about learning more about compilers and linkers and such. I got through data structures in school but no farther)
I think that the two links above give you plenty of information. If you want to further go into details, you could start from this wikipedia article and browse from there.
The library you are building can have external dependencies. That means that you can link it with SDL or any other external libraries that you like.
I think this page explains all your other questions: DLL Creation in MingW