How to avoid 'cannot open shared object file' when using CMake? - c++

Situation
My project uses CMake and compiles without problems on Ubuntu 16.04.
When starting the compiled application I get the message cannot
open shared object file.
All the shared object libs are available in the same non-standard
folder (and I need them there).
For some reason some can be found, but others cannot.
What I need
The reason why some shared objects can be found and others cannot be found. The project is pretty big with many CMake files. I tried to find the differences between the libs that can be loaded and the ones that can't, but without success. Any help that point me to the right place is welcome.
A solution within CMake to make it find all the shared objects.
ldd
The ldd output shows me that most shared objects can be found. Here are some examples:
libboost_filesystem.so.1.55.0 => /path/to/libs/boost/lib/libboost_filesystem.so.1.55.0 (0x00007f2ed1fa0000)
libboost_filesystem.so.1.55.0 => /path/to/libs/boost/lib/libboost_filesystem.so.1.55.0 (0x00007f96af1f5000)
libboost_program_options.so.1.55.0 => /path/to/libs/boost/lib/libboost_program_options.so.1.55.0 (0x00007f96aef85000)
libboost_system.so.1.55.0 => /path/to/libs/boost/lib/libboost_system.so.1.55.0 (0x00007f96aed80000)
For some reason a few others cannot be found. For example:
libboost_iostreams.so.1.55.0 => not found
libboost_chrono.so.1.55.0 => not found
There are other non-boost libs that showing the same behavior, but for simplicity I am just showing the boost examples.
Workarounds already tried
Below are the workarounds that already work successfully. But I am really interested in the two points in the What I need section.
Copying to standard folders like /usr/lib and running ldconfig
Adding the path to LD_LIBRARY_PATH

tldr; Check that the imported library is imported as SHARED or UNKNOWN and not STATIC, and has an IMPORTED_SONAME property.
You should check how the library is being imported that you're linking.
I've analyzed a few of the config mode exported targets for static and dynamic libraries, and they are a little different in the properties they set for the target.
For example for zlib, here's the version for static libraries:
add_library(ZLIB::zlibstatic STATIC IMPORTED)
set_target_properties(ZLIB::zlibstatic PROPERTIES
IMPORTED_LINK_INTERFACE_LANGUAGES_NOCONFIG "C"
IMPORTED_LOCATION_NOCONFIG "${_IMPORT_PREFIX}/lib/libz.a"
)
However, for dynamic libraries it's:
add_library(ZLIB::zlib SHARED IMPORTED)
set_target_properties(ZLIB::zlib PROPERTIES
IMPORTED_LOCATION_NOCONFIG "${_IMPORT_PREFIX}/lib/libz.so.1.2.11"
IMPORTED_SONAME_NOCONFIG "libz.so.1"
)
In a find_package module mode script, you might think you're importing a static library, when in fact the found library is a .so, so it may use the incorrect target properties. In config mode this is not likely because it's much more explicit when defining the target. Though, module mode (cmake/findXXX.cmake), you're often defining these properties as a result of FIND_PACKAGE_HANDLE_STANDARD_ARGS _LIBRARIES variable, and it's hard to tell what you're going to get. You can use libFoo.a in the search to be more explicit or playing with CMAKE_FIND_LIBRARY_SUFFIXES.

Related

How to link external C++ library to my project on Mac OSX [duplicate]

I have 2 folders "inc" and "lib" in my project which have headers and static libs respectively. How do I tell cmake to use those 2 directories for include and linking respectively?
The simplest way of doing this would be to add
include_directories(${CMAKE_SOURCE_DIR}/inc)
link_directories(${CMAKE_SOURCE_DIR}/lib)
add_executable(foo ${FOO_SRCS})
target_link_libraries(foo bar) # libbar.so is found in ${CMAKE_SOURCE_DIR}/lib
The modern CMake version that doesn't add the -I and -L flags to every compiler invocation would be to use imported libraries:
add_library(bar SHARED IMPORTED) # or STATIC instead of SHARED
set_target_properties(bar PROPERTIES
IMPORTED_LOCATION "${CMAKE_SOURCE_DIR}/lib/libbar.so"
INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_SOURCE_DIR}/include/libbar"
)
set(FOO_SRCS "foo.cpp")
add_executable(foo ${FOO_SRCS})
target_link_libraries(foo bar) # also adds the required include path
If setting the INTERFACE_INCLUDE_DIRECTORIES doesn't add the path, older versions of CMake also allow you to use target_include_directories(bar PUBLIC /path/to/include). However, this no longer works with CMake 3.6 or newer.
You had better use find_library command instead of link_directories. Concretely speaking there are two ways:
designate the path within the command
find_library(NAMES gtest PATHS path1 path2 ... pathN)
set the variable CMAKE_LIBRARY_PATH
set(CMAKE_LIBRARY_PATH path1 path2)
find_library(NAMES gtest)
the reason is as flowings:
Note This command is rarely necessary and should be avoided where there are other choices. Prefer to pass full absolute paths to
libraries where possible, since this ensures the correct library will
always be linked. The find_library() command provides the full path,
which can generally be used directly in calls to
target_link_libraries(). Situations where a library search path may be
needed include: Project generators like Xcode where the user can
switch target architecture at build time, but a full path to a library
cannot be used because it only provides one architecture (i.e. it is
not a universal binary).
Libraries may themselves have other private library dependencies that
expect to be found via RPATH mechanisms, but some linkers are not able
to fully decode those paths (e.g. due to the presence of things like
$ORIGIN).
If a library search path must be provided, prefer to localize the
effect where possible by using the target_link_directories() command
rather than link_directories(). The target-specific command can also
control how the search directories propagate to other dependent
targets.
might fail working with link_directories, then add each static library like following:
target_link_libraries(foo /path_to_static_library/libbar.a)

C++ linux executable keeps trying to use library that does not exist

I am trying to write a simple application with GLFW on Linux. Right now the main file (the only file) is basically just a few lines of code to make sure the dynamic library linked correctly. Here it is:
#include <GLFW/glfw3.h>
#include <iostream>
int main()
{
glfwInit();
std::cout << "It works this far!" << std::endl;
glfwTerminate();
}
The include files are stored in a directory labelled "include" and the library files are stored in a directory labelled "lib". As of right now, I am compiling the program with the following line:
g++ -Wl,-Rlib -Iinclude -Llib test.cpp -o test -lglfw.3.2
It compiles and links just fine, but when I try to execute it, I get the following error:
./test: error while loading shared libraries: libglfw.so.3: cannot open shared object file: No such file or directory
Now, before you rush to downvote this question into oblivion and mark it as a duplicate, at least allow me to explain why I believe my question is different enough to not be a duplicate. I already attempted the solutions that the other questions presented, but it was unsuccessful. As you can see, I tried setting the path to the library during linking with the -Wl,-Rlib tag. I also tried setting LD_LIBRARY_PATH to point to the location of my libraries (the 'lib' folder), but it still threw the same error. (It didn't matter if the path was relative or absolute.)
So, the next thing I tried was running the ldd command on the executable. I got some other dependencies that were working just fine, but importantly, I got this:
libglfw.so.3 => not found
For some reason, it insists on looking for libglfw.so.3. It will not have it any other way. Upon renaming the library from libglfw.3.2.so to libglfw.so.3, the program executed just fine and printed It works this far! as if there were no problems at all.
Why would this happen?
For some reason, it insists on looking for libglfw.so.3. ... Upon renaming the library from libglfw.3.2.so to libglfw.so.3 ...
The ELF executables contain the exact name of the dynamic libraries used.
If the executable contains the library name "libglfw.so.3" the file must be named exactly like this.
The file naming scheme is intentionally done in a way that not the "full" version is coded into the file name: This way a later version ("libglfw.so.3.15") will work with the executable.
Normally there should be a symbolic link to the latest version of the library installed:
libglfw.so.3 -> libglfw.so.3.2
This symbolic link seems to be missing on your computer. I would say that this is an installation problem!
EDIT
The question could be: Why is the file name stored in the executable file not libglfw.3.2.so but libglfw.so.3?
The answer has to do with the backward compatibility when a new version of a library is installed:
Normally you would use the switch -lglfw and a symbolic link named libglfw.so is looked up.
If you stored the file name libglfw.so in the executable file and a new, incompatible version if this library (libglfw.so.4) is installed you would have no chance to get the program running by having both versions of the library installed.
To enable backward compatibility by having both versions of the library installed the "real" symbolic link name of the library (libglfw.so.3) must be stored in the executable file.
Therefore the "expected" file name of a library is stored in the library itself: Inside the file libglfw.so.3.2 you'll find some information that the file expects itself to be stored as libglfw.so.3.
The linker will use this information about the file name because it assumes that the library name given in the linker switch (-lglfw) is less "precise" than the name stored in the library itself.
For some reason, it insists on looking for libglfw.so.3. It will not have it any other way.
This is the Linux convention for shared libraries which is described here among other places. For Linux libfoo.so.x.y.z is considered to have the same ABI as libfoo.so.x. Usually when shared libraries are installed (e.g. via rpm, dpkg, etc.) there's an invocation of ldconfig that happens so that the just installed libraries have a symlink following the convention installed that references the library. Also these libs (if installed to a "trusted location"), are added to a linker cache for performance reasons.
It compiles and links just fine, but when I try to execute it, I get the following error:
./test: error while loading shared libraries: libglfw.so.3: cannot open shared object file: No such file or directory
libglfw.so.3 isn't on ld-linux.so's path.
As you can see, I tried setting the path to the library during linking with the -Wl,-Rlib
Still won't find it -- libglfw.so.3 isn't on ld-linux.so's path. You can add it by doing something like:
ldconfig -n /path/to/lib
Which should output the requisite libglfw.so.3 symlink for your lib.
IIRC setting the rpath might require a full path.
I also tried setting LD_LIBRARY_PATH to point to the location of my libraries
Again, libglfw.so.3 isn't on ld-linux.so's path.

Cmake produces file extensions in static library archives

I recently switch from Autotools to CMake because CMake seems to be better for cross platform development and what I've noticed is when I build a static library of my C++ code all the files inside have a suffix .cpp.o
ar -t PA8/libgenericTZR.a
genericTZR.cpp.o
I've looked at other libraries built by other tools and they don't do that. I'm not sure if this is really a bad thing but, how can I get CMake to build the static libraries without the .cpp added file extension?
This is my CMake File
add_executable(PA8 ISP_Charges.cpp genericTZR.cpp genericTZR.h)
set(LIBSRC genericTZR.c genericTZR.h)
add_library(genericTZR SHARED ${LIBSRC})
add_library(genericTZR SHARED $<TARGET_OBJECTS:myObjects>)
add_library(genericTZR-static STATIC ${LIBSRC})
set_target_properties(genericTZR-static PROPERTIES OUTPUT_NAME $
install (TARGETS PA8 DESTINATION bin)
install (TARGETS genericTZR genericTZR-static DESTINATION lib)
install (FILES "${PROJECT_BINARY_DIR}/PA8/genericTZR.h" DESTINA$
Thanks
According to this thread on cmake list (and I agree with it), it is not a good idea to try to change the output file names.
Consider the example gave in the link:
add_executable(foo foo.c foo.cpp)
Generated objects would be foo.c.o and foo.cpp.o. They would conflict if you forced them to have just the .o extension.
You can try to use the non-documented, internal, might-change-in-the-future
set(CMAKE_C_OUTPUT_EXTENSION_REPLACE 1)
set(CMAKE_CXX_OUTPUT_EXTENSION_REPLACE 1)
I don't know if they're still available or if works at all. I've never used them.
As comments on your post have clarified it, the names inside the static lib aren't really used, you shouldn't worry about them.

Cannot open shared object file

I am trying to compile one of the projects found here
USB-I2C/SPI/GPIO Interface Adapter.
I downloaded the i2c_bridge-0.0.1-rc2.tgz package. I installed libusb and that seemed to go well with no issues. I go into the i2c_bridge-0.0.1-rc2/ directory and make. That compiles. I move into the i2c_bridge-0.0.1-rc2/i2c folder and make. It compiles and gives me ./i2c. However, when I run it, it says error while loading shared libraries: libi2cbrdg.so: cannot open shared object file: No such file or directory
The makefile in i2c_bridge-0.0.1-rc2/i2c has the library directory as ../. The libi2cbrdg.so is in this directory (i2c_bridge-0.0.1-rc2). I also copied the file to /usr/local/lib. An ls of the i2c_bridge-0.0.1-rc2/ directory is
i2c i2cbrdg.d i2cbrdg.o libi2cbrdg.a Makefile tests
i2cbrdg.c i2cbrdg.h INSTALL libi2cbrdg.so README u2c4all.sh
(That i2c is a directory)
If I sudo ./i2c, it still gives me the problem.
I had to take away the -Werror and -noWdecrepated (spelling?) options in all the makefiles to get them to compile, but that shouldn't affect this should it?
What else is necessary for it to find the .so file? If anyone can help me find out what is wrong I would be very grateful. If more information is needed I can post it.
You have to distinguish between finding so's at compile-time and at run-time. The -L flag you give at compile-time has nothing to do with localizing the library at run-time. This is rather done via a number of variables and some paths embedded in the library.
The best hot-fix for this problem is often setting LD_LIBRARY_PATH to the directory with the .so file, e.g.:
$ LD_LIBRARY_PATH=.. ./i2c
For a long-term solution, you need to either have a close look at the whole LD system with rpath and runpath, or use libtool (which solves these issues for your portably).
Copying a file to /usr/local/lib is often insufficient because ld caches the available libraries, so you need to re-run ldconfig (as root) after you copied a library to /usr/local/lib.
If you are building the code from source that needs the the library, you can put the path that the library is in in the environment variable LD_RUN_PATH before building, and the linker will save that path into the binary, so that it will automatically be looked for in the right place at runtime.
Linux specific: Alternately, put the library in /lib, /usr/lib, or some other path referenced in your /etc/ld.so.conf or its imported config fragments, and then all you need to do is run /sbin/ldconfig to refresh ld.so (the dynamic linker)'s cache of libraries.
This works for my issue,hope will help anyone.
gcc test.c -Wl,-rpath /usr/local/lib -lfcgi -o test.fcg
And -Wl,-rpath option is the key trick.

organizing external libraries and include files

Over the years my projects use more and more external libraries, and the way I did it starts feeling more and more awkward (although, that has to be said, it does work flawlessly). I use VS on Windows, CMake on others, and CodeComposer for targetting Digital Signal Processors (DSPs) on Windows. Except for the DSPs, both 32bit and 64bit platforms are used.
Here's a sample of what I am doing now; note that as shown, the different external libraries themselves are not always organized in the same way. Some have different lib/include/src folders, others have a single src folder. Some came ready-to-use with static and/or shared libraries, others were built
/path/to/projects
/projectA
/projectB
/path/to/apis
/apiA
/src
/include
/lib
/apiB
/include
/i386/lib
/amd64/lib
/path/to/otherapis
/apiC
/src
/path/to/sharedlibs
/apiA_x86.lib -->some libs were built in all possible configurations
/apiA_x86d.lib
/apiA_x64.lib
/apiA_x64d.lib
/apiA_static_x86.lib
/apiB.lib -->other libs have just one import library
/path/to/dlls -->most of this directory also gets distributed to clients
/apiA_x86.dll and it's in the PATH
/apiB.dll
Each time I add an external libary, I roughly use this process:
build it, if needed, for different configurations (release/debug/platform)
copy it's static and/or import libraries to 'sharedlibs'
copy it's shared libraries to 'dlls'
add an environment variable, eg 'API_A_DIR' that points to the root for ApiA, like '/path/to/apis/apiA'
create a VS property sheet and a CMake file to state include path and eventually the library name, like include = '$(API_A_DIR)/Include' and lib = apiA.lib
add the propertysheet/cmake file to the project needing the library
It's especially step 4 and 5 that are bothering me. I am pretty sure I am not the only one facing this problem, and would like see how others deal with this.
I was thinking to get rid of the environment variables per library, and use just one 'API_INCLUDE_DIR' and populating it with the include files in an organized way:
/path/to/api/include
/apiA
/apiB
/apiC
This way I do not need the include path in the propertysheets nor the environment variables. For libs that are only used on windows I even don't need a propertysheet at all as I can use #pragmas to instruct the linker what library to link to.
Also in the code it will be more clear what gets included, and no need for wrappers to include files having the same name but are from different libraries:
#include <apiA/header.h>
#include <apiB/header.h>
#include <apiC_version1/header.h>
The withdrawal is off course that I have to copy include files, and possibly** introduce duplicates on the filesystem, but that looks like a minor price to pay, doesn't it?
** actually once libraries are built, the only thing I need from them is the include files and thie libs. Since each of those would have a dedicated directory, the original source tree is not needed anymore so can be deleted..
Why not use file system links?
ln -s /path/to/apis/apiA/include /path/to/api/include/apiA
Voilá. Similar can be done on Windows, but I don't have the command line handy right now.