CMake dynamically links `.a` files in `/usr/local/lib` - c++

I want to statically compile my program against another static library, for this example I'm using zeromq. Here is my CMakeLists.txt
cmake_minimum_required(VERSION 2.6)
add_executable( test test.cpp )
find_library(ZMQ NAMES libzmq.a)
message(STATUS ${ZMQ})
target_link_libraries( test ${ZMQ} )
It finds the .a file when I run mkdir build && cd build && cmake ..
-- /usr/local/lib/libzmq.a
However, if I examine the link.txt file, the library is dynamically linked:
/usr/bin/c++ CMakeFiles/test.dir/test.cpp.o \
-o test -rdynamic /usr/local/lib/libzmq.a
The weird bit is that if I move the file to a different directory, say /usr/lib and run cmake .. once more, it locates the new path to the library:
-- /usr/lib/libzmq.a
But now it has magically changed to static linking:
/usr/bin/c++ CMakeFiles/test.dir/test.cpp.o \
-o test -rdynamic -Wl,-Bstatic -lzmq -Wl,-Bdynamic
The same thing applies to other libraries I'm linking to.
Why are all my libraries in /usr/local/lib being dynamically linked?

You should not use the path directly, and create an imported target instead, so you can explicitly declare it static:
cmake_minimum_required(VERSION 2.6)
add_executable( test test.cpp )
find_library(zmq_location NAMES libzmq.a)
message(STATUS ${zmq_location})
add_library(zmq STATIC IMPORTED)
set_target_properties(zmq PROPERTIES IMPORTED_LOCATION ${zmq_location})
target_link_libraries( test zmq )
This may lead to a situation where the library appears to be linked dynamically, but the cmake source code has the answer:
If the target is not a static library make sure the link
type is shared. This is because dynamic-mode linking can handle
both shared and static libraries but static-mode can handle only
static libraries. If a previous user item changed the link type to
static we need to make sure it is back to shared.
Essentially, it's letting the linker handle detecting that the library is static if currently in dynamic-mode.

The answer to my original question about the difference between /usr/local/lib and /usr/lib is that, by default, /usr/local/lib is not one of the implicit link directories. Therefore, a quick fix is to include this line in the config:
set(CMAKE_CXX_IMPLICIT_LINK_DIRECTORIES /usr/local/lib ${CMAKE_CXX_IMPLICIT_LINK_DIRECTORIES})
However, as pointed out in this other answer, referencing files directly is not the way to go, instead one should use add_library.

Here is my DEMO:
find_library(VLQ_LIB NAMES vlq-shared PATHS /usr/local/lib/ REQUIRED)
find_path(VLQ_HEADER vlq.hpp /usr/local/include)
target_link_libraries(myproj PUBLIC VLQ_LIB)
target_include_directories(myproj PUBLIC VLQ_HEADER)
but during run, you still need to copy the shared lib to /lib/ folder, for static lib, you can keep it under /usr/lib

Related

CMake refuses to link shared library

Probably the error is trivial for CMake experts but I seem to be blind to it. I've tried two approaches.
I have a library written in C and compiled using gcc 9.4.0 as both, static and dynamic library. I've compiled the library apart and it successfully builds and links against its main.c to produce an executable using a Makefile. When I do nm -D mylib.so I get all the symbols and functions I need.
Now, I am trying to use this library with ROS (meaning build and link with CMake) but the program fails to link. I've tried both approaches: use the library as is, and re-build the library using CMake. Both fail. For the sake of keeping the example short I've removed the lines used by catkin that basically tell CMake to link against all ROS infrastructure. Suffices to say that the main.cpp of the target (my_node) consumes a void* CreateEnvironment(); function from the library which is not found (same with any other function). Say:
#include <mylib/mylib.h>
int main(int argc, char** argv){
foo();
return 0;
}
void foo(){
void ptr = CreateEnvironment();
}
Approach #1: Use pre-built library (preferred)
With CMake it refuses to link. CMakeLists.txt as follows:
cmake_minimum_required(VERSION 3.0.2)
find_library(
MY_LIB
NAME mylib
PATHS "${PROJECT_SOURCE_DIR}/lib"
NO_DEFAULT_PATH # Can add or remove this, no difference
)
add_library(mylib SHARED IMPORTED)
set_target_properties(mylib PROPERTIES IMPORTED_LOCATION "${MY_LIB}")
add_executable(my_node src/main.cpp)
target_link_libraries(my_node mylib ${catkin_LIBRARIES})
It runs:
/usr/bin/c++ -rdynamic CMakeFiles/…/main.cpp.o -o /home/…/clips_node -Wl,-rpath,/home/…/lib:/opt/ros/noetic/lib /home/…/lib/libmylib.so […]
/usr/bin/ld: CMakeFiles/…/main.cpp.o: in function `foo()':
main.cpp:(.text+0x11): undefined reference to `CreateEnvironment()'
Approach #2: Build library with CMake
Tried this approach thinking maybe the library I have was built with different standard, or not all symbols were exported, or exports were incompatible or whatever. A build made by the same system and forcing C++ rather than C should make it linkable... but did not.
Once again, the SO is created and all symbols are visible with nm -D. It is the linking with the executable target what fails.
cmake_minimum_required(VERSION 3.0.2)
## Build library
SET(MYLIB_INCLUDE_DIR "${PROJECT_SOURCE_DIR}/include/mylib")
AUX_SOURCE_DIRECTORY(${PROJECT_SOURCE_DIR}/src/mylib MYLIB_SOURCES)
add_library(mylib ${MYLIB_SOURCES})
target_include_directories(mylib PRIVATE ${MYLIB_INCLUDE_DIR})
set_target_properties(mylib PROPERTIES
LINKER_LANGUAGE "CXX"
ENABLE_EXPORTS 1
)
## Build node
add_executable(my_node src/main.cpp)
target_include_directories(my_node PUBLIC ${MYLIB_INCLUDE_DIR})
target_link_libraries(my_node mylib ${catkin_LIBRARIES})
Output is almost identical to that of the Approach #1, only the path of the library changes.
What else have I tried
Changing the order of the elements in target_link_libraries
Changing PRIVATE for PUBLIC
Add -lmylib to target_link_libraries
Add the full path to libmylib.so to target_link_libraries
Replace the shared object with a static library
So far it seems that the shared object is passed to the linker but, for some reason, it refuses to link the file. With a Makefile I know I can change the order since some libraries must be prepended and some appended, but CMake seems to be all automagic and I failed to find the error so far.

CMake - Changing shared library link options

I want to manipulate the linker call in my linker trampoline python script, but for the life of me I cannot get CMake to respect the CMAKE_CXX_CREATE_SHARED_LIBRARY value. It always uses the CMAKE_CXX_COMPILER for linking. If I switch to an add_executable and instead use CMAKE_CXX_LINK_EXECUTABLE this works perfectly. Is there anyway to convince CMake to respect the setting of CMAKE_CXX_CREATE_SHARED_LIBRARY?
A trivial CMakeLists.txt
cmake_minimum_required( VERSION 3.12 )
project( mylib )
include( CMakePrintSystemInformation )
add_library( mylib SHARED mylib.cpp )
A toolchain excerpt
I've tried this in both a toolchain file and an -C pre-cache option.
set( CMAKE_LINKER "/path/to/linker_trampoline.py" )
set( CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_LINKER> --CMAKE_CXX_COMPILER <CMAKE_CXX_COMPILER> --CMAKE_SHARED_LIBRARY_CXX_FLAGS <CMAKE_SHARED_LIBRARY_CXX_FLAGS> --LANGUAGE_COMPILE_FLAGS <LANGUAGE_COMPILE_FLAGS> --LINK_FLAGS <LINK_FLAGS> --CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS <CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS> --SONAME_FLAG <SONAME_FLAG> --TARGET_SONAME <TARGET_SONAME> --TARGET <TARGET> --OBJECTS <OBJECTS> --LINK_LIBRARIES <LINK_LIBRARIES>" CACHE STRING "" FORCE )
Works as expected with CMake version 3.10
I have tried your example. No issues with your intention. The only differences are, I tried this with an older version of CMake and placed the variable declarations directly in the file CMakeLists.txt (Should be placed in a toolchain file).
CMakeLists.txt:
cmake_minimum_required(VERSION 3.10)
project(mylib)
include(CMakePrintSystemInformation)
set (CMAKE_LINKER "/path/to/link.sh")
set (CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_LINKER> <OBJECTS>")
add_library(mylib SHARED mylib.cpp)
I have shortened the declaration of CMAKE_CXX_CREATE_SHARED_LIBRARY here because I only pass the object files to the following script.
link.sh: (Simple print the object files):
#!/bin/bash
echo "$#"
Output:
$ cmake CMakeLists.txt
$ make
[ 50%] Building CXX object CMakeFiles/mylib.dir/mylib.cpp.o
[100%] Linking CXX shared library libmylib.so
CMakeFiles/mylib.dir/mylib.cpp.o
[100%] Built target mylib
$
link.sh will be definitely executed. You an ignore the formal message about linking libmylib.so. The library will be never created (with my linker command).
CMake version:
$ cmake --version
cmake version 3.10.2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
$
I have seen you have already created an issue or rather a feature request on the CMake project site.
Response from Kitware
Brad King commented:
CMAKE_CXX_LINK_EXECUTABLE and CMAKE_CXX_CREATE_SHARED_LIBRARY are not meant to be set by toolchain files directly (and if either seems to work it is only by accident). They are not meant to be cached either. They are set in CMake's platform/compiler information modules which are loaded after the toolchain file. For toolchain files meant to cross-compile to a custom platform, they can set CMAKE_SYSTEM_NAME to the name of the target platform, set CMAKE_MODULE_PATH to a custom directory, and then add Platform/<os>*.cmake modules with such settings for that platform.
To hook in to the link line for an existing platform another approach will be needed. #18316 is related.

cmake: how to install the include/ and lib/ directories as autotool does

I have an old project and I want to make use of cMake instead of the old autotools.
What the old program does is that, after type make, it will make libtest.a, libtest.la, libtest.so.1.0.0 etc. inside a hidden folder called .libs, then after I type make install, it will intall all libraries to a target folder $TEST_ROOT/lib (environment variable), it will also install all .h files into $TEST_ROOT/include folder.
in the Makefile.am:
source_list=test1.cpp test2.cpp
include_HEADERS=test1.h test2.h
AM_LDFLAGS="-pthread -lm -lrt"
lib_LTLIBRARIES=libtest.la
libtest_la_SOURCES=$(source_list)
libtest_la_LDFLAGS=$(AM_LDFLAGS) -static-libgcc
in configure.ac, I only see one relevant line,
if test -n "${TEST_ROOT}"; then
ac_default_prefix=${TEST_ROOT}
includedir=${ac_default_prefix}/include
fi
Frankly I don't really understand why the above codes will make .a, .la, .so etc. libraries all together, and then install them into the curresponding folder. Probably autotools can recognize the "ac_default_prefix" and "includeddir"?
Anyway, I want to do the same thing with cmake, the following is my attempt, but not a complete solution.
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/.libs)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY $(CMAKE_BINARY_DIR}/.libs)
set(CMAKE_CXX_FLAGS "-O3 -Wall")
set(CMAKE_EXE_LINKER_FLAGS "-pthread -lm -lrt")
file(GLOB SOURCES "*.cpp")
add_library(test STATIC ${SOURCES})
The above code will compile libtest.a in the build folder, not in the .libs folder inside build folder (meaning that CMAKE_RUNTIME_OUTPUT_DIRECTORY etc doesn't work).
Secondly, it will only build libtest.a, there is no libtest.la, no libtest.so.1.0.0 etc.
Thirdly, I am still not sure how let make install work like the auto tools. Can I just set the target include directory and target lib directory, then make install will install all .h files and .so, .a, .la files into the target directory?
Please help.
Thanks.
You have to go to the coreesponding CMakeLists.txt and add e.g.
INSTALL(TARGETS test DESTINATION lib)
In your root CMakeLists.txt you can determine the standard installation path:
SET(CMAKE_INSTALL_PREFIX ".")
In a similar way you can install your header files:
FILE(GLOB files "${CMAKE_CURRENT_SOURCE_DIR}/*.hxx")
INSTALL(FILES ${files} DESTINATION include)
You can find even more examples at: https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/Install-Commands
You can then install the files and libs with make (https://cmake.org/cmake/help/v3.1/variable/CMAKE_INSTALL_PREFIX.html):
make DESTDIR=/home/mistapink install
You might need to set CMAKE_ARCHIVE_OUTPUT_DIRECTORY too.
Aside from that, be aware that CMake doesn't work like autotools. If you want multiple libraries you need multiple add_library calls.

Cmake linking to shared library cannot find library

On Ubuntu, I have two directories: build and src. In src, my CMakeLists.txt file has the lines:
add_executable(Test main.cpp)
target_link_libraries(Test libCamera.so)
After running cmake in the build directory (cmake ../src), I then copy my library file libCamera.so into the build directory. After running make, the main.cpp.o file compiles successfully, but I receive the following error during linking:
/usr/bin/ld: cannot find -lCamera
Why is this? The shared library is in the same directory that I am building in... and the same thing happens if I copy the library to /usr/bin...
You should not put prefix lib and suffix .so of the library, so just use:
target_link_libraries(Test Camera)
if your library not found you may need to add directory, where library is located:
link_directories( /home/user/blah ) # for specific path
link_directories( ${CMAKE_CURRENT_BINARY_DIR} ) # if you put library where binary is generated
Note: you copied lib to /usr/bin but unlike Windows where dll files stored with executables, in Linux that is not the case, so it would be /usr/lib, not /usr/bin. Also you may change LD_LIBRARY_PATH variable to make your program to find a library in a custom location.

Using CMake to statically link to a library outside of the project

I would like to use CMake to link my project to my shared library. The library is only shared between a handful of projects and is rather small, so I would really like to build it before it is linked. Building it every time seems a better idea than having to maintain an up-to-date precompiled version, because I ten to change it together with the project. It is separate, because it contains stuff I will almost certainly need in the next project.
How can I configure CMake to do it?
My current CMakeLists.txt for the relevant project looks like this:
find_package( Boost REQUIRED COMPONENTS unit_test_framework)
include_directories(${BaumWelch_SOURCE_DIR}/../../grzesLib/src
${BaumWelch_SOURCE_DIR}/src
${Boost_INCLUDE_DIRS})
if(CMAKE_COMPILER_IS_GNUCXX)
add_definitions(-g -std=c++11 -Wall -Werror -Wextra -pedantic -Wuninitialized)
endif()
# Create the unit tests executable
add_executable(
baumwelchtests stateindextest.cpp baumiterationtest.cpp baumwelchtest.cpp sampleparameters.cpp sdetest.cpp
# Key includes for setting up Boost.Test
testrunner.cpp
# Just for handy reference
exampletests.cpp
)
# Link the libraries
target_link_libraries( baumwelchtests ${Boost_LIBRARIES} baumwelchlib grzeslib)
but obviously the compilation fails with:
/usr/bin/ld: cannot find -lgrzeslib
You mentioned you'd like to build the library rather than use a precompiled version. If the library has a CMakeList, you should add it using add_subdirectory(path/to/the/library/source/directory). It will then become a subproject of your project and you can use names of its targets normally in your CMakeList.
Note that while the command is called add_subdirectory, it can be an arbitrary directory on disk; it doesn't have to be a subdirectory of the master project's source dir. In case it's not a subdirectory, you have to explicitly specify a binary directory for it as well. Example:
add_subdirectory(/path/to/the/library/source/directory subproject/grzeslib)
The second argument, if given as a relative path, is interpreted relative to CMAKE_CURRENT_BINARY_DIR.