Linking against built static libraries rather than using add_subdirectory? - c++

Given a project with "app" and "lib" sibling directories, where "app" builds an executable depending on the (static) library built by "lib". What I'd like is a situation where the normal build process builds only the library, but if I build "app" it builds both "lib" and "app".
What I'm currently doing now is that in app, I include lib with add_subdirectory, but for whatever reason this is pulling in all of lib's indirect dependencies into the link line through a mechanism I'm not aware of. What I'd like is to have my app just build libmylib.a and libmylib.pc, then app could just calculate its own link line from libmylib.pc (or specify it manually), but I'm not sure how that's done.
Here's a minimum working example I've got set up now:
lib/CMakeLists.txt
cmake_minimum_required(VERSION 3.8.0)
project(mylib CXX)
find_package(PkgConfig REQUIRED)
pkg_check_modules("mylib" "libssl")
find_package(Boost REQUIRED)
set(LIBDIR "${PROJECT_SOURCE_DIR}")
set(HEADERS "${LIBDIR}/map_printer.hpp")
set(SOURCES "${LIBDIR}/map_printer.cpp")
add_library("mylib" "${SOURCES}")
target_include_directories("mylib" PUBLIC "${LIBDIR}"
"${Boost_INCLUDE_DIR}"
"${mylib_INCLUDE_DIRS}")
target_link_libraries("mylib" "${Boost_LIBRARIES}" "${mylib_LIBRARIES}")
install(TARGETS "mylib" ARCHIVE DESTINATION "lib")
install(FILES ${HEADERS} DESTINATION "include")
app/CMakeLists.txt
cmake_minimum_required(VERSION 3.8.0)
project(mylib CXX)
set(APPDIR "${PROJECT_SOURCE_DIR}")
set(LIBDIR "${APPDIR}/../lib")
set(SOURCES "${APPDIR}/main.cpp")
add_subdirectory("${LIBDIR}" "build")
list(APPEND LIBS "mylib")
add_executable("myapp" "${SOURCES}")
target_include_directories("myapp" PUBLIC "${LIBDIR}")
target_link_libraries("myapp" "${LIBS}")
install(TARGETS "myapp" DESTINATION "bin")
To get a working example, here are some source files that pull in libssl in the lib (but this function is not used in the app) - I put them in gists because they are only included for completeness and I didn't want to clutter up the question text:
lib/map_printer.cpp
lib/map_printer.hpp
app/main.cpp
The problem is that when I cmake app and then do make VERBOSE=1, the linker command generated is:
/usr/lib/hardening-wrapper/bin/c++ CMakeFiles/myapp.dir/main.cpp.o -o myapp build/libmylib.a -lssl
But I have not specified -lssl anywhere in app. Normally this would be fine, but in my real app, -lssl and several other unnecessary symbols are being included as .so files because of indirect dependencies. When I remove them from the linker command manually, the task builds and runs just fine. Ideally I'd be pulling in the built .a with its .pc file (not generated in this example) and if excess dependencies are necessarily being pulled in, I could tweak the link line manually, but with this method, the linker flags (and possibly other things) are leaking out of the lib scope in some way I don't understand.

Linking is all about resolving symbols so that the final target (standalone executable or shared object) has everything it needs to be launched. Things are simple when you depend only on a single library or upon a collection of libraries that in turn depend on nothing else. This is less likely to be the case for any program of moderate or larger size. The underlying issue is how to deal with transitive dependencies, e.g. the dependencies of the things directly used by your program.
CMake understands all of this and is designed to make it simple for you to consume libraries without understanding the entire dependency graph. If you look at the documentation for target_link_libraries, you'll see the PRIVATE, PUBLIC, and INTERFACE keywords described. This allows you to describe the private requirements of the library (compile definitions, compile arguments, dependent libraries, etc.) that it needs when you build the library. The public part allows you to specify things that both the library and its dependents (consumers) need. The interface part lets you specify things that the dependents need, but not the library itself. The commands target_compile_definitions and target_include_directories operate similarly.
The upshot of all of this is that with a properly declared dependency in CMake, the client of that dependency just adds it to its list of dependencies in its own target_link_libraries command and it naturally picks up all compile definitions, include directories and transitory link dependencies necessary for successful compilation and linking.
A CppCon 2017 presentation Modern CMake for modular design goes over this in more detail.

Related

Where does target_link_libraries look for the required files?

I am building a project from source and am trying to understand what is happening in the CMakeLists.txt files.
Where exactly is target_link_libraries looking for the required library files?
The specific CMakeLists.txt file I have has:
target_link_libraries(MyApplication PRIVATE
Magnum::Application
Magnum::GL
Magnum::Magnum
Magnum::Shaders)
I found folders that have the names GL and Shaders in a directory called Magnum in the project, and they have a collection of header files in them. I believed that target_link_libraries is telling CMake to include the libraries in the GL and Shaders folder.
However, I cannot seem to find a corresponding folder for Application, hence my line of reasoning must be flawed.
I do know target_link_libraries is doing something related to allowing the finally-put-together program to be able to use a set of libraries.
What exactly does target_link_libraries do? Where does it look for the required files in order to be able to use the libraries it needs to?
target_link_libraries doesn't link anything automatically. You should have a target previously created via add_library (or add_executable), where all files are listed.
The way these targets added into your CMake project may differ. E.g. you may have a library source files with CMakeLists.txt configuration (where the said add_library command is) under some folder libs/mylib. Then in your CMakeLists.txt you may have the library added with add_subdirectory(libs/mylib). Another option is to add the library with find_package.
In your specific case, you pass targets to target_link_libraries(). Targets with a shape like Magnum::Application etc are either imported targets or ALIAS targets.
From what you say, it seems than Magnum is vendored into your project, so I guess your are linking ALIAS targets, like the one defined here https://github.com/mosra/magnum/blob/cfc02599e54e02337dd56bb61f70b2e61eb9ce8d/src/Magnum/CMakeLists.txt#L295
Targets defined by add_library() or add_executable() in CMake are an abstraction carrying several informations, including location of files.

CMake `INSTALL` for targets and its SO dependencies

My target linked with several libraries using TARGET_LINK_LIBRARIES with PUBLIC keyword, The INSTALL command looks like INSTALL(TARGETS foo DESTINATION ${CMAKE_INSTALL_PREFIX}/bin). I want somehow to force the cmake to include all (preferably excluding system libraries) libraries (SOs only) I link with to be included in the installation process. I've tried EXPORT keyword but looks like it affects only libraries which I build with in my project and marked with the same EXPORT as foo library.
Is it possible?
EDIT001: Additional information that may affect answer.
I'm using vcpkg to manage third parties. So the TARGET_LINK_LIBRARIES looks like
TARGET_LINK_LIBRARIES(foo PUBLIC
GTest::GTest
GTest::Main
${GOOGLE_MOCK}
event
${THRIFT_LIBRARIES}
${Boost_LIBRARIES}
lzo2
sqlite3
${ZeroMQ_LIBRARY}
gRPC::grpc
gRPC::grpc++
xml2
stdc++fs
bfd
-l:libisal.so.2
sgutils2
pthread
uuid
rt
)
So, essentially what I want to achieve is to take all these libraries which are macro'ed by vcpkg, like ${THRIFT_LIBRARIES}, ${Boost_LIBRARIES} and gRPC::grpc and so on
As of cmake 3.21, you can now do with:
install(IMPORTED_RUNTIME_ARTIFACTS gRPC::grpc)
install(IMPORTED_RUNTIME_ARTIFACTS ${Boost_LIBRARIES})
etc.
See new Install command.
CMake itself does not allow to install dependencies automatically. This would be a rather hard task, because it would have to consider a lot of corner cases.
Just think of transitive dependencies (I don't know if this is the right word), like: Your libA depends on libB, which depends on libC. How should CMake get this from the CMakeLists, where only libB is listed?
Or: What do you consider a system library? Everything that is not in PATH? How do you know which libraries are installed system-wide on the client's machine?
You see, there are some really tricky things to consider.
Here are some possibilities you have:
Ask your users to install the dependencies.
Statically link libraries into your binary.
Copy library files using install(FILES files... DESTINATION <dir>). Maybe your dependency manager can help creating the list of files.
Write a script that does something like windeployqt for Qt-based applications on Windows: Analyze the binary file (e.g. using ldd myApp) and automatically copy over the required dependencies.

Linking a static library to a shared library in cmake

I have been working on a fun project (a game engine) for awhile now and figured i could make it more portable and easy to compile across platforms if I use cmake. Right now i have it set up like so, with a main executable and then a bunch of shared libraries that the executable is linked to. I've found all the material needed to produce the libraries and the executable, and linking those to the executable, but what of linking a dependency like a static library or another shared library to one of the libraries i produce? Here is a visual
Sentiment (name of project)
-Root (all the interfaces and components of the engine. main.cpp is here
-Sentiment_OGL4Renderer (the files for the Renderer library)
-Sentiment_SFMLGui (the files for the Gui library)
-Sentiment_TestGame (the code for a game)
now i want all of these, the executable and the shared libraries built and put into the bin folder in the top level directory. What i found suggested online for a setup like this was to make cmakelists.txt files in each folder, and then one in the root, for each project. What i have thus far is this.
#Sentiment
cmake_minimum_required(VERSION 2.6)
project(Sentiment)
set(RENDERER Sentiment_OGL4Renderer)
set(GUI Sentiment_SFMLGui)
set(GAME Test_Game)
add_definitions(-DBUILD_DLL)
list( APPEND CMAKE_CXX_FLAGS "-std=c++11 ${CMAKE_CXX_FLAGS} -g -ftest-coverage -fprofile-arcs")
set(EXECUTABLE_OUTPUT_PATH "${Sentiment_SOURCE_DIR}/bin")
set(LIBRARY_OUTPUT_PATH "${EXECUTABLE_OUTPUT_PATH}")
link_directories("${LIBRARY_OUTPUT_PATH}")
add_subdirectory("${RENDERER}")
add_subdirectory("${GUI}")
add_subdirectory("${GAME}")
add_subdirectory(Root)
in root
project(Sentiment_exe)
link_directories("${Sentiment_SOURCE_DIR}/bin")
AUX_SOURCE_DIRECTORY(. new_source_list)
add_executable("${CMAKE_PROJECT_NAME}" ${new_source_list})
target_link_libraries("${CMAKE_PROJECT_NAME}" "${LIBRARY_OUTPUT_PATH}/${RENDERER}" "${LIBRARY_OUPUT_PATH}/${GUI}" "${LIBRARY_OUTPUT_PATH}/${GAME}" "${ADDITIONAL_DEPENDENCIES}")
in Sentiment_OGL4Renderer
project(Sentiment_OGL4-3Renderer)
include_directories(${Sentiment_SOURCE_DIR})
add_definitions(-DGLEW_STATIC)
add_library(Sentiment_OGL4-3Renderer SHARED Sentiment_OGL4Renderer.cpp GL/glew.cpp)
in Sentiment_SFMLGui
project(Sentiment_SFMLGui)
include_directories(${Sentiment_SOURCE_DIR})
add_library(Sentiment_SFMLGui SHARED Sentiment_SFMLGui.cpp)
in Sentiment_TestGame
project(Sentiment_Game)
include_directories(${Sentiment_SOURCE_DIR})
add_library(Sentiment_Game SHARED Game.cpp)
As you can tell there are a lot of third party libraries, and i tried various methods of linking, like with target_link_libraries, and i cannot for the life of me figure how to link an external library to the ones i've made. First off, the renderer uses GLEW but it needs no external dependency so ignore that. Then it needs OpenGL32.lib and Gdi32.lib from the windows sdk (only for windows). As for SFML, i've got the DLL's in the bin folder which need to be linked, (can easily get the .so's when working in linux and can distribute with the final product if I ever choose to do so). I need these all linked as dependencies to the libraries i create, but nothing seems to work. The project is all c++ and I am currently using mingw32 to compile it. I'm brand new to cmake so please be gentle if it is really simple.
To link external libraries, best practice is to use or create FindModule for given external library.
CMake comes with numerous modules that aid in finding various well-known libraries and packages.
The list of standard modules is in the official documentation
In case there is no standard module for your external library, you should write your own.
The OpenGL library has standard module FindOpenGL:
find_package (OpenGL)
if (OPENGL_FOUND)
include_directories(${OPENGL_INCLUDE_DIR})
target_link_libraries (Sentiment_OGL4-3Renderer ${OPENGL_gl_LIBRARY})
endif (OPENGL_FOUND)

How can I reduce unnecessary C++ library dependency introduced by CMake?

Now I am building a C++ project with CMake. I find CMake will introduce unnecessary library dependency in the project. Take an example, my project is composed of four parts: 1)lib1 2)lib2 3)lib3 and 4)app:
------lib1---
|----
lib2---
|----
lib3---
|----
app---
The source code in app will build a program, which relies on the dynamic library created in lib3. lib3 however, relies on the dynamic library created in lib2 and so on. I build the following CMake scripts to build a VC10 project:
1) Root CMakeLists:
cmake_minimum_required( VERSION 2.6 )
project (test)
add_subdirectory(lib1)
add_subdirectory(lib2)
add_subdirectory(lib3)
add_subdirectory(app)
2) lib1 CMakeLists.txt
add_definitions (-DEXP_STL )
add_library(lib1 SHARED lib1.cxx)
3) lib3 CMakeLists.txt
add_definitions (-DEXP_STL )
add_library(lib3 SHARED lib3.cxx)
target_link_libraries(lib3 lib2)
4) app CMakeLists.txt
add_executable(app main.cpp)
target_link_libraries(app lib3)
With these CMake scripts I have no problem in building a VC10 project. However, I notice that CMake will introduce unnecessary library dependency between libraries for VC10. For example, for the app application program, it only relies on one library, and that is, lib3. However, in VC10 project, I notice that it added the following additional dependencies:
..\lib3\Debug\lib3.lib
..\lib2\Debug\lib2.lib
..\lib1\Debug\lib1.lib
In the CMake script, however, only lib3 dependency is supposed to be introduced. For our example project, it may not be a problem, but in other cases the introduced redundant libraries can lead to compiling errors as they demand the proper searching paths. I am therefore wondering whether there is a way to eliminate these unnecessary libraries. Thanks!
CMake adds in dependent libraries transitively, which can be turned off by setting the property LINK_INTERFACE_LIBRARIES to an empty string. If you do
SET_TARGET_PROPERTIES(lib3 PROPERTIES LINK_INTERFACE_LIBRARIES "")
then CMake will not generate a dependency from app to lib1 and lib2, when linking app.
If you were creating staic libraries, then I would agree that CMake was pulling in unnecessary dependencies. However, you're building dynamic libraries and I don't believe that CMake is adding unnecessary dependencies here. You have a chain of dynamic libraries. In this case, if you link in lib3, the linker also needs to pull in lib2, which would then cause the linker to pull in lib1 in order to satisfy all the symbol dependencies for linking your application.

How do I set up CMake to generate header-only projects?

I want to set up header-only C++ (or C) library projects, but can't find a clean way.
After some searches I've found that you can't set up a normal library using add_library to do this because it requires a compilable source file.
A way to do this would be to use add_custom_target instead, this way:
# Get all headers (using search instead of explicit filenames for the example)
file( GLOB_RECURSE XSD_HEADERS
*.hxx
)
add_custom_target( libsxsd SOURCES ${XSD_HEADERS} )
But that doesn't seem to work completely here as I can't see the sources in the project generated in VS2010. I don't know if it's a bug or if I'm doing it wrong or if there is a preferred way to do this.
Update: CMake will soon include a library target called INTERFACE that is ideal for header-only projects. This feature is currently in the master branch. Reference.
Using the command add_custom_target as you propose works for me (VS2010). The files are neatly listed within my project but it has the drawback that you can't define any "Additional Include Directories" with a custom target. Instead, I now use the following:
add_library(HEADER_ONLY_TARGET STATIC test1.hpp test2.hpp)
set_target_properties(HEADER_ONLY_TARGET PROPERTIES LINKER_LANGUAGE CXX)
This sets up your header-only project as a dummy archive target. Don't worry, no actual binaries will be generated if you should try and build it (at least not in VS2010 and Xcode 4). The command set_target_properties is there because CMake will otherwise complain that it cannot infer the target language from .hpp files only.
You can do this using the recent Interface Library feature:
add_library(mylib INTERFACE)
target_include_directories(mylib INTERFACE my_include_dir1 my_include_dir2)
This creates a library target without any source files, and adds the include directories to the INTERFACE_INCLUDE_DIRECTORIES property of the target. This means that any target that links to this library will get these directories as include paths (-I) when built.
For instance, to use the library with an executable target, just do:
add_executable(myexec ${MY_SOURCES})
target_link_libraries(myexec mylib)
I think what you are looking for is just adding an include directory using the
"include_directories" command for cmake.
When doing this, if it is a third party tool that you don't have control over, I would also add the "SYSTEM" flag.
So you command would look like something like this:
include_directories(SYSTEM ${GTEST_INCLUDE_DIRS})