I'm working on a project with a lot of external dependencies, which are included in the project tree. I would like to pre-build all of the dependencies and share the importable targets within the project tree.
I was planning to use cmake install with a CMAKE_INSTALL_PREFIX in the source tree, and use CMAKE_PREFIX_PATH to reference it for find_package. However, I'm beginning to wonder how maintainable this strategy is going to be? For example here's something I noticed in one of the installed cmake scripts:
${CMAKE_PREFIX_PATH}/lib/cmake/glfw3/glfw3Targets.cmake:
set_target_properties(glfw PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
INTERFACE_LINK_LIBRARIES "/usr/lib/x86_64-linux-gnu/librt.so;/usr/lib/x86_64-linux-gnu/libm.so;dl;/usr/lib/x86_64-linux-gnu/libX11.so;-lpthread"
)
It seems really suspicious to me that all of those link libraries are fully resolved to paths on the host machine.
I guess the question is: Are cmake installs to a prefix meant to be distributable and this is just a bad example, or are they meant to be tied to the machine you "install" them on? Ie. Is prefix really meant to just relocate where on the system things are supposed to be "installed", and my hope to use it as a shared package manager likely to be problematic?
Yes, EXPORT'ed CMake targets can be "distributable", but the project should follow some principles for achieve that.
If you link with a (external) library, but do not want export file to contain absolute path to it, then do not pass absolute path directly to target_link_libraries.
In case a linked library is shipped with a compiler (e.g. m or rt), things are simple: just pass the library's name to the target_link_libraries.
In case a linked library YYY comes from other package and is detected by find_package(YYY), this implies several things:
Script FindYYY.cmake or YYYConfig.cmake should return IMPORTED target. If this is not true, you may try to wrap its results into IMPORTED target.
target_link_libraries should use this IMPORTED target.
Script XXXConfig.cmake, shipped with your project, should use find_dependency(YYY) for discover a library on a user machine.
For find_dependency(YYY) work, a user machine should have FindYYY.cmake or YYYConfig.cmake script. Alternatively, you may ship FindYYY.cmake with your project, and adjust CMAKE_MODULE_PATH variable before find_dependency() call (in your XXXConfig.cmake).
Related
I would like to know how to add external libraries into my project. Is there a standard way of doing so?
The way I do it and that I don't like is:
Have a folder called vendors where I add submodules e.g. boost, openssl...
I build the external libraries (as they come with a cmake to build in general).
I add a premake (I could have used a cmake) to each external library and I configure so I can see the project in VS as well as the cpp and the hpp files.
I don't like this because I do copy the binaries of the external libraries manually, hence if I delete the bin folder I can't build my solution just by clicking build but I have to build the external libraries first using there cmake and then I copy the binaries manually to the bin/ folder of my solution.
Could you please give me a "standard" way I can do this? I feel like there could be better ways by just using to the max the CMake that comes with the external library. Also, I don't like changing the external libs too much, I just want to be able to update them anytime and everything works without me touching stuff.
How can this be done?
One approach is to use CMake's FetchContent functionality.
The FetchContent module allows specifying Git repositories of CMake projects to fetch at configure time. The default setting assumes that that repository has a CMakeLists.txt file at the repo's root directory. It clones the repo to a default (but configurable) location, and then just calls add_subdirectory() on the cloned directory.
You can read about how to use it in the reference documentation, and you can read about how that approach compares with some other CMake-supported approaches for using dependencies in the official Using Dependencies Guide. Do brace yourself when reading the reference docs, though. They're not designed to be like a beginner-friendly tutorial, and since FetchContent is built upon another module called ExternalProject, some of the docs for FetchContent just point you to go read sections from the ExternalProject docs. Be prepared to do a bit of digging.
Here's a basic example I used in a project at one point.
include(FetchContent)
FetchContent_Declare(
range-v3
GIT_REPOSITORY git#github.com:ericniebler/range-v3.git
GIT_TAG "0.12.0" # https://github.com/ericniebler/range-v3/releases
GIT_SHALLOW TRUE
GIT_PROGRESS ON
SYSTEM
)
# more `FetchContent_Declare`s (if any). They should all be declared
# before any calls to FetchContent_Make_available (see docs for why).
FetchContent_MakeAvailable(range-v3)
# more `FetchContent_MakeAvailable`s (if any).
FetchContent in some ways is designed to be a little bit "low level". It has a lot of machinery and customization points. If your project is super simple, you might find it useful to try out a CMake-external wrapper module called "CPM" (CMake Package Manager) that attempts to cater to sensible defaults for more common, simple use-cases.
If you use FetchContent or CPM, be aware that since they eventually just call add_subdirectory, you might need to take some steps to avoid naming conflicts of target names and CMake variable names between your CMake configs and the CMake configs of the dependencies you pull in. For more info, see How can I avoid clashes with targets "imported" with FetchContent_MakeAvailable?.
As others have mentioned, you can also look into using package managers like vcpkg or Conan. I don't know much about those so I can't comment.
I am working on a C++ project using CMake where I build an executable foo that uses a shared library libbar (that is being added via ExternalProject_add).
The executable build/src/foo in the build directory works perfectly fine. However, if I run make install, the installed executable /bin/foo gives me the following error.
./foo: error while loading shared libraries: libbar.so.11: cannot open shared object file: No such file or directory
I know I am not the only one with this problem (see e.g. here), and I am also aware of the handling of rpath by CMake, see here. As I understand, the install step strips the rpath variable, which explains that the library file cannot be found.
I checked this by running ldd foo in the directory /build/src/ resulting in
libbar.so => /PATH/TO/LIBBAR/libbar.so
When I run the same command in the directory /build/bin/, I get
libbar.so => not found
Now my question. How can I avoid in general that the executable "forgets" the location of the shared library during installation? I basically want the installed executable to have the same paths in rpath as the one in the build directory.
What I have tried so far
I read that you can avoid the stripping of the path via
SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
However, that does not work at all. I have no idea why not, as it is the precise solution suggested here and in the documentation.
I can set the path manually of course via
SET(CMAKE_INSTALL_RPATH "$LIBBAR_PATH}/lib")
and that does work, but this solution is too specific to libbar and does e.g. not work, if I import this project in another code that also uses libbar via my project.
EDIT
I should add that this problem does not appear on all machines. I get it on Linux machines, where it also says
-- Set runtime path of "/PATH/TO/foo" to ""
during the installation. I do not get that line on my Mac, where I don't have that problem at all.
EDIT 2
I just saw that my problem is even mentioned explicitly on the documentation under Common questions. They also say that adding set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) is the solution. But it simply does not work for me, what am I doing wrong here?
EDIT 3
Could it be that the CMAKE_INSTALL_RPATH_USE_LINK_PATH = True solution does not work here, because I am adding libbar via ExternalProject? The documentation states that
CMAKE_INSTALL_RPATH_USE_LINK_PATH is a boolean that if set to true will append directories in the linker search path and outside the project to the INSTALL_RPATH. This is used to initialize the target property INSTALL_RPATH_USE_LINK_PATH for all targets.
First of all, thanks for all the comments and answers. I didn't find a perfect and simple answer to my question, but I want to share what I am going with for now.
As I understand it, the stripping of the RPATH during installation is the desired behavior, see the documentation. In general, clearing of the RPATH cannot be simply deactivated. From other questions on stackoverflow, I see that there are situations where it can work to set
SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
but that did not work for me. I believe that is because in my specific setup the shared library is getting downloaded and build via ExternalProject_add. Therefore, the shared library is not from outside the project in this scenario, see here. (Please correct me if I am wrong on this.)
In the end, I don't see a better way than setting the install RPATH manually via
SET(CMAKE_INSTALL_RPATH "${LIBBAR_PATH}/lib")
If I want to import the code that uses libbar (let's call it Project_A) into another project Project_B, I will have to set the INSTALL_RPATH in the CMakeLists.txt of Project_B as well. Something along the lines of
SET(CMAKE_INSTALL_RPATH "${EXTERNAL_DIR}/Project_A/external/libbar/lib")
I don't think this is a fully satisfying soluation, but it is the best I could come up with. If anyone knows a better way, e.g. how to use the CMAKE_INSTALL_RPATH_USE_LINK_PATH option that applies to a shared library build as an ExternalProject in the same build tree, please feel free to show a better way.
Your problem is not that the build RPATH is deleted. Those RPATHs are absolute paths to your build directory and will not work once they leave your machine. You don't want them there, anyway. The problem is that you aren't setting the install RPATH correctly.
Place the following code early in your project, before any targets are created.
include(GNUInstallDirs)
if (APPLE)
set(rbase "#loader_path")
else ()
set(rbase "$ORIGIN")
endif ()
file(RELATIVE_PATH lib_dir
"${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_BINDIR}"
"${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}")
# Honor user overrides, whether at the command line or FetchContent
set(CMAKE_INSTALL_RPATH "${rbase};${rbase}/${libdir}"
CACHE STRING "Install RPATH")
This results in relocatable installation trees, and is sensitive to both GNU and Apple loader implementations that use different symbols for relative RPATHs.
This also assumes your install rules are standard and will install runtime objects to CMAKE_INSTALL_BINDIR and libraries to CMAKE_INSTALL_LIBDIR. Adjust these as needed.
CMake strips the rpath’s of targets installed as such:
install(
TARGETS
target_name
DESTINATION
...
)
To avoid that, install the library as file instead:
install(
FILES
$<TARGET_FILE:target_name>
DESTINATION
...
)
I'm creating a very small project that depends on the following library: https://github.com/CopernicaMarketingSoftware/AMQP-CPP
I'm doing what i always do with third-party libraries: i add their git repo as a submodule, and build them along with my code:
option(COOL_LIBRARY_OPTION ON)
add_subdirectory(deps/cool-library)
include_directories(deps/cool-library/include)
target_link_libraries(${PROJECT_NAME} coollib)
This has worked perfectly for libraries like Bullet, GLFW and others. However, this AMQP library does quite an ugly hack. Their include directory is called include, but in their CMake install() command, they rename it to amqpcpp. And their main header, deps/cool-library/amqpcpp.h, is referencing all other headers using that "fake" directory.
What happens is: when CMake tries to compile my sources which depend on deps/cool-library/amqpcpp.h, it fails because it's not finding deps/cool-library/amqpcpp/*.h, only deps/cool-library/include.
Does anyone have any idea how i can fix this without having to bundle the library into my codebase?
This is not how CMake is supposed to work.
CMake usually builds an entire distributive package of a library once and then installs it to some prefix path. It is then accessible for every other build process on the system by saying "find_package()". This command finds the installed distibution, and all the libs, includes etc. automagically. Whatever weird stuff library implementers did, the resulting distros are more or less alike.
So, in this case you do a lot of unnecessary work by adding includes manually. As you see it can also be unreliable.
What you can do is:
to still have all the dependencies source distributions in submodules (usually people don't bother doing this though)
build and install each dependency package into another (.gitignored) folder within the project or outside by using their own CMakeLists.txt. Let's say with a custom build step in your CMakeLists.txt
use "find_package()" in your CMakeLists.txt when build your application
Two small addition to Drop's answer: If the library set up their install routines correctly, you can use find_package directly on the library's binary tree, skipping the install step. This is mostly useful when you make changes to both the library and the dependent project, as you don't have to run the INSTALL target everytime to make library changes available downstream.
Also, check out the ExternalProject module of CMake which is very convenient for having external dependencies being built automatically as part of your project. The general idea is that you still pull in the library's source as a submodule, but instead of using add_subdirectory to pull the source into your project, you use ExternalProject_Add to build it on its own and then just link against it from your project.
I am writing a C++ library (header-only) and am using CMake to generate my (Visual Studio) project and solution files. I'm also writing a test suite, which is part of the same CMake project.
My problem occurs when I call target_include_directories() on the target that represents my header-only library, so that consumers of my library may find its header files. I get the following error message (even though generation is NOT aborted).
CMake Error in CMakeLists.txt:
Target "Fonts" INTERFACE_INCLUDE_DIRECTORIES property contains path:
"D:/Projects/GPC/fonts/include"
which is prefixed in the source directory.
(D:/Projects/GPC/Fonts being the top-level directory of my library project. Btw the problem remains if I move my header files to the top directory.)
The offending line in my CMakeLists.txt is this (adapted for simplicity):
target_include_directories(Fonts INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}/include")
I do not understand what I'm doing wrong. Without target_include_directories(), code of consumer projects simply can't include my header files (unless in installed form, but I haven't gotten to that yet, and in any case I want to be able to use my library from its build tree, without installation.)
I feel like I'm missing something basic here; yet I've searched for hours without finding a solution or explanation.
The origin of the problem is not the target_include_directories command itself, but the attempt to install a target that has a public or interface include directory prefixed in the source path (i.e. the include directory is a subdirectory of your ${PROJECT_SOURCE_DIR}.)
While it is perfectly fine and desirable to use absolute paths when building the library from scratch, a third party library that pulls in a prebuilt version of that library will probably want to use a different include path. After all, you do not want all of your users to mirror the directory structure of your build machine, just to end up in the right include path.
CMake's packaging mechanism provides support for both of these use cases: You may pull in a library directly from the build tree (that is, check out the source, build it, and point find_package() to the directory), or from an install directory (run make INSTALL to copy built stuff to the install directory and point find_package() to that directory). The latter approach needs to be relocatable (that is, I build and install on my machine, send you the resulting directory and you will be able to use it on your machine from a different directory structure), while the former is not.
This is a very neat feature, but you have to account for it when setting up the include directories. Quoting the manual for target_include_directories:
Include directories usage requirements commonly differ between the
build-tree and the install-tree. The BUILD_INTERFACE and
INSTALL_INTERFACE generator expressions can be used to describe
separate usage requirements based on the usage location. Relative
paths are allowed within the INSTALL_INTERFACE expression and are
interpreted relative to the installation prefix. For example:
target_include_directories(mylib PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include/mylib>
$<INSTALL_INTERFACE:include/mylib> # <prefix>/include/mylib
)
The BUILD_INTERFACE and INSTALL_INTERFACE generator expressions do all the magic:
$<INSTALL_INTERFACE:...>
Content of ... when the property is exported using install(EXPORT), and empty otherwise.
$<BUILD_INTERFACE:...>
Content of ... when the property is exported using export(), or when the target is used by another target in the same buildsystem.
Expands to the empty string otherwise.
I'm experimenting with CMake a bit for a C++ repository, but I'm running into some trouble trying to make it easy to build applications against libraries in the same source tree without a lot of extra CMake code.
The layout of the source tree is basically the following:
ROOT
libs/
lib1/
lib2/
lib3/
apps/
app1/
app2/
app3/
The libraries are independent of one another, and the applications may link against one or more of the libraries.
Currently I have a root CMakeLists.txt that lists out each application and library as a subdirectory so that if the library is changed and the application is rebuilt, so is the library. This works fine and CMake links it in without me having to specify where the library lives, but I don't see a way to do something similar for include directories.
Is there a common way to handle this? I'd prefer not to have each application's CMakeLists.txt have to manually list out the path to the libraries it needs.
If you are not afraid of making more headers available than you actually need for each application, you could list all lib directories in an INCLUDE_DIRECTORIES statement e.g. in the CMakeListst.txt adding all application sublists. But there is no such concept of managing "belonging" include folders per target built-in.
This question is pretty old. It was asked on 2012-07-18. I'm adding an answer here to attempt to explain some history. Warning: I may have gotten it wrong or misunderstood the history.
At that time, the latest non-release-candidate relase was CMake v2.8.8 (released on 2012-04-18). The INCLUDE_DIRECTORIES target property already existed in v2.8.8, but there was no corresponding INTERFACE_INCLUDE_DIRECTORIES target property yet. Its target_link_libraries documentation didn't say anything about a target automatically adding the include directories of the targets it depended upon.
About ten months after this question was asked (2012-07-18), CMake v.2.8.11 was released. It added the target_include_directories command signature with the INTERFACE|PUBLIC|PRIVATE argument, and added the INTERFACE_INCLUDE_DIRECTORIES target property.
So you can create a target B, specify public/interface include directories for it, and then make a target A depend on it via target_linK_libraries, and then A will automatically add/use the include directories of B. That can be made transitive by making the link public/interface, or non-transitive by making the link private.
You can find Kitware's list of software releases here, and a list of the documentation revisions here.