I am learning to use CMake and trying to understand where should I place files from different open-source libraries when I download them. I am talking about install location.
On linux include directory is this by convention: /usr/local/include/
What is default location for Windows and Mac OS?
I know that these locations can change, but what is the most common location I should place them?
Can something like this work?
install(FILES include/sort/sort.hpp DESTINATION ${CMAKE_INSTALL_PREFIX}/include/${CMAKE_PROJECT_NAME}/sort)
On windows usually the install prefix is something like C:/Program Files/<Package Name>, the usual install prefix on Unix being /usr/local. Usually this directory contains subdirectories like include containing the headers, lib containing (import) libraries, ect..
Using MSVC for the Windows targets those include directories are not automatically available to the compiler.
This is the reason why you should try to use find_package, if package configuration scripts or find scripts are provided by the installation which has the added benefit of also adding any dependencies with just 2 commands in your cmake project (find_package and target_link_libraries). If the package is installed in the default location, find_package should be able to locate the scripts to load without specifying the install location manually.
For packages not providing this kind of info, writing a find script of your own creating imported targets allows for easy reuse.
Can something like this work?
install(FILES include/sort/sort.hpp DESTINATION ${CMAKE_INSTALL_PREFIX}/include/${CMAKE_PROJECT_NAME}/sort)
In general you should avoid hardcoding absolute destination paths into your install paths. Instead use destination paths relative to the install prefix, since the install prefix may be overwritten, e.g. if the user runs the installation via cmake --install <build dir> --prefix <install directory> or when using cpack to generate a package.
Usually you place public headers of your library in a separate directory, even in your sources (which you seem to be doing) and install the directory with
install(DIRECTORY include/sort TYPE INCLUDE)
Resulting in cmake choosing the exact location based on its defaults for the include files for the target system.
Use a cross-platform package manager instead of manually managing the included directories. It handles the differences between different platforms.
vcpkg is one of the best package managers currently available. You can easily use it via project_options.
Related
For my project, I am building specific versions of the dependency libraries in a separate folder, say, /home/ubuntu/libs. I will use real libraries as an example, however, the question is pretty generic.
I was able to build the freetype library and make installed the headers into /home/ubuntu/libs/include, the built library into /home/ubuntu/libs/lib and also added the freetype-config.cmake to /home/ubuntu/libs/lib/cmake.
Now, I am trying to build the freetype-gl library that depends on freetype and has a line
find_package(freetype REQUIRED) in its CMakeLists.txt.
Typically, when I install the freetype library to a common path like /usr/local/lib or /usr/lib, cmake picks up the *-config.cmake files from the corresponding ./cmake directory. However, when I call it with
cmake -DCMAKE_TOOLCHAIN_FILE=/my/custom/toolchain -DCMAKE_LIBRARY_PATH=/home/ubuntu/libs/lib -DCMAKE_INCLUDE_PATH=/home/ubuntu/libs/include /path/to/freetype-gl
it fails with the following error
CMake Error at CMakeLists.txt:102 (find_package):
By not providing "Findfreetype.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "freetype",
but CMake did not find one.
Could not find a package configuration file provided by "freetype" with any
of the following names:
freetypeConfig.cmake
freetype-config.cmake
Add the installation prefix of "freetype" to CMAKE_PREFIX_PATH or set
"freetype_DIR" to a directory containing one of the above files. If
"freetype" provides a separate development package or SDK, be sure it has
been installed.
What am I doing wrong? How to show the place of "freetype-config.cmake" to cmake.
as you state in your second paragraph:
I was able to build the freetype library and make installed the headers into /home/ubuntu/libs/include, the built library into /home/ubuntu/libs/lib and also added the freetype-config.cmake to /home/ubuntu/libs/lib/cmake.
find_package() in CMake works in a way that it checks the standard system paths on default. If you are using CMake version 3.17 you can (instead of reading the documentation) view these paths simply by adding this line to your CMakeLists.txt
SET(CMAKE_FIND_DEBUG_MODE TRUE)
In your case what you need to do is point CMake in the right direction of the cmake file you are looking for. In your case that would be the /home/ubuntu/libs/lib/cmake. So somewhere in the top of your CMakeLists.txt (before you call find_package()) add this line:
LIST(APPEND CMAKE_MODULE_PATH "/home/ubuntu/libs/lib/cmake")
Provided that this bash command:
ls /home/ubuntu/libs/lib/cmake | grep "*.cmake"
Returns an occurance of freetypeConfig.cmake (you get the drill :) )
You can read about CMAKE_MODULE_PATH variable here: https://cmake.org/cmake/help/latest/variable/CMAKE_MODULE_PATH.html
In short your CMake doesn't find the config file because it is not in the standard path where it expects to find it.
EDIT: You can ofcourse do the same via these variables as the error suggests - CMAKE_PREFIX_PATH or freetype_DIR
Presumably you also don't want the libraries you build to find their dependencies installed on our system, if any.
I've spent some time investigating how to isolate CMake builds, and my best recipe is below.
First, the terminology:
An installation "prefix" of a library is a directory where the binaries and headers are installed (normally in include and lib subdirectories). I preder distinct prefixes for each library.
A "dependency" of a library is any of other library it needs when compiling.
CMake path separator - the character used to separate path lists for CMake. It's : character on Windows and ; on Linux (when cross-compiling, the host system matters, not the target).
Yes, it's the opposite of what the documentation claims.
By Windows I mean MSYS2. If you want to build outside of it, check that : is still the right separator.
Environment variables for CMake:
PKG_CONFIG_PATH to empty string
PKG_CONFIG_LIBDIR to a :-separated list, for each dependency add <prefix>/lib/pkgconfig and <prefix>/share/pkgconfig.
(I didn't need to do it for any library I used, so this is theoretical. But in any case don't leave this variable unset, at least use an empty string. Otherwise some undesired dependencies from your system might leak in.)
CMake flags:
-DCMAKE_INSTALL_PREFIX=... - the installation prefix for this library.
-DCMAKE_PREFIX_PATH=... - a path list: the installation prefix, followed by the prefixes of all dependencies. Use the CMake path separator, as described above. All paths here must be absolute.
-DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=OFF
This prevents CMake from finding some system-wide dependencies. If I remember correctly, not doing this makes CMake look for dependencies in directories listed in PATH and their parent directories, which is annoyting.
This has an undesired effect of disabling some ..._SYSTEM_... CMake variables, so we can't use those, even though some of them would be more appropriate.
-DCMAKE_STAGING_PREFIX=/. On Windows hosts replace / with the current drive name, e.g. C:.
This is only useful when cross-compiling, to disable the effects of CMAKE_FIND_ROOT_PATH in the toolchain file, which otherwise limits dependency search to that path.
This also messes up the installation path that would otherwise be taken from CMAKE_INSTALL_PREFIX, the fix is explained below.
Only on Windows hosts:
-DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF - otherwise CMake tends to look for dependencies in the PATH and parent directories, which is annoying. This is hardcoded to only happen on Windows hosts, which is nonsense and was reported here.
We don't want to set this unconditionally, because it has a side effect of requiring all your toolchain executables to be in the same directory, which is annoying in general, but IMO tolerable on Windows.
It also prevents CMake from searching for executables in PATH, so we also need...
-DCMAKE_PROGRAM_PATH= - set this to the contents of PATH, with the original separator replaced with the CMake path separator. At least on MSYS2 the separators are the same, so no modifications are needed.
-DCMAKE_MAKE_PROGRAM=ninja -GNinja - here CMAKE_MAKE_PROGRAM is strictly necessary on Windows hosts because of CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF, and doesn't hurt on other platforms. -G... is to make sure CMake doesn't pick a different generator that doesn't match CMAKE_MAKE_PROGRAM. You can use any other generator+program if you wish.
Then build with cmake --build <build_dir> -j<num_threads> as usual.
Install with cmake --install <build_dir> --prefix <prefix>. We need to explicitly set prefix because of CMAKE_STAGING_PREFIX=/.
I have downloaded a ROS2 demo from the examples repository.
Specifically, I have used minimal_subscriber and minimal_publisher.
I have ROS2 installed in /opt/ros2 and when I build these two examples with colcon build, it generates an install/ directory with lib/, shared/ and the other usual directory structure.
I can execute and run these demos perfectly fine with my current setup in my machine, but these executables link to libraries present in /opt/ros2, so when I want to execute them in another machine without ROS2 installed or I move my ROS2 installation in my machine, the executables cannot find the shared objects.
I've added a really simple script that adds all dependencies to install/lib when building but the executables don't seem to care, they aren't looking for shared libraries in the generated lib directory, they keep trying to search in /opt/ros2.
I believe this is something I should solve in CMake and it's not ROS2 specific, so, is there any way I can tell my generated executables to search in a diferent directory? (../lib or ./lib in my case)
If you are building them yourself (assumed since you mention CMake), you can set the RPATH in CMake (docs here). Specifically set the CMAKE_INSTALL_RPATH something like:
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
If you can't rebuild them, you can set LD_LIBRARY_PATH in your environment to where the libraries are located, or you can patch the executables themselves with an updated RPATH by using patchelf.
In order to get a relative RPATH rather than an absolute RPATH, use the $ORIGIN variable in your rpath spec. See "Recommendations" the the link above for more details.
I'm working on a project with a lot of external dependencies, which are included in the project tree. I would like to pre-build all of the dependencies and share the importable targets within the project tree.
I was planning to use cmake install with a CMAKE_INSTALL_PREFIX in the source tree, and use CMAKE_PREFIX_PATH to reference it for find_package. However, I'm beginning to wonder how maintainable this strategy is going to be? For example here's something I noticed in one of the installed cmake scripts:
${CMAKE_PREFIX_PATH}/lib/cmake/glfw3/glfw3Targets.cmake:
set_target_properties(glfw PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
INTERFACE_LINK_LIBRARIES "/usr/lib/x86_64-linux-gnu/librt.so;/usr/lib/x86_64-linux-gnu/libm.so;dl;/usr/lib/x86_64-linux-gnu/libX11.so;-lpthread"
)
It seems really suspicious to me that all of those link libraries are fully resolved to paths on the host machine.
I guess the question is: Are cmake installs to a prefix meant to be distributable and this is just a bad example, or are they meant to be tied to the machine you "install" them on? Ie. Is prefix really meant to just relocate where on the system things are supposed to be "installed", and my hope to use it as a shared package manager likely to be problematic?
Yes, EXPORT'ed CMake targets can be "distributable", but the project should follow some principles for achieve that.
If you link with a (external) library, but do not want export file to contain absolute path to it, then do not pass absolute path directly to target_link_libraries.
In case a linked library is shipped with a compiler (e.g. m or rt), things are simple: just pass the library's name to the target_link_libraries.
In case a linked library YYY comes from other package and is detected by find_package(YYY), this implies several things:
Script FindYYY.cmake or YYYConfig.cmake should return IMPORTED target. If this is not true, you may try to wrap its results into IMPORTED target.
target_link_libraries should use this IMPORTED target.
Script XXXConfig.cmake, shipped with your project, should use find_dependency(YYY) for discover a library on a user machine.
For find_dependency(YYY) work, a user machine should have FindYYY.cmake or YYYConfig.cmake script. Alternatively, you may ship FindYYY.cmake with your project, and adjust CMAKE_MODULE_PATH variable before find_dependency() call (in your XXXConfig.cmake).
I'm working on multiple projects at the same time (some libraries, and some games that depend on them). They're all on GitHub, in separate repos.
For convenience, I pull every repo in a "workspace folder", like this:
/home/myWorkspace/Library1/
/home/myWorkspace/Library2/
/home/myWorkspace/Library3/
/home/myWorkspace/App1/
/home/myWorkspace/App2/
I have FindLibrary1, FindLibrary2 and FindLibrary3 .cmake files, which, in order, look for the library in ../ (which corresponds to the "myWorkspace" folder), then /usr/lib/.
While on Windows CMake finds the libraries in myWorkspace/, on Linux, no matter what, installed libraries are always found first.
Since I'd like to work in myWorkspace/ folder, then installing the libraries after I'm done, I'd prefer CMake to find and link everything within the myWorkspace/ folder.
I'd also like CMake to search for the libraries in /usr/lib/ and /usr/local/lib/ if there is no myWorkspace/ folder, but if myWorkspace/ exists, it should have the priority.
Examples of CMake files I'm using:
SSVUtils: library with no dependancies
SSVUtils CMakeLists: https://github.com/SuperV1234/SSVUtils/blob/master/CMakeLists.txt
FindSSVUtils.cmake: https://github.com/SuperV1234/SSVUtils/blob/master/cmake/modules/FindSSVUtils.cmake
SSVUtilsJson: library that depends on SSVUtils and SSVJsonCpp
SSVUtilsJson CMakeLists: https://github.com/SuperV1234/SSVUtilsJson/blob/master/CMakeLists.txt
FindSSVUtilsJson.cmake: https://github.com/SuperV1234/SSVUtilsJson/blob/master/cmake/modules/FindSSVUtilsJson.cmake
Any ideas how I can prioritize the myWorkspace/ folder while still having the possibility to find libs in file system paths?
By default, find_path (or find_library, etc.) first checks for files in the system standard locations, before searching in the values provided in PATH. That's why "installed" libraries are always found first on Linux (but not on Windows, that doesn't have standard locations for installed libraries).
You can disable that behavior by using the NO_CMAKE_SYSTEM_PATH option: it will skip detection of files in standard locations.
Now... if you still want to use the installed libraries as a fallback when local versions are not found, you can do it in a two step process:
find_path(... NO_CMAKE_SYSTEM_PATH)
if (nothing_found)
find_path(...)
endif()
I'm refactoring a large platform independent c++ framework so that it's libraries and execuables no longer have to be in the same directory (or even repository) and it is proving quite challenging. The framework was currently used only by me and it should now address our whole working group so I have to keep it as modular and as automatized as possible.
My basic structure looks like this
apps/
app1, app2, ...
libs/
core, lib1, lib2, ...
bin:
app1, app2, libcore, liblib1, liblib2, ... (on NIX)
app1.exe, app2.exe, core.dll, lib1.dll, lib2.dll, ... (on Windows)
Apps depend on libs, and all libs depend on the core lib. This all works fine in the same root directory and with the add_subdirectory mechanism.
Project dependencies were handeled by the order in which I was calling add_subdirectory: first libs (core being the first), then apps. Cmake was kind enough to set ${core_SOURCE_IR} to the respective directory and all binaries (libs and apps) were generated in the same directory.
What I need advice with is:
should I move to a find_package based approach (write for each lib and FindLib.cmake file)
how the apps and libs will find each other
where to put the binaries
how to pass along include directories of dependent targets
ExternalProject_Add ?
Thank you
I'd recommend using INSTALL(TARGETS... along with INSTALL(EXPORT...
For full details, run:
cmake --help-command INSTALL
FIND_PACKAGE is generally used to find an external project which is already installed (and which wasn't installed with CMake's INSTALL(EXPORT... command) and ExternalProject_Add is used to download, configure, build and install an external project.
If you use INSTALL(EXPORT... with each of your libs and exes, and then just INCLUDE the installed <target>.cmake in your main CMakeLists.txt, these will become available as proper CMake targets, with their dependencies per configuration already set.
As to where to install these export files - that's up to you, but as long as the path you specify is the same place you use when you INCLUDE the file later, it should work fine.
First, you can add_subdirectory()es in any order without caring about dependencies. CMake handles them when all CMakeLists.txt are parsed.
Second, if you wish any your lib or app to be buildable stand-alone, you certainly should use something like find_package(). It can be find_library() and find_file(lib.h) for simple cases.
In this case after find_package() invocation you will have LIB_INCLUDE_DIRS, LIB_LIBRARY_DIRS, LIB_LIBRARIES variables defined, which can be passed to include_directories(), link_directories() and target_link_libraries() respectively. That's how apps will find libs.
As for where to put binaries - in *NIX it's wide practice to place them into ${CMAKE_INSTALL_PREFIX}/bin.
Please comment on this answer, if there is something unclear for you.