Custom library directory for cmake - c++

For my project, I am building specific versions of the dependency libraries in a separate folder, say, /home/ubuntu/libs. I will use real libraries as an example, however, the question is pretty generic.
I was able to build the freetype library and make installed the headers into /home/ubuntu/libs/include, the built library into /home/ubuntu/libs/lib and also added the freetype-config.cmake to /home/ubuntu/libs/lib/cmake.
Now, I am trying to build the freetype-gl library that depends on freetype and has a line
find_package(freetype REQUIRED) in its CMakeLists.txt.
Typically, when I install the freetype library to a common path like /usr/local/lib or /usr/lib, cmake picks up the *-config.cmake files from the corresponding ./cmake directory. However, when I call it with
cmake -DCMAKE_TOOLCHAIN_FILE=/my/custom/toolchain -DCMAKE_LIBRARY_PATH=/home/ubuntu/libs/lib -DCMAKE_INCLUDE_PATH=/home/ubuntu/libs/include /path/to/freetype-gl
it fails with the following error
CMake Error at CMakeLists.txt:102 (find_package):
By not providing "Findfreetype.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "freetype",
but CMake did not find one.
Could not find a package configuration file provided by "freetype" with any
of the following names:
freetypeConfig.cmake
freetype-config.cmake
Add the installation prefix of "freetype" to CMAKE_PREFIX_PATH or set
"freetype_DIR" to a directory containing one of the above files. If
"freetype" provides a separate development package or SDK, be sure it has
been installed.
What am I doing wrong? How to show the place of "freetype-config.cmake" to cmake.

as you state in your second paragraph:
I was able to build the freetype library and make installed the headers into /home/ubuntu/libs/include, the built library into /home/ubuntu/libs/lib and also added the freetype-config.cmake to /home/ubuntu/libs/lib/cmake.
find_package() in CMake works in a way that it checks the standard system paths on default. If you are using CMake version 3.17 you can (instead of reading the documentation) view these paths simply by adding this line to your CMakeLists.txt
SET(CMAKE_FIND_DEBUG_MODE TRUE)
In your case what you need to do is point CMake in the right direction of the cmake file you are looking for. In your case that would be the /home/ubuntu/libs/lib/cmake. So somewhere in the top of your CMakeLists.txt (before you call find_package()) add this line:
LIST(APPEND CMAKE_MODULE_PATH "/home/ubuntu/libs/lib/cmake")
Provided that this bash command:
ls /home/ubuntu/libs/lib/cmake | grep "*.cmake"
Returns an occurance of freetypeConfig.cmake (you get the drill :) )
You can read about CMAKE_MODULE_PATH variable here: https://cmake.org/cmake/help/latest/variable/CMAKE_MODULE_PATH.html
In short your CMake doesn't find the config file because it is not in the standard path where it expects to find it.
EDIT: You can ofcourse do the same via these variables as the error suggests - CMAKE_PREFIX_PATH or freetype_DIR

Presumably you also don't want the libraries you build to find their dependencies installed on our system, if any.
I've spent some time investigating how to isolate CMake builds, and my best recipe is below.
First, the terminology:
An installation "prefix" of a library is a directory where the binaries and headers are installed (normally in include and lib subdirectories). I preder distinct prefixes for each library.
A "dependency" of a library is any of other library it needs when compiling.
CMake path separator - the character used to separate path lists for CMake. It's : character on Windows and ; on Linux (when cross-compiling, the host system matters, not the target).
Yes, it's the opposite of what the documentation claims.
By Windows I mean MSYS2. If you want to build outside of it, check that : is still the right separator.
Environment variables for CMake:
PKG_CONFIG_PATH to empty string
PKG_CONFIG_LIBDIR to a :-separated list, for each dependency add <prefix>/lib/pkgconfig and <prefix>/share/pkgconfig.
(I didn't need to do it for any library I used, so this is theoretical. But in any case don't leave this variable unset, at least use an empty string. Otherwise some undesired dependencies from your system might leak in.)
CMake flags:
-DCMAKE_INSTALL_PREFIX=... - the installation prefix for this library.
-DCMAKE_PREFIX_PATH=... - a path list: the installation prefix, followed by the prefixes of all dependencies. Use the CMake path separator, as described above. All paths here must be absolute.
-DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=OFF
This prevents CMake from finding some system-wide dependencies. If I remember correctly, not doing this makes CMake look for dependencies in directories listed in PATH and their parent directories, which is annoyting.
This has an undesired effect of disabling some ..._SYSTEM_... CMake variables, so we can't use those, even though some of them would be more appropriate.
-DCMAKE_STAGING_PREFIX=/. On Windows hosts replace / with the current drive name, e.g. C:.
This is only useful when cross-compiling, to disable the effects of CMAKE_FIND_ROOT_PATH in the toolchain file, which otherwise limits dependency search to that path.
This also messes up the installation path that would otherwise be taken from CMAKE_INSTALL_PREFIX, the fix is explained below.
Only on Windows hosts:
-DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF - otherwise CMake tends to look for dependencies in the PATH and parent directories, which is annoying. This is hardcoded to only happen on Windows hosts, which is nonsense and was reported here.
We don't want to set this unconditionally, because it has a side effect of requiring all your toolchain executables to be in the same directory, which is annoying in general, but IMO tolerable on Windows.
It also prevents CMake from searching for executables in PATH, so we also need...
-DCMAKE_PROGRAM_PATH= - set this to the contents of PATH, with the original separator replaced with the CMake path separator. At least on MSYS2 the separators are the same, so no modifications are needed.
-DCMAKE_MAKE_PROGRAM=ninja -GNinja - here CMAKE_MAKE_PROGRAM is strictly necessary on Windows hosts because of CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF, and doesn't hurt on other platforms. -G... is to make sure CMake doesn't pick a different generator that doesn't match CMAKE_MAKE_PROGRAM. You can use any other generator+program if you wish.
Then build with cmake --build <build_dir> -j<num_threads> as usual.
Install with cmake --install <build_dir> --prefix <prefix>. We need to explicitly set prefix because of CMAKE_STAGING_PREFIX=/.

Related

CMake FindXercesC.cmake does not find my XercesC

On my system (Fedora 26), I installed XercesC using yum (aka dnf). The XercesC header files are located in
/usr/include/xercesc-2.7.0/xercesc
and the library is
/usr/lib64/libxerces-c.so.27.
I have tried the official FindXercesC.cmake as well as a number of posted versions of this package finder. I also attempted many edits to the posted *.cmake files. None of them can locate XercesC and I have to resort to manually entering the locations for the headers and library.
Is there a CMake package finder for XercesC that will automatically locate XercesC on my system?
Normally, CMake search things only in default directories. E.g., /usr/include directory is automatically searched for the header files.
But directory /usr/include/xercesc-2.7.0 isn't a default for CMake (it is not default for compiler too), so CMake cannot find things there without an explicit hint. For hint CMake about include directory to search, set CMAKE_INCLUDE_PATH variable. E.g., via command line:
cmake -DCMAKE_INCLUDE_PATH=/usr/include/xercesc-2.7.0 <other_params>
Similar is true for searching library files: CMake automatically searches libraries under /usr/lib64/, but not under /usr/lib64/xercesc-2.7.0. Also, CMake can find only library without so-version, so it cannot find file /usr/lib64/libxerces-c.so.27. For finding a file /usr/lib64/xercesc-2.7.0/libxerces-c.so you need to hint CMake with CMAKE_LIBRARY_PATH variable.
According to the xercesc sources, it supports searching the package via pkg-config and via CONFIG mode of find_package. Probably, these variants won't require additional hints.
Searching via pkg-config can be performed with pkg_check_modules, for use find_package in CONFIG mode either add this option to the call find_package(XercesC), or simply remove FindXercesC.cmake script.

Using CMake to find dependencies in an application-specific subfolder

In spite of many years of coding large-scale C++ applications, I do not understand how find_package is supposed to work in a medium-size CMake project, ASSUMING that I want to build the source to dependent packages myself and not simply rely on large systems like opencv, pcl or boost being installed somewhere in a system folder. I can't can't believe that I'm the only person in the world who has shipped multiple OpenCV and other open-source apps, has worked with meta-build systems like NAnt and SCons on major game projects, yet can't understand the most basic things about how CMake works or find a tutorial answering these questions.
In the past, I have essentially hacked around not understaning find_package by setting all the foo_DIR values by hand as CMake complains until I get a working folder.
I would like to run through a simple example which I'm working on right now, and dearly hope someone can explain what I'm doing so wrong.
Firstly, some assumptions:
I want to build everything for both MacOS and Windows, ideally via CMakeGUI. MacOS should build XCodeProjects and Windows should build Visual Studio Solutions.
Where there are dependencies, I want to compile them myself, so I have debug symbols and can modify the dependency source (or at least debug into it.)
No installation of pre-built binaries into system folders, i.e. no use of sudo port install opencv/pcl, etc on mac.
I have multiple projects, and prefer to keep a project and its dependencies in a single folder.
For the purposes of a concrete example, suppose I am building this project, although it's an arbitrary choice to illustrate the process and confusion I suffer:
https://github.com/krips89/opendetection
This lists dependencies, which I have intentionally reordered here so that I can take them in order, as follows:
find_package(OpenCV REQUIRED)
find_package(Eigen REQUIRED)
find_package(Boost 1.40 COMPONENTS program_options REQUIRED )
find_package(PCL REQUIRED)
find_package(VTK REQUIRED)
I would like to have all of these dependencies downloaded and configured in a single path (let's say c:\src on Windows, and ~\src on Mac for simplicity), NOT in a system path. Assume that the actual folder is a sub-folder for this project, and no a sub-folder for all projects. This should also allow for side-by-side installation of multiple projects on the same computer.
Taking this one step at a time:
(1) I clone openCV from https://github.com/opencv/opencv, sync to tag 3.1, configure into the folder opencv_build folder, build and install into opencv_install. I've done this so many times it's pretty straightforward.
(2) As above, but for eigen (although building for eigen doesn't actually do anything s it's a template library. I install to a folder eigen_install
Taking directory shows a series of folders for downloaded dependencies. I have assumed a convention where , and are source repos, and their following _build folders are the "WHere to build the binaries" folders in CMakeGui.
$ ls
boost_1_40_0 opencv opendetection_build
eigen opencv-build opendetection_data
eigen_build opencv_contrib pcl
eigen_install opendetection
All good so far, now let's try to configure opendetection and generate a solution into opendetection_build, and find pendetection's dependencies from within the ~/src folder, that is for the first two dependencies, I hope to find opencv and eigen in the opencv-build and eigen-build folders.
OpenCV immediately fails, as expected, saying:
Could not find a package configuration file provided by "OpenCV" with any of the following names:
OpenCVConfig.cmake
opencv-config.cmake
Add the installation prefix of "OpenCV" to CMAKE_PREFIX_PATH or set "OpenCV_DIR" to a directory containing one of the above files. If "OpenCV" provides a separate development package or SDK, be sure it has been installed.
That's good, because I want to explicitly tell CMake to look for dependent packages under my ~/src folder. Question: Is the use of CMAKE_PREFIX_PATH=/users/foo/src the recommended way to accomplish what I want - looking for all sub-packages under a specific path?
Following this, CMake finds OpenCV (good), and sets OpenCV_DIR = /Users/foo/src/opencv-build.
Question: Given that I have made an "install" to opencv-install (using CMAKE_INSTALL_PREFIX and building the Install Target Of OpenCV, shouldn't it find OpenCV in the opencv-install folder not opencv-build?
Moving on to eigen, I have configured and built eigen, and installed it to ~/src/eigen-install, which since it is a subfolder of CMAKE_PREFIX_PATH (~/src) I might expect to be found. But it doesn't seem to be. Can somebody explain to me what I'm not understanding? Particularly given that Eigen in a template library, and that there are at least three folders (eigen, eigen_build and eigen_install) under CMAKE_PREFIX_PATH which I would have thought CMake would find something in, I assume I must be doing something wrong here. I KNOW from past experience, I can set EIGEN_INCLUDE_DIR by hand in CMakeGUI by hand, and continue hacking forth, but that just seems wrong.
I'm more than willing to write up a web page explaining this for future people as dumb as me if one does not already exist, although I can't understand how use of CMake for basic project configuration and generation is apparently so obvious to everyone but so opaque for me. I have actually been using CMake for some years, usually by just manually setting Boost_INCLUDE_Dir, Foo_INCLUDE_PATH etc manually, but clearly this is not the right solution. Generally, after spending a couple of days fighting through the various packages to generate a solution by manually setting INCLUDE PATHS, LIBRARY PATHS and other options, I just deal with the solution and don't touch CMake again. But I would love to understand what I'm missing about find_package for my (surely not uncommon) use case of wanting to control my project dependencies rather than just using sudo port install * and installing random versions of projects to my global system folders.
As error message says, CMAKE_PREFIX_PATH should be set to installation prefix of the package. E.g., if the package has been built using CMake, this is CMAKE_INSTALL_PREFIX variable's value, if the package has been build using Autotools, this is value of --prefix option used for configure it, and so on.
CMake doesn't search every directory under CMAKE_PREFIX_PATH. That is why specifying it as /users/foo/src is useless if you have the package installed at /users/foo/src/eigen-install.
Instead, you may install all 3d-party packages into /users/foo/src/install, and use that path as CMAKE_PREFIX_PATH in your main project.

Possible causes for Boost not being found by CMake in certain situations?

I build a C++ project depending on the Boost library using CMake (3.4.1). Host platform is Linux, targets are that host and cross-build Android NDK.
I'm only using Boost header files and I just downloaded/extracted the boost folder (and I don't have a /usr/include/boost directory).
In my CMakeLists.txt file I declare the dependency to Boost like this:
find_package(Boost 1.57 REQUIRED)
And I configure my build like this:
BOOST_ROOT=/path/to/boost cmake ../src
Which actually works as expected for my native build.
When I now configure a build exactly the same way (only specifying some more environment variables and a CMAKE_TOOLCHAIN_FILE) CMake gives me:
BOOST_ROOT=/path/to/boost JAVA_HOME=/bla/bla/bla \
ANDROID_NDK=/bla/bla/bla \
ANDROID_SDK=/bla/bla/bla \
ANT=/usr/bin/ant \
cmake ../src -DCMAKE_TOOLCHAIN_FILE=/bla/bla/android.toolchain.cmake
CMake Error at /usr/share/cmake/Modules/FindBoost.cmake:1247 (message):
Unable to find the requested Boost libraries.
Unable to find the Boost header files. Please set BOOST_ROOT to the root
directory containing Boost or BOOST_INCLUDEDIR to the directory containing
Boost's headers.
Call Stack (most recent call first):
CMakeLists.txt:4 (find_package)
So I believe I did almost the same to build for the Android target but the very same method that finds Boost for the host-build doesn't work here.
I tried to set Boost_DIR, BOOSTROOT and BOOST_INCLUDEDIR all with the same effect. Also I've deleted all content in the build directory before trying anything new.
What can be possible reasons for this behavior? I've already tried to print BOOST_ROOT directly in the FindBoost.cmake script like this:
message("BOOST_ROOT: $ENV{BOOST_ROOT}")
With the expected behavior (writing BOOST_ROOT: /path/to/boost).
Of course I can cheat now and just link the boost folder into the include folder of the cross compiler but that's not nice of course and I want to find out what's going on.
When cross-compiling, the toolchain file normally sets the variable CMAKE_FIND_ROOT_PATH. Combined with the CMAKE_FIND_ROOT_PATH_MODE_LIBRARY variable set to ONLY, CMAKE_FIND_ROOT_PATH variable is used as effective chroot for find_library calls, so only libraries under the given prefix(es) are searched.
Analogue variables exist to adjust the behavior for find_path (used for searching include paths) and find_program.
THe toolchain file you use actually sets CMAKE_FIND_ROOT_PATH at line 1521:
set( CMAKE_FIND_ROOT_PATH "${ANDROID_TOOLCHAIN_ROOT}/bin"
"${ANDROID_TOOLCHAIN_ROOT}/${ANDROID_TOOLCHAIN_MACHINE_NAME}"
"${ANDROID_SYSROOT}"
"${CMAKE_INSTALL_PREFIX}"
"${CMAKE_INSTALL_PREFIX}/share" )
and below sets CMAKE_FIND_ROOT_PATH_MODE_* variables to ONLY. So you need to have Boost installed under one of these directories, and give hints (like BOOST_ROOT) relative to it.
Note, that Boost should be built for the target platform (Android NDK in you case), not for the platform where you cross-compile (Linux).

CMake: target_include_directories() prints an error when I try to add the source directory itself, or one of its subdirectories

I am writing a C++ library (header-only) and am using CMake to generate my (Visual Studio) project and solution files. I'm also writing a test suite, which is part of the same CMake project.
My problem occurs when I call target_include_directories() on the target that represents my header-only library, so that consumers of my library may find its header files. I get the following error message (even though generation is NOT aborted).
CMake Error in CMakeLists.txt:
Target "Fonts" INTERFACE_INCLUDE_DIRECTORIES property contains path:
"D:/Projects/GPC/fonts/include"
which is prefixed in the source directory.
(D:/Projects/GPC/Fonts being the top-level directory of my library project. Btw the problem remains if I move my header files to the top directory.)
The offending line in my CMakeLists.txt is this (adapted for simplicity):
target_include_directories(Fonts INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}/include")
I do not understand what I'm doing wrong. Without target_include_directories(), code of consumer projects simply can't include my header files (unless in installed form, but I haven't gotten to that yet, and in any case I want to be able to use my library from its build tree, without installation.)
I feel like I'm missing something basic here; yet I've searched for hours without finding a solution or explanation.
The origin of the problem is not the target_include_directories command itself, but the attempt to install a target that has a public or interface include directory prefixed in the source path (i.e. the include directory is a subdirectory of your ${PROJECT_SOURCE_DIR}.)
While it is perfectly fine and desirable to use absolute paths when building the library from scratch, a third party library that pulls in a prebuilt version of that library will probably want to use a different include path. After all, you do not want all of your users to mirror the directory structure of your build machine, just to end up in the right include path.
CMake's packaging mechanism provides support for both of these use cases: You may pull in a library directly from the build tree (that is, check out the source, build it, and point find_package() to the directory), or from an install directory (run make INSTALL to copy built stuff to the install directory and point find_package() to that directory). The latter approach needs to be relocatable (that is, I build and install on my machine, send you the resulting directory and you will be able to use it on your machine from a different directory structure), while the former is not.
This is a very neat feature, but you have to account for it when setting up the include directories. Quoting the manual for target_include_directories:
Include directories usage requirements commonly differ between the
build-tree and the install-tree. The BUILD_INTERFACE and
INSTALL_INTERFACE generator expressions can be used to describe
separate usage requirements based on the usage location. Relative
paths are allowed within the INSTALL_INTERFACE expression and are
interpreted relative to the installation prefix. For example:
target_include_directories(mylib PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include/mylib>
$<INSTALL_INTERFACE:include/mylib> # <prefix>/include/mylib
)
The BUILD_INTERFACE and INSTALL_INTERFACE generator expressions do all the magic:
$<INSTALL_INTERFACE:...>
Content of ... when the property is exported using install(EXPORT), and empty otherwise.
$<BUILD_INTERFACE:...>
Content of ... when the property is exported using export(), or when the target is used by another target in the same buildsystem.
Expands to the empty string otherwise.

cmake find modules wrt version

I have a question regarding CMake: recently a lot of modules have been added to the standard installation, such as GLEW or Mercurial.
However a lot of installation base might have an older version without all the new modules available, so you are forced to ship your own version of (eg) FindGLEW.cmake
Is it possible to check whether a given FindXXX module is available and in that case use it, otherwise supply a proper alternative? Or even check cmake version at runtime (but that's not always reliable and a pain to maintain)...?
Thanks for any help.
The default search behaviour of CMake's include command w.r.t modules is:
the file with name <modulename>.cmake is searched first in CMAKE_MODULE_PATH, then in the CMake module directory.
(there's a bit more to it - see the docs or run cmake --help-command include)
What you're asking for appears to be the opposite to this; first check for an official module in the CMake module directory, and if none exists, fall back to use your own.
Assuming your modules are in ${CMAKE_SOURCE_DIR}/my_cmake_modules, you could reverse CMake's default behaviour by doing:
set(CMAKE_MODULE_PATH ${CMAKE_ROOT}/Modules ${CMAKE_SOURCE_DIR}/my_cmake_modules)
If you name your own modules exactly as per the official ones, then this should achieve your goal. If you wish to revert to the normal CMake behaviour later in your script, you can do:
set(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/my_cmake_modules)
as a possible alternative to the solution above, I found this cmake policy
CMP0017
Prefer files from the CMake module directory when including from
there.
Starting with CMake 2.8.4, if a cmake-module shipped with CMake (i.e.
located in the CMake module directory) calls include() or
find_package(), the files located in the the CMake module directory
are preferred over the files in CMAKE_MODULE_PATH. This makes sure
that the modules belonging to CMake always get those files included
which they expect, and against which they were developed and tested.
In call other cases, the files found in CMAKE_MODULE_PATH still take
precedence over the ones in the CMake module directory. The OLD
behaviour is to always prefer files from CMAKE_MODULE_PATH over files
from the CMake modules directory.
This policy was introduced in CMake version 2.8.4. CMake version
2.8.9 warns when the policy is not set and uses OLD behavior. Use the
cmake_policy command to set it to OLD or NEW explicitly.
Hope this helps.