cmake find modules wrt version - build

I have a question regarding CMake: recently a lot of modules have been added to the standard installation, such as GLEW or Mercurial.
However a lot of installation base might have an older version without all the new modules available, so you are forced to ship your own version of (eg) FindGLEW.cmake
Is it possible to check whether a given FindXXX module is available and in that case use it, otherwise supply a proper alternative? Or even check cmake version at runtime (but that's not always reliable and a pain to maintain)...?
Thanks for any help.

The default search behaviour of CMake's include command w.r.t modules is:
the file with name <modulename>.cmake is searched first in CMAKE_MODULE_PATH, then in the CMake module directory.
(there's a bit more to it - see the docs or run cmake --help-command include)
What you're asking for appears to be the opposite to this; first check for an official module in the CMake module directory, and if none exists, fall back to use your own.
Assuming your modules are in ${CMAKE_SOURCE_DIR}/my_cmake_modules, you could reverse CMake's default behaviour by doing:
set(CMAKE_MODULE_PATH ${CMAKE_ROOT}/Modules ${CMAKE_SOURCE_DIR}/my_cmake_modules)
If you name your own modules exactly as per the official ones, then this should achieve your goal. If you wish to revert to the normal CMake behaviour later in your script, you can do:
set(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/my_cmake_modules)

as a possible alternative to the solution above, I found this cmake policy
CMP0017
Prefer files from the CMake module directory when including from
there.
Starting with CMake 2.8.4, if a cmake-module shipped with CMake (i.e.
located in the CMake module directory) calls include() or
find_package(), the files located in the the CMake module directory
are preferred over the files in CMAKE_MODULE_PATH. This makes sure
that the modules belonging to CMake always get those files included
which they expect, and against which they were developed and tested.
In call other cases, the files found in CMAKE_MODULE_PATH still take
precedence over the ones in the CMake module directory. The OLD
behaviour is to always prefer files from CMAKE_MODULE_PATH over files
from the CMake modules directory.
This policy was introduced in CMake version 2.8.4. CMake version
2.8.9 warns when the policy is not set and uses OLD behavior. Use the
cmake_policy command to set it to OLD or NEW explicitly.
Hope this helps.

Related

Custom library directory for cmake

For my project, I am building specific versions of the dependency libraries in a separate folder, say, /home/ubuntu/libs. I will use real libraries as an example, however, the question is pretty generic.
I was able to build the freetype library and make installed the headers into /home/ubuntu/libs/include, the built library into /home/ubuntu/libs/lib and also added the freetype-config.cmake to /home/ubuntu/libs/lib/cmake.
Now, I am trying to build the freetype-gl library that depends on freetype and has a line
find_package(freetype REQUIRED) in its CMakeLists.txt.
Typically, when I install the freetype library to a common path like /usr/local/lib or /usr/lib, cmake picks up the *-config.cmake files from the corresponding ./cmake directory. However, when I call it with
cmake -DCMAKE_TOOLCHAIN_FILE=/my/custom/toolchain -DCMAKE_LIBRARY_PATH=/home/ubuntu/libs/lib -DCMAKE_INCLUDE_PATH=/home/ubuntu/libs/include /path/to/freetype-gl
it fails with the following error
CMake Error at CMakeLists.txt:102 (find_package):
By not providing "Findfreetype.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "freetype",
but CMake did not find one.
Could not find a package configuration file provided by "freetype" with any
of the following names:
freetypeConfig.cmake
freetype-config.cmake
Add the installation prefix of "freetype" to CMAKE_PREFIX_PATH or set
"freetype_DIR" to a directory containing one of the above files. If
"freetype" provides a separate development package or SDK, be sure it has
been installed.
What am I doing wrong? How to show the place of "freetype-config.cmake" to cmake.
as you state in your second paragraph:
I was able to build the freetype library and make installed the headers into /home/ubuntu/libs/include, the built library into /home/ubuntu/libs/lib and also added the freetype-config.cmake to /home/ubuntu/libs/lib/cmake.
find_package() in CMake works in a way that it checks the standard system paths on default. If you are using CMake version 3.17 you can (instead of reading the documentation) view these paths simply by adding this line to your CMakeLists.txt
SET(CMAKE_FIND_DEBUG_MODE TRUE)
In your case what you need to do is point CMake in the right direction of the cmake file you are looking for. In your case that would be the /home/ubuntu/libs/lib/cmake. So somewhere in the top of your CMakeLists.txt (before you call find_package()) add this line:
LIST(APPEND CMAKE_MODULE_PATH "/home/ubuntu/libs/lib/cmake")
Provided that this bash command:
ls /home/ubuntu/libs/lib/cmake | grep "*.cmake"
Returns an occurance of freetypeConfig.cmake (you get the drill :) )
You can read about CMAKE_MODULE_PATH variable here: https://cmake.org/cmake/help/latest/variable/CMAKE_MODULE_PATH.html
In short your CMake doesn't find the config file because it is not in the standard path where it expects to find it.
EDIT: You can ofcourse do the same via these variables as the error suggests - CMAKE_PREFIX_PATH or freetype_DIR
Presumably you also don't want the libraries you build to find their dependencies installed on our system, if any.
I've spent some time investigating how to isolate CMake builds, and my best recipe is below.
First, the terminology:
An installation "prefix" of a library is a directory where the binaries and headers are installed (normally in include and lib subdirectories). I preder distinct prefixes for each library.
A "dependency" of a library is any of other library it needs when compiling.
CMake path separator - the character used to separate path lists for CMake. It's : character on Windows and ; on Linux (when cross-compiling, the host system matters, not the target).
Yes, it's the opposite of what the documentation claims.
By Windows I mean MSYS2. If you want to build outside of it, check that : is still the right separator.
Environment variables for CMake:
PKG_CONFIG_PATH to empty string
PKG_CONFIG_LIBDIR to a :-separated list, for each dependency add <prefix>/lib/pkgconfig and <prefix>/share/pkgconfig.
(I didn't need to do it for any library I used, so this is theoretical. But in any case don't leave this variable unset, at least use an empty string. Otherwise some undesired dependencies from your system might leak in.)
CMake flags:
-DCMAKE_INSTALL_PREFIX=... - the installation prefix for this library.
-DCMAKE_PREFIX_PATH=... - a path list: the installation prefix, followed by the prefixes of all dependencies. Use the CMake path separator, as described above. All paths here must be absolute.
-DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=OFF
This prevents CMake from finding some system-wide dependencies. If I remember correctly, not doing this makes CMake look for dependencies in directories listed in PATH and their parent directories, which is annoyting.
This has an undesired effect of disabling some ..._SYSTEM_... CMake variables, so we can't use those, even though some of them would be more appropriate.
-DCMAKE_STAGING_PREFIX=/. On Windows hosts replace / with the current drive name, e.g. C:.
This is only useful when cross-compiling, to disable the effects of CMAKE_FIND_ROOT_PATH in the toolchain file, which otherwise limits dependency search to that path.
This also messes up the installation path that would otherwise be taken from CMAKE_INSTALL_PREFIX, the fix is explained below.
Only on Windows hosts:
-DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF - otherwise CMake tends to look for dependencies in the PATH and parent directories, which is annoying. This is hardcoded to only happen on Windows hosts, which is nonsense and was reported here.
We don't want to set this unconditionally, because it has a side effect of requiring all your toolchain executables to be in the same directory, which is annoying in general, but IMO tolerable on Windows.
It also prevents CMake from searching for executables in PATH, so we also need...
-DCMAKE_PROGRAM_PATH= - set this to the contents of PATH, with the original separator replaced with the CMake path separator. At least on MSYS2 the separators are the same, so no modifications are needed.
-DCMAKE_MAKE_PROGRAM=ninja -GNinja - here CMAKE_MAKE_PROGRAM is strictly necessary on Windows hosts because of CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF, and doesn't hurt on other platforms. -G... is to make sure CMake doesn't pick a different generator that doesn't match CMAKE_MAKE_PROGRAM. You can use any other generator+program if you wish.
Then build with cmake --build <build_dir> -j<num_threads> as usual.
Install with cmake --install <build_dir> --prefix <prefix>. We need to explicitly set prefix because of CMAKE_STAGING_PREFIX=/.

Are exported (installed) cmake targets distributable?

I'm working on a project with a lot of external dependencies, which are included in the project tree. I would like to pre-build all of the dependencies and share the importable targets within the project tree.
I was planning to use cmake install with a CMAKE_INSTALL_PREFIX in the source tree, and use CMAKE_PREFIX_PATH to reference it for find_package. However, I'm beginning to wonder how maintainable this strategy is going to be? For example here's something I noticed in one of the installed cmake scripts:
${CMAKE_PREFIX_PATH}/lib/cmake/glfw3/glfw3Targets.cmake:
set_target_properties(glfw PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
INTERFACE_LINK_LIBRARIES "/usr/lib/x86_64-linux-gnu/librt.so;/usr/lib/x86_64-linux-gnu/libm.so;dl;/usr/lib/x86_64-linux-gnu/libX11.so;-lpthread"
)
It seems really suspicious to me that all of those link libraries are fully resolved to paths on the host machine.
I guess the question is: Are cmake installs to a prefix meant to be distributable and this is just a bad example, or are they meant to be tied to the machine you "install" them on? Ie. Is prefix really meant to just relocate where on the system things are supposed to be "installed", and my hope to use it as a shared package manager likely to be problematic?
Yes, EXPORT'ed CMake targets can be "distributable", but the project should follow some principles for achieve that.
If you link with a (external) library, but do not want export file to contain absolute path to it, then do not pass absolute path directly to target_link_libraries.
In case a linked library is shipped with a compiler (e.g. m or rt), things are simple: just pass the library's name to the target_link_libraries.
In case a linked library YYY comes from other package and is detected by find_package(YYY), this implies several things:
Script FindYYY.cmake or YYYConfig.cmake should return IMPORTED target. If this is not true, you may try to wrap its results into IMPORTED target.
target_link_libraries should use this IMPORTED target.
Script XXXConfig.cmake, shipped with your project, should use find_dependency(YYY) for discover a library on a user machine.
For find_dependency(YYY) work, a user machine should have FindYYY.cmake or YYYConfig.cmake script. Alternatively, you may ship FindYYY.cmake with your project, and adjust CMAKE_MODULE_PATH variable before find_dependency() call (in your XXXConfig.cmake).

How to know variable such as 'OpenCV' in CMake

I am using OpenCV with gcc and cmake. And I found a tutorial https://docs.opencv.org/3.4.0/db/df5/tutorial_linux_gcc_cmake.html .In the file CMakeLists.txt, there are some variables such as OpenCV and OpenCV_INCLUDE_DIRS.
cmake_minimum_required(VERSION 3.9)
project(VideoRecord)
set(CMAKE_CXX_STANDARD 11)
find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
add_executable(VideoRecord main.cpp)
target_link_libraries(VideoRecord ${OpenCV_LIBS})
I want to know where to find these variables definition.
EDIT
Thanks #qbranchmaster's answer. I tried to search FindOpenCV.cmake but failed.
First try.
➜ ~ cmake --help-module-list | grep "FindOpen"
FindOpenACC
FindOpenAL
FindOpenCL
FindOpenGL
FindOpenMP
FindOpenSSL
FindOpenSceneGraph
FindOpenThreads
Another try.
➜ / find . "FindOpenCV.cmake"
In addition, my os is osx and I install cmake with brew. I comiple and install OpenCV manually.
These variables are part of the package config script shipping with OpenCV.
Note that find_package is a two-headed beast. The classic mode of operation is finding libraries through find-scripts. This is still the approach being used today for third-party libraries that are not aware of CMake. However, if your dependency is itself being built with CMake, it can provide a package config file instead, which allows for a more powerful mode of operation.
The idea here is that instead of you telling CMake how to find a dependency, the dependency itself tells CMake how clients can find it. This is the approach that is taken by libraries like OpenCV and Qt.
To answer your question, those variables are being set by the package config file in your local OpenCV installation, the template of which can be found in the OpenCV source code under cmake/templates/OpenCVConfig.cmake.in.
They are defined in CMake OpenCV module. CMake has numerous modules that aid in finding various libraries like OpenCV (FindOpenCV.cmake module).
Using this command you can get a list of modules that your CMake supports:
cmake --help-module-list
Some libraries come with their own *.cmake modules which should be installed in some system path. If you are using Ubuntu, your cmake modules should be localised in:
/usr/share/cmake/Modules/
If not, just search system for file FindOpenCV.cmake. In that file you will find these variables.
In general, you get variable names from the documentation or source code of the package you want to find.
Often you can derive the name to put into find_package from the provided FindFoo.cmake module file name, because "Foo" would be the name. The find module is either part of CMake or comes with the third-party library.
If there is no find module, some modules provide FooConfig.cmake files, where "Foo" is again the string to put into find_package.
If you have neither a find nor a config file, you need to find the library by other means, e.g., FindPkgConfig or find_library / find_file.

CMake FindXercesC.cmake does not find my XercesC

On my system (Fedora 26), I installed XercesC using yum (aka dnf). The XercesC header files are located in
/usr/include/xercesc-2.7.0/xercesc
and the library is
/usr/lib64/libxerces-c.so.27.
I have tried the official FindXercesC.cmake as well as a number of posted versions of this package finder. I also attempted many edits to the posted *.cmake files. None of them can locate XercesC and I have to resort to manually entering the locations for the headers and library.
Is there a CMake package finder for XercesC that will automatically locate XercesC on my system?
Normally, CMake search things only in default directories. E.g., /usr/include directory is automatically searched for the header files.
But directory /usr/include/xercesc-2.7.0 isn't a default for CMake (it is not default for compiler too), so CMake cannot find things there without an explicit hint. For hint CMake about include directory to search, set CMAKE_INCLUDE_PATH variable. E.g., via command line:
cmake -DCMAKE_INCLUDE_PATH=/usr/include/xercesc-2.7.0 <other_params>
Similar is true for searching library files: CMake automatically searches libraries under /usr/lib64/, but not under /usr/lib64/xercesc-2.7.0. Also, CMake can find only library without so-version, so it cannot find file /usr/lib64/libxerces-c.so.27. For finding a file /usr/lib64/xercesc-2.7.0/libxerces-c.so you need to hint CMake with CMAKE_LIBRARY_PATH variable.
According to the xercesc sources, it supports searching the package via pkg-config and via CONFIG mode of find_package. Probably, these variants won't require additional hints.
Searching via pkg-config can be performed with pkg_check_modules, for use find_package in CONFIG mode either add this option to the call find_package(XercesC), or simply remove FindXercesC.cmake script.

Using CMake to find dependencies in an application-specific subfolder

In spite of many years of coding large-scale C++ applications, I do not understand how find_package is supposed to work in a medium-size CMake project, ASSUMING that I want to build the source to dependent packages myself and not simply rely on large systems like opencv, pcl or boost being installed somewhere in a system folder. I can't can't believe that I'm the only person in the world who has shipped multiple OpenCV and other open-source apps, has worked with meta-build systems like NAnt and SCons on major game projects, yet can't understand the most basic things about how CMake works or find a tutorial answering these questions.
In the past, I have essentially hacked around not understaning find_package by setting all the foo_DIR values by hand as CMake complains until I get a working folder.
I would like to run through a simple example which I'm working on right now, and dearly hope someone can explain what I'm doing so wrong.
Firstly, some assumptions:
I want to build everything for both MacOS and Windows, ideally via CMakeGUI. MacOS should build XCodeProjects and Windows should build Visual Studio Solutions.
Where there are dependencies, I want to compile them myself, so I have debug symbols and can modify the dependency source (or at least debug into it.)
No installation of pre-built binaries into system folders, i.e. no use of sudo port install opencv/pcl, etc on mac.
I have multiple projects, and prefer to keep a project and its dependencies in a single folder.
For the purposes of a concrete example, suppose I am building this project, although it's an arbitrary choice to illustrate the process and confusion I suffer:
https://github.com/krips89/opendetection
This lists dependencies, which I have intentionally reordered here so that I can take them in order, as follows:
find_package(OpenCV REQUIRED)
find_package(Eigen REQUIRED)
find_package(Boost 1.40 COMPONENTS program_options REQUIRED )
find_package(PCL REQUIRED)
find_package(VTK REQUIRED)
I would like to have all of these dependencies downloaded and configured in a single path (let's say c:\src on Windows, and ~\src on Mac for simplicity), NOT in a system path. Assume that the actual folder is a sub-folder for this project, and no a sub-folder for all projects. This should also allow for side-by-side installation of multiple projects on the same computer.
Taking this one step at a time:
(1) I clone openCV from https://github.com/opencv/opencv, sync to tag 3.1, configure into the folder opencv_build folder, build and install into opencv_install. I've done this so many times it's pretty straightforward.
(2) As above, but for eigen (although building for eigen doesn't actually do anything s it's a template library. I install to a folder eigen_install
Taking directory shows a series of folders for downloaded dependencies. I have assumed a convention where , and are source repos, and their following _build folders are the "WHere to build the binaries" folders in CMakeGui.
$ ls
boost_1_40_0 opencv opendetection_build
eigen opencv-build opendetection_data
eigen_build opencv_contrib pcl
eigen_install opendetection
All good so far, now let's try to configure opendetection and generate a solution into opendetection_build, and find pendetection's dependencies from within the ~/src folder, that is for the first two dependencies, I hope to find opencv and eigen in the opencv-build and eigen-build folders.
OpenCV immediately fails, as expected, saying:
Could not find a package configuration file provided by "OpenCV" with any of the following names:
OpenCVConfig.cmake
opencv-config.cmake
Add the installation prefix of "OpenCV" to CMAKE_PREFIX_PATH or set "OpenCV_DIR" to a directory containing one of the above files. If "OpenCV" provides a separate development package or SDK, be sure it has been installed.
That's good, because I want to explicitly tell CMake to look for dependent packages under my ~/src folder. Question: Is the use of CMAKE_PREFIX_PATH=/users/foo/src the recommended way to accomplish what I want - looking for all sub-packages under a specific path?
Following this, CMake finds OpenCV (good), and sets OpenCV_DIR = /Users/foo/src/opencv-build.
Question: Given that I have made an "install" to opencv-install (using CMAKE_INSTALL_PREFIX and building the Install Target Of OpenCV, shouldn't it find OpenCV in the opencv-install folder not opencv-build?
Moving on to eigen, I have configured and built eigen, and installed it to ~/src/eigen-install, which since it is a subfolder of CMAKE_PREFIX_PATH (~/src) I might expect to be found. But it doesn't seem to be. Can somebody explain to me what I'm not understanding? Particularly given that Eigen in a template library, and that there are at least three folders (eigen, eigen_build and eigen_install) under CMAKE_PREFIX_PATH which I would have thought CMake would find something in, I assume I must be doing something wrong here. I KNOW from past experience, I can set EIGEN_INCLUDE_DIR by hand in CMakeGUI by hand, and continue hacking forth, but that just seems wrong.
I'm more than willing to write up a web page explaining this for future people as dumb as me if one does not already exist, although I can't understand how use of CMake for basic project configuration and generation is apparently so obvious to everyone but so opaque for me. I have actually been using CMake for some years, usually by just manually setting Boost_INCLUDE_Dir, Foo_INCLUDE_PATH etc manually, but clearly this is not the right solution. Generally, after spending a couple of days fighting through the various packages to generate a solution by manually setting INCLUDE PATHS, LIBRARY PATHS and other options, I just deal with the solution and don't touch CMake again. But I would love to understand what I'm missing about find_package for my (surely not uncommon) use case of wanting to control my project dependencies rather than just using sudo port install * and installing random versions of projects to my global system folders.
As error message says, CMAKE_PREFIX_PATH should be set to installation prefix of the package. E.g., if the package has been built using CMake, this is CMAKE_INSTALL_PREFIX variable's value, if the package has been build using Autotools, this is value of --prefix option used for configure it, and so on.
CMake doesn't search every directory under CMAKE_PREFIX_PATH. That is why specifying it as /users/foo/src is useless if you have the package installed at /users/foo/src/eigen-install.
Instead, you may install all 3d-party packages into /users/foo/src/install, and use that path as CMAKE_PREFIX_PATH in your main project.