I am making a CMake library around some installable SDK. So the dependency tree looks like:
Application --> MyLibrary --> OfficialSDK
This SDK is installed by some setup.exe and does not have a CMake module.
So instead I include a custom find script inside MyLibrary: MyLibrary/cmake/FindOfficialSDK.cmake. Then inside the CMakeLists.txt of MyLibrary I can use find_package(OfficialSDK).
This works well for MyLibrary. I can build it and install it, together with the CMake export. So then Application can run find_package(MyLibrary) out of the box, since MyLibrary was installed properly using CMake.
However, when configuring Application I get an error:
Target "Application" links to target "OfficialSDK" but the target was not found.
Okay, so MyLibrary remembers it needs OfficialSDK, but it cannot find it in this CMake project.
I could solve this by including cmake/FindOfficialSDK.cmake in Application, but I would rather not make my users to copy the find script in case I need to update it in the future.
Is there some way of including the imported target OfficialSDK and install it together with MyLibrary, so Application doesn't need to search for it?
I found a solution, largely based on https://discourse.cmake.org/t/install-findpackage-script/5307.
Another example I found inside Pagmo2, for a custom FindBoost script: https://github.com/esa/pagmo2/blob/master/pagmo-config.cmake.in#L10
In a nutshell, I've added the following:
Added an install for my custom find script: install(FILE ${CMAKE_CURRENT_DIR}/cmake/FindOfficialSDK ...)
Added a custom *-config.cmake.in script that will be installed, aside from the more automatic *-targets.cmake export
Added an explicit find_dependency(OfficialSDK) (not find_package) inside this config script, but inside a clause that adds the MyLibrary cmake install directory to the CMake module path, so the custom find script is used.
A complete example can be seen this PR: https://github.com/ET-BE/AdsClient/pull/1/files (as well as a bunch of other stuff). The OfficialSDK is TwinCAT's ADS library.
Related
CMake has a find_package() backed by a bunch of FindXYZ scripts (which you can also add to).
What mechanism, if any, is available to me to tell cmake: "Find this package, and if you haven't found it, download it and trigger its build" - with the downloading and building part also backed by per-package scripts or settings (so that downloading could be with wget or git clone, building could be with cmake or maven or a package-specific command, etc.) ?
Yeah, I was bitten by that Friday.
So, CMake has an ExternalProject directive, meant for exactly that, get/update if necessary, configure, build and install this and that external project. Awesome!
Sadly, CMake isn't that awesome.
You can't use the target defined by ExternalProject as a library in target_link_libraries. I've really tried to.
The basic problem is that the updating, building and installation of the external project happens at build time, whereas CMake insists on only using libraries that it found during pre-build (i.e. during the CMake run); you can't re-detect stuff while running make/ninja/msvc… .
You can define a custom target, tell it where the .so you'd want to link against later will be, and try to coerce CMake into believing you without checking at pre-build. Sadly, at least in the CMake versions I had, that broke dependency tracking, so that it simply didn't build the external library, because nothing needed it.
From the error messages you get when trying to use an external project in target_link_library, it seems CMake assumes you'd only want to install tools you need at build time that way, not libraries. A bummer.
You can roll your own version of download-on-demand using execute_process() (which runs on the CMake configure step) with ${CMAKE_COMMAND} as the command invoked on a CMakeLists.txt containing ExternalProject_Add().
You could even either configure_file() the CMakeLists.txt to fill out custom variables or dynamically create the CMakeLists.txt file.
I'm creating a very small project that depends on the following library: https://github.com/CopernicaMarketingSoftware/AMQP-CPP
I'm doing what i always do with third-party libraries: i add their git repo as a submodule, and build them along with my code:
option(COOL_LIBRARY_OPTION ON)
add_subdirectory(deps/cool-library)
include_directories(deps/cool-library/include)
target_link_libraries(${PROJECT_NAME} coollib)
This has worked perfectly for libraries like Bullet, GLFW and others. However, this AMQP library does quite an ugly hack. Their include directory is called include, but in their CMake install() command, they rename it to amqpcpp. And their main header, deps/cool-library/amqpcpp.h, is referencing all other headers using that "fake" directory.
What happens is: when CMake tries to compile my sources which depend on deps/cool-library/amqpcpp.h, it fails because it's not finding deps/cool-library/amqpcpp/*.h, only deps/cool-library/include.
Does anyone have any idea how i can fix this without having to bundle the library into my codebase?
This is not how CMake is supposed to work.
CMake usually builds an entire distributive package of a library once and then installs it to some prefix path. It is then accessible for every other build process on the system by saying "find_package()". This command finds the installed distibution, and all the libs, includes etc. automagically. Whatever weird stuff library implementers did, the resulting distros are more or less alike.
So, in this case you do a lot of unnecessary work by adding includes manually. As you see it can also be unreliable.
What you can do is:
to still have all the dependencies source distributions in submodules (usually people don't bother doing this though)
build and install each dependency package into another (.gitignored) folder within the project or outside by using their own CMakeLists.txt. Let's say with a custom build step in your CMakeLists.txt
use "find_package()" in your CMakeLists.txt when build your application
Two small addition to Drop's answer: If the library set up their install routines correctly, you can use find_package directly on the library's binary tree, skipping the install step. This is mostly useful when you make changes to both the library and the dependent project, as you don't have to run the INSTALL target everytime to make library changes available downstream.
Also, check out the ExternalProject module of CMake which is very convenient for having external dependencies being built automatically as part of your project. The general idea is that you still pull in the library's source as a submodule, but instead of using add_subdirectory to pull the source into your project, you use ExternalProject_Add to build it on its own and then just link against it from your project.
I have the following situation:
I'm working on an application that depends on a number of third party libs, among them wxwidgets
I build the application for multiple target configurations (x86, arm, Linux, Windows) using Linux as my build host system
Due to the above mentioned multiple target configurations, I have chosen to build those third-party libs from source, using CMake's ExternalProject_Add function.
The third-party libs are built 'on-demand' at a location separate from my application's CMAKE_BINARY_DIR so that I can wipe the build tree for my application without having to rebuild the third-party libs (takes a looooong time).
The location of the third-party libs is different depending on what target configuration I build them for (obviously).
I'm quite new to CMake and the problem I currently face is this:
The source files in my application can't find the wx include files and I need to set the correct linker flags to be able to link my application against wxwidgets.
This seems to be handled by a utility 'wx-config' that provides exactly that info as output when run with either the --cppflags or --libs flag. I can not however, figure out how to catch that output and append it to the include dirs and linked libraries I setup from my CMakeLists.txt files.
So basically what I want is.
Build wxwidgets (if it doesn't exist) for the current target configuration
Run wx-config --cppflags and --libs to find out the correct include dirs and linker flags for the current target configuration
Use the info from step 2 when building targets that are my own application
So far I've tried something like this:
# Set a target-configuration-specific location
set(wxwidgetsTop ${MYPROJECT_EXTERNAL_DIR}/wxwidgets/wxwidgets_${MYPROJECT_CURRENT_TARGET_CFG})
# Build the project
ExternalProject_Add( wxWidgetsExternal
PREFIX ${wxwidgetsTop}
URL ${MYPROJECT_EXTERNAL_DIR}/tarballs/wxWidgets-3.0.2.tar.bz2
SOURCE_DIR ${wxwidgetsTop}/src/wxwidgets
CONFIGURE_COMMAND ${configure_cmdline}
BUILD_COMMAND make -j${MYPROJECT_NCPU}
INSTALL_COMMAND make install
)
# Create a wxwidgets target to be used as a dependency from other code
add_library(wxWidgets IMPORTED STATIC GLOBAL)
add_dependencies(wxWidgets wxWidgetsExternal)
# (non-working) attempt to get the correct include dirs and linker
# flags for wxwidgets
add_custom_command(TARGET wxWidgetsExternal
POST_BUILD
COMMAND ${INSTALL_DIR}/bin/wx-config ARGS --cppflags
COMMENT "Running wx-config"
)
but the above does not provide a way to actually use the result from the custom command to append the cppflags and linker options when building the targets that make up my application.
What is a good way to achieve what I want?
I see three different ways of doing this:
Method 1: use find_package
Use wxWidgets as a standalone requirement for your project, and expect the devs to install it before building your project. In your CMakeLists.txt you will need to call find_package(wxWidgets), like this:
find_package(wxWidgets COMPONENTS net gl core base)
if(wxWidgets_FOUND)
include(${wxWidgets_USE_FILE})
# and for each of your dependent executable/library targets:
target_link_libraries(<YourTarget> ${wxWidgets_LIBRARIES})
endif()
This has the advantage of not rebuilding the lib if you rebuild your project, however it requires some work for your user (they need to handle the installation of wxWidgets by hand) and for you (you need to setup include paths / compile definitions / ... by hand).
Method 2: embed wxWidgets
The second option is to bundle wxWidgets in your repo (svn external or git submodule) and usually (re)write the CMakeLists.txt of this lib to be target-oriented. Then, in your top-most CMakeLists.txt, you can do the following:
# for example, if you just need core and net:
target_link_librairies(my_app PUBLIC wxWidgetsCore wxWidgetsNet)
# No need to manually setup include dirs, etc...
To make a CMakeLists.txt target-oriented, you define include directories and other compilation properties for a target, not a directory. Example:
# When defining wxWidgetsCore, for example
add_library(wxWidgetsCore ...)
target_include_directories(wxWidgetsCore PUBLIC someDir)
target_compile_definitions(wxWidgetsCore PUBLIC -pedantic)
target_link_libraries(wxWidgetsCore PUBLIC someLib)
The drawback of this approach is that rebuilding your project will trigger a rebuild of wxWidgets. However, it is possible to trick this by not using "rebuild" but "clean just my app, then build". Here is some insight on how to achieve this.
Method 3: some sort of hybrid
The big drawback of method 2 leads to the third approach: don't put wxWidgets in your project, but create a CMakeLists.txt that will "import" the lib. The idea: you ask your user for the directory where wxWidgets is installed, then this script will setup everything for your project. First, put the CMakeLists.txt here:
/your-project-root
/thirdparty
/wxWidgets
CMakeLists.txt
/dir-where-wxwidgets-is-installed
...
Now, you define an imported target:
# When defining wxWidgetsCore, for example
set(WX_INCLUDE_DIR ${USER_SPECIFIED_WX_ROOT}/include)
add_library(wxWidgetsCore IMPORTED GLOBAL)
set_property(TARGET wxWidgetsCore APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES ${WX_INCLUDE_DIR})
See INTERFACE_INCLUDE_DIRECTORIES and INTERFACE_LINK_LIBRARIES. You need your user to have build wxWidgets somewhere in his system, but from your point of view you just do target_link_libraries(your_app PUBLIC wxWidgets...), as in method 2. The advantage is that this approach is interchangeable with method 2 transparently, and you don't put the whole dependency in your project.
Setting cppflags and linker flags has to be done at CMake time, but you are trying to run wx-config at build time and you are not capturing its output anyway, so your add_custom_command() isn't doing anything useful other than printing things to the build tool's output.
Ideally, you would use the FindwxWidgets module CMake already provides. It requires wxWidgets to already be built (but see further below). Have a look at the CMake documentation for it and see if that at least sounds like what you are trying to achieve manually by using wx-config. If you can get FindwxWidgets to do the job for you, that would be a much cleaner approach.
Getting something to build at configure time so you can use it later on in your CMakeLists.txt file is a bit more tricky. ExternalProject_Add() downloads and builds things at build time, but you need wxWidgets to be built earlier at configure time. I wrote an article recently for how to do at least the downloading part at configure time and you should be able to adapt it to do the whole build at configure time instead. The article uses Google Test as its example and can be found here:
https://crascit.com/2015/07/25/cmake-gtest/
It would be trivial to make it put the wxWidgets build wherever you like, not just in the CMAKE_BINARY_DIR area. That would allow you to have different wxWidgets builds for each build configuration and to be able to wipe out your application's build tree independently of the wxWidgets builds.
Hope that points you in the right direction.
The solution I use checks for wxWidgets installation in the system using find_package, if it's not found, then the script downloads wxWidgets from github and links the program against downloaded library. The lib is installed in the build directory, so only the first build is slow - subsequent builds do not even check wxWidgets sources timestamps, so the process is as fast as building using preinstalled wxWidgets library.
Here's how my script does it:
It quietly checks for wxWidgets installation using find_package(wxWidgets QUIET),
If it's found, the script adds a dummy library wxWidgets_external,
If it's not, then it creates an ExternalProject named wxWidgets_external which downloads, builds and installs the library in the build dir, setting wxWidgets_ROOT_DIR to point to the wxWidgets installation dir,
Then we add another ExternalProject pointing to a folder with the main program's source files and CMakeLists.txt build script. This external projects depends on wxWidgets_external which is either a dummy library in case wxWidgets is preinstalled in the system, or an external project set up to download the library from github,
In the aforementioned CMakeLists.txt we again call find_package, this time with REQUIRED parameter and use the library the standard way (https://docs.wxwidgets.org/trunk/overview_cmake.html). Because we set up the dependencies and variables correctly, this call will use either preinstalled wxWidgets (if it's available) or the one downloaded from github.
There are more quirks to it, but that's the gist of it. The full sample code (tested on Linux, Windows and Mac) is available on github (https://github.com/lszl84/wx_cmake_template).
Also see full blog post which explains this in more detail: https://justdevtutorials.medium.com/wxwidgets-cmake-multiplatform-superbuild-4ea86c4e6eda
I'm trying to run this example: http://www.itk.org/Wiki/ITK/Examples/IO/ImageFileReader
But, get the following when I "configure" in CMake:
CMake Error at CMakeLists.txt:11 (find_package):
By not providing "FindItkVtkGlue.cmake" in CMAKE_MODULE_PATH this project
has asked CMake to find a package configuration file provided by
"ItkVtkGlue", but CMake did not find one.
Could not find a package configuration file provided by "ItkVtkGlue" with
any of the following names:
ItkVtkGlueConfig.cmake
itkvtkglue-config.cmake
Add the installation prefix of "ItkVtkGlue" to CMAKE_PREFIX_PATH or set
"ItkVtkGlue_DIR" to a directory containing one of the above files. If
"ItkVtkGlue" provides a separate development package or SDK, be sure it has
been installed.
It seems I need "ItkVtkGlue"? Where can I download it? And, what should I do to combine it with the program?
VtkGlue is a module within ITK, which by default is not built. You need to enable this module with cmake when you build ITK. Also, before you do this, you need to download and build VTK (which is very similar to building ITK). Assuming you are on a unix-like system, have VTK installed, have a src and bin folder for ITK, and are sitting in the bin folder, I would do:
$ ccmake ../src -DModule_ItkVtkGlue=ON
The last option turns the ItkVtkGlue module on by default. You can also do this in the curses GUI without passing this option: press t to get the advanced options, use the arrow keys to scroll down to Module_ItkVtkGlue, and, once there, use the enter key to toggle this option ON. Finally, configure, configure, configure, generate as usual, and then:
$ make
$ sudo make install
Many of the examples on the ITK Wiki use VtkGlue. You can see how to format your CMakeLists.txt file by looking at any of those.
Is there any way to create a FindXXX.cmake automatically where XXX is my Cmake project? I see many projects that they created their FindXXX.cmake manually but I believe it's possible to create it automatically.
And, where I should install my project on Linux?
Thanks!
Take a look at CMake's project config file mechanism (along with the CMakePackageConfigHelper module; you might also want to take a look at this wiki page).
Find scripts are most useful for locating dependencies that are not aware of CMake themselves. If on the other hand the dependency was also built using CMake, you can let CMake auto-generate a project config file for you as part of that project's build process. This config file will allow you to refer to the targets of that project from an enclosing project as if they were being built as part of the enclosing project's CMake run. This is even more powerful than using find scripts, as it allows for example distinct handling of configurations beyond the debug/optimized options available to traditional find scripts.
On Windows, projects generating config files this way will register themselves with CMake, so that depending projects building on the same machine can find them automatically without any additional configuration. If you are building on non-Windows platforms (or you are building the two libraries on different machines) you will have to place the config file in a default directory (the docs for find_package describe which directories are searched) or explicitly point CMake to the location using CMAKE_MODULE_PATH.
Modern CMake-aware libraries should always prefer this approach over traditional find scripts. A prominent example of a library that does this already is Qt5.
CMake supports templating with configure_file() command.
Standard dirs where CMake searches for FindXXX.cmake modules are listed in the documentation of find_package() command.