How do I build a parameterized third-party library in cmake? - c++

I have a project in which I have a third party library checked out as a git submodule. The third party library is just source code with no build system. In addition, The third party library must configured on a per-executable basis by way of compiler definitions and selectively compiling only the parts of the library that I need. I use this library in a lot of different repositories, so I want to make a reusable component to generate an instantiation of the library for any particular executable.
The way I've currently attempted this is by creating a generate_thirdparty.cmake. This file looks something like this:
function(generate_thirdparty parameters)
# several calls to add_library for the different components of this third party library
# the generated libraries depend on the parameters argument
# the parameters configure compile definitions and which source files are compiled
endfunction()
Then in my project CMakeLists.txt I have something like:
cmake_minimum_required(VERSION 3.8)
project(test C)
include(generate_thirdparty.cmake)
set(parameters <some parameters here>)
generate_thirdparty(${parameters})
add_executable(my_exe main.c)
target_link_libraries(my_exe <library names from generate_thirdparty>)
It seems like what I have works, but I'm confused on how you're supposed to do this. I've read through other posts and seen people suggest using find_package or ExternalProject_add. Given a third party repository that contains source code and no build system, which you have no control over, how do you create a reusable way to build that library, especially in the case that the library must be parameterized any given executable?
EDIT: I would like to have the flexibility to have multiple instantiations of the library in the same project.

Let's sum up:
The third-party library does not provide its own build.
You need many instantiations of the library within a single build.
You use these instantiations across multiple different repositories.
I think you're pretty much taking the right approach. Let's call the third-party library libFoo for brevity. Here's what I think you should do...
Create a wrapper repository for libFoo that contains a FindFoo.cmake file and the actual foo repository submodule next to it. This is to avoid the contents of FindFoo.cmake from being independently versioned across your various projects.
Include the wrapper as your submodule in dependent projects, say in the directory third_party/foo_wrapper
In those dependent projects, write:
list(APPEND CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/third_party/foo_wrapper")
find_package(Foo REQUIRED)
generate_foo(config1 ...)
generate_foo(config2 ...)
add_executable(app1 src/app1/main.cpp)
target_link_libraries(app1 PRIVATE foo::config1)
add_executable(app2 src/app2/main.cpp)
target_link_libraries(app2 PRIVATE foo::config2)
The contents of FindFoo.cmake will simply be:
cmake_minimum_required(VERSION 3.18)
function(generate_foo target)
# Use CMAKE_CURRENT_FUNCTION_LIST_DIR to reference foo's source files
# for example:
set(sources "src/src1.cpp" "src/src2.cpp" ...)
list(TRANSFORM sources PREPEND "${CMAKE_CURRENT_FUNCTION_LIST_DIR}/foo/")
add_library(${target} ${sources})
add_library(foo::${target} ALIAS ${target})
# Do other things with ARGN
endfunction()
# Define a version for this dependency + script combo.
set(Foo_VERSION 0.1.0)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Foo VERSION_VAR Foo_VERSION
HANDLE_COMPONENTS)
The CMAKE_CURRENT_FUNCTION_LIST_DIR is the absolute path to the file containing the function being called. Remember that this is the root of your wrapper repository, so the actual sources for libFoo will be in the adjacent foo directory. list(TRANSFORM) lets us write relative paths in the sources list that get converted to absolute paths for the sake of add_library (passing a relative path to add_library would be relative to the caller's source directory).
I also create an ALIAS target so that callers of generate_foo can link to the alias. This is important because CMake considers names that contain :: to be targets when a library is expected. This very helpfully turns typos from unintended linker flags into configure-time "target not found" errors.
Then we define the function like normal and call the find_package_handle_standard_args to handle find_package arguments like REQUIRED, COMPONENTS (even just to check that none were incorrectly specified), and VERSION. See the docs for it, here: https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html

Related

What's a simple straightforward find_package set of files for CMake

I'm new(ish) to CMake (meaning off and on over a few years I've been forced to use it even though it's something that's made me question my career path).
I have a folder with an include folder, and a lib folder containing .lib files and their corresponding .dlls for a dependency I'll call "mydep". I need to provide the infrastructure files, mydep-config.cmake, mydep-target.cmake, etc, which will add the includes folders and .lib files to the command line for compiling and linking and then move the .dlls to a specific location.
Can anyone point me to a simple example anywhere on the net that illustrates the way to do this? The CMake documentation is utterly useless.
Thanks!
There are basically 2 files you need to put in the correct location on the file system. Let's assume the library is to import is called ExternLib with version 1.2.3 and compiled for 64 bit. With the and you've got it stored in your repo as
external_dependencies/lib/ExternLib.lib
external_dependencies/bin/ExternLib.dll
external_dependencies/include/ExternLib/ExternHeader.hpp
external_dependencies/include/ExternLib/...
...
and you want the include paths used in your project to be
#include "ExternLib/ExternHeader.hpp"
First of all we'll use the file names externlib-config.cmake and externlib-version-config.cmake to allow the user to use arbitrary case when using find_package.
externlib-version-config.cmake
This file is used to check, if the version of the config script is compatible with the cmake configuration trying to find the package. Adding this file allows you to place multiple versions of the library to be found for different target architectures (e.g. Windows x64, Win32, linux x86_64, ...). find_package won't read the externlib-config.cmake in the same directory, if the version is marked as non-compatible via the version file.
set(PACKAGE_VERSION "1.2.3")
# basically ignore the version passed
set(PACKAGE_VERSION_COMPATIBLE TRUE) # consider every version compatible
#usually you'd add logic for ignoring any unspecified parts of the version here instead of treating unspecified version parts as 0
# we'll keep it simple though
if(PACKAGE_FIND_VERSION VERSION_EQUAL PACKAGE_VERSION)
set(PACKAGE_VERSION_EXACT TRUE) # make version as exact match
endif()
if(NOT WIN32)
set(PACKAGE_VERSION_COMPATIBLE FALSE) # disallow older version
else()
if (NOT CMAKE_SYSTEM_NAME STREQUAL "Windows") # note: we'll ignore the possibility of find_package before the first project() command
set(PACKAGE_VERSION_UNSUITABLE TRUE)
endif()
if(NOT CMAKE_SIZEOF_VOID_P STREQUAL "8") # check for 64 bit
set(PACKAGE_VERSION_UNSUITABLE TRUE)
endif()
externlib-config.cmake
Here we'll put the creation of the actual library in addition to providing any functions/macros/variables the user should have access to. We'll use ExternLib::ExternLib as target name, but you can simply replace this by any name of your choosing unique in your project.
if(NOT TARGET ExternLib::ExternLib) # prevent recreation on multiple uses of find_package
add_library(ExternLib::ExternLib SHARED IMPORTED)
get_filename_component(EXTERNLIB_BASE_DIR ${CMAKE_CURRENT_LISTS_DIR}/../../.. ABSOLUTE)
set(EXTERNLIB_BINARY_DIR ${EXTERNLIB_BASE_DIR} CACHE INTERNAL "Directory containing the dlls for ExternLib")
# add info about locations of the dll/lib files
set_target_properties(ExternLib::ExternLib PROPERTIES
IMPORTED_LOCATION "$CACHE{EXTERNLIB_BINARY_DIR}/ExternLib.dll"
IMPORTED_IMPLIB "${EXTERNLIB_BASE_DIR}/lib/ExternLib.lib"
)
# add include directory information
target_include_directories(ExternLib::ExternLib INTERFACE "${EXTERNLIB_BASE_DIR}/include")
# add dependencies, if required; you may need to use find_package to locate those
# target_link_libraries(ExternLib::ExternLib INTERFACE ...)
endif()
# any helper function/macro definitions should go here, since they may need to get reintroduced
Note: I did ignore the possibility of components of a package being specified here. Any components specifying components for find_package are simply ignored for the script above.
Placement of the files
You'll need to place the files in one of some possible paths below one of the directories mentioned in CMAKE_PREFIX_PATH or in one of the default dirs, see find_package Config Mode Search Procedure.
Note: we'll use a path that isn't documented for windows, since this path would also work for linux. (Boost is doing this too.)
lib/cmake/ExternLib-x64-1.2.3
(You could choose a different suffix to lib/cmake/ExternLib or just use lib/cmake/ExternLib. The search procedure picks up on any directory names which starts with the package name ignoring case, if it expects the lib name.)
Place both the files in this directory. externlib-config.cmake assumes lib is external_dependencies/lib here. Otherwise you may need to adjust EXTERNLIB_BASE_DIR accordingly.
Usage
We'll assume the CMakeLists.txt file is placed in the same directory as external_dependencies
project(...)
...
add_executable(my_exe ...)
...
# this allows the user to pass directories to be searched first via -D option during configuration
list(APPEND CMAKE_PREFIX_PATH ${CMAKE_CURRENT_SOURCE_DIR}/external_dependencies)
find_package(ExternLib REQUIRED)
target_link_libraries(my_exe PRIVATE ExternLib::ExternLib)
# allow vs debugger to find the dll without copying the dlls around
set_target_properties(my_exe PROPERTIES
VS_DEBUGGER_ENVIRONMENT "PATH=${EXTERNLIB_BINARY_DIR};$ENV{PATH}"
)

CMake make add_library depend on ExternalProject_Add

TLDR: I would like to ask CMake to wait for ExternalProject_Add to complete before it attempts to move on to the next subdirectory and build a library that happens to use one of the files of the external project. In other words, I like to declare an external project as a dependency for a shared library.
More Description:
Suppose that my CMake project has two directories: thirdparty and src. My top-level CMakeLists.txt file has:
add_subdirectory(thirdparty)
add_subdirectory(src)
thirdparty/CMakeLists.txt contains multiple ExternalProject_Add commands. My original intention was to pull and build all these external projects and then move on to building my own libraries and executables in src. Unfortunately, this didn't go as I planned:
One of my external projects is called libsvm. And my src/CMakeLists.txt has the following:
set(Libsvm_SOURCE_FILES
${PROJECT_BINARY_DIR}/thirdparty/libsvm/src/libsvm/svm.cpp
)
include_directories(
${Libsvm_INCLUDE_DIR}
)
add_library(
mysvm
SHARED
${Libsvm_SOURCE_FILES}
)
Now the problem I am facing with is that CMake is unable to find ${Libsvm_SOURCE_FILES}, apparently because this step is being executed before the ExternalProject_Add in my thirdparty/CMakeLists.txt file is executed.
I would like to declare this external project as a dependency for this library.
Broader Question:
Is there a clean way to force CMake to finish everything in first subdirectory before moving on to the next? If not, do you recommend that I make any change in the hierarchy and organization of my CMakeLists files?
Thanks!
CMake expects every source file, passed to add_library or add_executable, to be existed unless it is marked as GENERATED. This property is automatically set for files listed as OUTPUT for add_custom_command. In other cases one need to set this property explicitly:
set_source_files_properties(${Libsvm_SOURCE_FILES} PROPERTIES GENERATED TRUE)

How to make cmake find a shared library in a subfolder

I'm trying to learn how to make a shared library. And the following seems to work (please comment if you have some feedback to this method, I basically have no idea what I'm doing).
In my library project, I've put the header files into a folder named "include", and the source files into "src".
My library's CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(mycustomlib)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Include header files
include_directories(include)
# Create shared library
add_library(${PROJECT_NAME} SHARED ${SOURCES})
# Install library
install(TARGETS ${PROJECT_NAME} DESTINATION lib)
# Install library headers
file(GLOB HEADERS include/*.h)
install(FILES ${HEADERS} DESTINATION include)
My application's CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(myprogram)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Create executable
add_executable(${PROJECT_NAME} ${SOURCES})
# Find and link library
find_library(MYCUSTOMLIB mycustomlib)
target_link_libraries(${PROJECT_NAME} ${MYCUSTOMLIB})
And this is working. The problem is that I want to put both the headers and the library into subfolders (specifically: /usr/local/include/mycustomlib/ for the headers, and /usr/local/lib/mycustomlib/ for the library).
So this is my attempt:
My library's new CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(mycustomlib)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Include header files
include_directories(include)
# Create shared library
add_library(${PROJECT_NAME} SHARED ${SOURCES})
# Install library
install(TARGETS ${PROJECT_NAME} DESTINATION lib/${PROJECT_NAME})
# Install library headers
file(GLOB HEADERS include/*.h)
install(FILES ${HEADERS} DESTINATION include/${PROJECT_NAME})
My application's new CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(myprogram)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Create executable
add_executable(${PROJECT_NAME} ${SOURCES})
# Find and link library
find_library(MYCUSTOMLIB mycustomlib/mycustomlib)
target_link_libraries(${PROJECT_NAME} ${MYCUSTOMLIB})
And this is not working. Now, I'm forced to specify the .so file of the library like this:
find_library(MYCUSTOMLIB mycustomlib/libmycustomlib.so)
How come?
I'll deal with your actual problem first and offer additional comments after that. Technically speaking, you are asking CMake to find a library named mycustomlib/mycustomlib, but what you really want to say is you want find mycustomlib and it can be found in a subdirectory called mycustomlib. A couple of alternative ways to call find_library() to achieve this for your second case would be:
find_library(MYCUSTOMLIB mycustomlib PATH_SUFFIXES mycustomlib)
find_library(MYCUSTOMLIB mycustomlib PATHS /usr/local/lib/mycustomlib)
The latter is making more assumptions than it should about where you have the library installed, so I'd favour the first option. The first option assumes CMake would already find libraries in /usr/local/lib, which it seems it is from your question. You can influence where CMake looks for libraries by modifying CMAKE_PREFIX_PATH and CMAKE_LIBRARY_PATH. I'd expect either of the above options to make your second case work.
Now to other observations. You've requested a very old minimum CMake version in the first line of each of your CMakeLists.txt files. You probably want to consider at the very least making this 2.8 (personally, I'd suggest more like 3.2 or later, but it depends on what your project needs to support).
You have used file globbing to obtain your list of sources and headers. This is not robust and should generally be avoided (see a discussion of this here). You will see plenty of example code use method this for simplicity, but it is not recommended for real world projects (the CMake documentation even says not to use it). Explicitly list out your source and header files individually if you want robust builds.
If you are happy to require CMake 2.8.11 or later (and you should be these days), rather than calling include_directories() which makes everything pick up the header search path you specified, you should prefer to attach the search path requirement to the target that needs it. You do this with target_include_directories(). The equivalent of your code above would be:
target_include_directories(${PROJECT_NAME} PUBLIC include)
This gives much better control of your inter-target dependencies as your project grows in size and complexity. For a more in-depth discussion of this topic, see this article and perhaps also this one (disclosure: I wrote both articles).
Are your library and program totally separate source code repositories? Can they be built in the same project? You can build multiple targets in one CMakeLists.txt file. The project name doesn't have to have any relationship to the names of any of the targets (you often see the PROJECT_NAME variable re-used for the target name in simple examples, which is unfortunate since it suggests a relationship between the two, but for all but simple projects this won't be the case). If they are in the same repository, building them together would be a much simpler build since you wouldn't have to install the library for the executable to find it and link to it.
If they must be built in separate projects, then something like the following for the application's project should get you close:
cmake_minimum_required(VERSION 2.8.11)
project(myprogram)
# List your program's sources here explicitly
add_executable(myprogram src/foo.cpp src/bar.cpp)
# Find and link library
find_library(MYCUSTOMLIB mycustomlib PATH_SUFFIXES mycustomlib)
target_link_libraries(myprogram PUBLIC ${MYCUSTOMLIB})
# Find library's headers and add it as a search path.
# Provide the name of one header file you know should
# be present in mycustomlib's include dir.
find_path(MCL_HEADER_PATH mycustomlib.h PATH_SUFFIXES mycustomlib)
target_include_directories(myprogram PUBLIC ${MCL_HEADER_PATH})
For extra points, you could try to confirm that the header path is in the same area as the library by checking the common path prefix, or you could just derive
the MCL_HEADER_PATH from the MYCUSTOMLIB path by assuming a directory structure. Both approaches have advantages and drawbacks. If you want to explore the latter, the get_filename_component() command will be your friend.
Hopefully that points you in the right direction.

Cmake and Gtest

I want to use Google C++ Testing and I am completely beginner to cmake and gtest.
I have a class called Filter which uses a 3d party library called jane.
For this case I have a cmakeFile which builds my project nicely as follows:
cmake_minimum_required(VERSION 3.1.2)
project(Filter)
include(../../../cmake/CMakeMacros.txt)
set_variables()
#add 3rdparty libraries
add_jane()
#add framework libraries
add_framework_libs(
ip/Image
)
include_directories(
../include
${FW_INCLUDE_DIRS}
)
#set project's source and include files
set(INCS
../include/${PROJECT_NAME}.h
../include/${PROJECT_NAME}.tpp
../include/FilterMask.h
)
set(SRCS
../src/${PROJECT_NAME}.cpp
../src/FilterMask.cpp
)
#set link directories
link_directories(
${FW_LIBRARY_DIRS}
)
#build project as static library (*.lib)
add_library(${PROJECT_NAME} STATIC
${INCS}
${SRCS}
)
#link libraries against project
target_link_libraries( ${PROJECT_NAME}
${FW_LIBRARIES}
)
#if a test executable should be build
if(Test_BUILD_EXAMPLES)
#build test executable
add_executable(${PROJECT_NAME}Test
../src/main.cpp
)
#link library against executable
target_link_libraries(${PROJECT_NAME}Test
${PROJECT_NAME}
)
endif(Test_BUILD_EXAMPLES)
and also I have read this simple tutorial on https://github.com/snikulov/google-test-examples with this cmake file https://github.com/snikulov/google-test-examples/blob/master/CMakeLists.txt and tried to build my project again to combine these cmake files together (may be in very silly way) but I can not achieve it since days.
The problem is that when I want to test a simple project with just a header file I can use this cmake file but as soon as I try to test my project containing a 3rd party library I run into different errors.
Can someone please tell me how I can edit a correct cmake file to test my project with googleTest using a cmake file !?
If you want to link against a 3rd party library you typically first:
find_package() or use the pkg config support to check the library is available on the build host and pick up a reference to it.
Include the reference from step #1 in target_link_libraries()
So that's what you need to do for your 3rd party lib. For your own code which you want to bring under test you probably want to put it all inside your own libraries and then link your tests against those.
If you have multiple test executables to separate & isolate each test suite into its own binaries you probably want an alternative technique to avoid over linking and limit the code inside the test suite to the actual unit under test. (This is also quite useful when your code base is in flux and builds only partially but you still wish to check that what builds continues to pass relevant tests.)
In that case you may want to define your units under test as OBJECT type libraries and then instead of doing target_link_libraries() against those object libs you include the objects as part of the sources for the executable using this syntax: $<TARGET_OBJECTS:NameOfObjLibHere> (cmake generator expressions).
So in the case of units which depend on a 3rd party lib, say, Qt5 Core, you'd have snippets like this:
# define the dependency on 3rd party project Qt5, (sub)component: Core, Test)
set(MY_QT_VERSION "5.4.0")
find_package(Qt5 ${MY_QT_VERSION} REQUIRED COMPONENTS Core CONFIG)
# define the object lib for a unit to be tested (Item1)
set(item1_srcs item1.cpp util1.cpp)
add_library(Item1 TYPE OBJECT ${item1_srcs})
# ensure that necessary compiler flags are passed
# when building "Item1" separately
# note that PRIVATE may also be INTERFACE or PUBLIC
# read the cmake docs on target_include_*** to determine which applies.
# you probably want to hide this behind a convenience macro.
target_include_directories(Item1 PRIVATE $<TARGET_PROPERTY:Qt5::Core,INTERFACE_INCLUDE_DIRECTORIES>)
target_compile_options(Item1 PRIVATE $<TARGET_PROPERTY:Qt5::Core,INTERFACE_COMPILE_OPTIONS>)
# find the unit testing framework (dependency)
# this sample uses Qt5 Test (Qt5::Test) but you could use GTest, too
find_package(Qt5 ${MY_QT_VERSION} REQUIRED COMPONENTS Test CONFIG)
# define an executable which contains test sources + unit under test
# link against the testing framework (obviously) as per normal
# note the Qt5::Core dependency here: remember Item1 depends on Qt5::Core (!)
set(test_item1_srcs, test_item1.cpp $<TARGET_OBJECTS:Item1>)
add_executable(test_item1 ${test_item1_srcs)
target_link_libraries(test_item1 Qt5::Core Qt5::Test)
# inform cmake/ctest integration about our test
# so it knows to execute it during `make test` phase.
# and other cmake/ctest integration falls into place as well, possibly
add_test(test_item1 test_item1)

CMake: build target library if any of the executable targets needs it

The layout of my project is as follow :
src/
include/
include1.h
include2.h
include3.h
lib/
lib1/
source1_lib1.c
source2_lib1.c
lib2/
source1_lib2.c
source2_lib2.c
source3_lib2.c
lib3/
source1_lib3.c
lib4/
source1_lib4.c
source2_lib4.c
module_A/ (this module will need lib1 and lib4)
source1_moduleA.c
source2_moduleA.c
module_B/ (this module will need lib2 and lib3)
source1_moduleB.c
source2_moduleB.c
source3_moduleB.c
module_C/ (this module will need lib1, lib2, lib3 and lib4)
source1_moduleC.c
module_D/ (this module will need lib1 and lib3)
source1_moduleD.c
source2_moduleD.c
source3_moduleD.c
source4_moduleD.c
The global solution can be made by any number of module_X (it depends on the customer)
My project CMake file located under "/src" includes a configuration file (it is defined by a customer needs). This configuration file indicates which modules must be built and packaged to the target customer.
Let's say I have a customer X and the modules he selected are module_A and module_D. In this case my build system should only builds lib_1, lib_3 and lib_4.
What I am looking for is a way to define target libraries without being built until I do reference them in one the CMakeLists files under module_X directories.
Oh my bad I missed to say A BIG THANKS FOR YOUR HELP
CMake can not do this out of the box. Any library that is added through an add_library call will be built.
But you can implement the behavior you want inside the CMake. Instead of calling add_library and add_executable directly you would call custom wrapper functions. The add_library wrapper simply stores the information required to call add_library, but does not call it directly. The add_executable wrapper iterates over all dependencies of the executable and makes the call to add_library if required.
You will have to design your own system for maintaining the state whether a particular library has already been added and what the parameters are for constructing the library target. Things get slightly more difficult when respecting transient dependencies as well (module_1 depends on lib_a which in turn depends on lib_b; but no executable depends on lib_b directly). But it is perfectly possible to build such a system with a few hundred lines of CMake code.
cmake ..
make module_A module_D