The layout of my project is as follow :
src/
include/
include1.h
include2.h
include3.h
lib/
lib1/
source1_lib1.c
source2_lib1.c
lib2/
source1_lib2.c
source2_lib2.c
source3_lib2.c
lib3/
source1_lib3.c
lib4/
source1_lib4.c
source2_lib4.c
module_A/ (this module will need lib1 and lib4)
source1_moduleA.c
source2_moduleA.c
module_B/ (this module will need lib2 and lib3)
source1_moduleB.c
source2_moduleB.c
source3_moduleB.c
module_C/ (this module will need lib1, lib2, lib3 and lib4)
source1_moduleC.c
module_D/ (this module will need lib1 and lib3)
source1_moduleD.c
source2_moduleD.c
source3_moduleD.c
source4_moduleD.c
The global solution can be made by any number of module_X (it depends on the customer)
My project CMake file located under "/src" includes a configuration file (it is defined by a customer needs). This configuration file indicates which modules must be built and packaged to the target customer.
Let's say I have a customer X and the modules he selected are module_A and module_D. In this case my build system should only builds lib_1, lib_3 and lib_4.
What I am looking for is a way to define target libraries without being built until I do reference them in one the CMakeLists files under module_X directories.
Oh my bad I missed to say A BIG THANKS FOR YOUR HELP
CMake can not do this out of the box. Any library that is added through an add_library call will be built.
But you can implement the behavior you want inside the CMake. Instead of calling add_library and add_executable directly you would call custom wrapper functions. The add_library wrapper simply stores the information required to call add_library, but does not call it directly. The add_executable wrapper iterates over all dependencies of the executable and makes the call to add_library if required.
You will have to design your own system for maintaining the state whether a particular library has already been added and what the parameters are for constructing the library target. Things get slightly more difficult when respecting transient dependencies as well (module_1 depends on lib_a which in turn depends on lib_b; but no executable depends on lib_b directly). But it is perfectly possible to build such a system with a few hundred lines of CMake code.
cmake ..
make module_A module_D
Related
I have a project in which I have a third party library checked out as a git submodule. The third party library is just source code with no build system. In addition, The third party library must configured on a per-executable basis by way of compiler definitions and selectively compiling only the parts of the library that I need. I use this library in a lot of different repositories, so I want to make a reusable component to generate an instantiation of the library for any particular executable.
The way I've currently attempted this is by creating a generate_thirdparty.cmake. This file looks something like this:
function(generate_thirdparty parameters)
# several calls to add_library for the different components of this third party library
# the generated libraries depend on the parameters argument
# the parameters configure compile definitions and which source files are compiled
endfunction()
Then in my project CMakeLists.txt I have something like:
cmake_minimum_required(VERSION 3.8)
project(test C)
include(generate_thirdparty.cmake)
set(parameters <some parameters here>)
generate_thirdparty(${parameters})
add_executable(my_exe main.c)
target_link_libraries(my_exe <library names from generate_thirdparty>)
It seems like what I have works, but I'm confused on how you're supposed to do this. I've read through other posts and seen people suggest using find_package or ExternalProject_add. Given a third party repository that contains source code and no build system, which you have no control over, how do you create a reusable way to build that library, especially in the case that the library must be parameterized any given executable?
EDIT: I would like to have the flexibility to have multiple instantiations of the library in the same project.
Let's sum up:
The third-party library does not provide its own build.
You need many instantiations of the library within a single build.
You use these instantiations across multiple different repositories.
I think you're pretty much taking the right approach. Let's call the third-party library libFoo for brevity. Here's what I think you should do...
Create a wrapper repository for libFoo that contains a FindFoo.cmake file and the actual foo repository submodule next to it. This is to avoid the contents of FindFoo.cmake from being independently versioned across your various projects.
Include the wrapper as your submodule in dependent projects, say in the directory third_party/foo_wrapper
In those dependent projects, write:
list(APPEND CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/third_party/foo_wrapper")
find_package(Foo REQUIRED)
generate_foo(config1 ...)
generate_foo(config2 ...)
add_executable(app1 src/app1/main.cpp)
target_link_libraries(app1 PRIVATE foo::config1)
add_executable(app2 src/app2/main.cpp)
target_link_libraries(app2 PRIVATE foo::config2)
The contents of FindFoo.cmake will simply be:
cmake_minimum_required(VERSION 3.18)
function(generate_foo target)
# Use CMAKE_CURRENT_FUNCTION_LIST_DIR to reference foo's source files
# for example:
set(sources "src/src1.cpp" "src/src2.cpp" ...)
list(TRANSFORM sources PREPEND "${CMAKE_CURRENT_FUNCTION_LIST_DIR}/foo/")
add_library(${target} ${sources})
add_library(foo::${target} ALIAS ${target})
# Do other things with ARGN
endfunction()
# Define a version for this dependency + script combo.
set(Foo_VERSION 0.1.0)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Foo VERSION_VAR Foo_VERSION
HANDLE_COMPONENTS)
The CMAKE_CURRENT_FUNCTION_LIST_DIR is the absolute path to the file containing the function being called. Remember that this is the root of your wrapper repository, so the actual sources for libFoo will be in the adjacent foo directory. list(TRANSFORM) lets us write relative paths in the sources list that get converted to absolute paths for the sake of add_library (passing a relative path to add_library would be relative to the caller's source directory).
I also create an ALIAS target so that callers of generate_foo can link to the alias. This is important because CMake considers names that contain :: to be targets when a library is expected. This very helpfully turns typos from unintended linker flags into configure-time "target not found" errors.
Then we define the function like normal and call the find_package_handle_standard_args to handle find_package arguments like REQUIRED, COMPONENTS (even just to check that none were incorrectly specified), and VERSION. See the docs for it, here: https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
I have a cross-compiler cmake project that depends on libraries from a separate project that happens to also use cmake:
/myProject/CMakeLists.txt (uses cross-compiler)
/anotherProject/CMakeLists.txt (platform-agnostic)
anotherProject can be built completely separately on its own. It has no knowledge of myProject at all.
Now, anotherProject has numerous modules that I need, like:
anotherProject/A/CMakeLists.txt (produces static lib A.a)
anotherProject/B/CMakeLists.txt (produces static lib B.a)
etc
When I build myProject, I want to build and link against anotherProject/A and anotherProject/B, to produce shared lib myproject.so. I'd like to leverage the existing cmake-ness of anotherProject if possible, as opposed to manually globbing its various source sets from myProject.
What's the correct way to achieve this with cmake? I feel like I'm missing something obvious.
It would be straightforward if, say, myProject were just a subdirectory under anotherProject, or if there were a top-level CMakeLists.txt that could reference both myProject and anotherProject; but neither is what I'm after. I know I could build anotherProject and export its libraries to a well-known location, and then reference the export directory from myProject - but I would like to avoid that setup as well.
A solution is to use CMake packages.
Basically, in anotherProject, you craft a CMake configuration file where you set variables to be used by myProject (eg. include directory, list of libraries, compilation flags...), or even targets.
Then, in myProject, you use the find_package() mechanism so that CMake finds this configuration file and imports the variables/targets in your current project.
There is a tutorial on the CMake wiki.
The only alternative setup that I can think of based on your requirements is to allow your main (dependent) project to discover the other (dependee) project using find_package.
In your main project CMakeLists.txt you should add something like this:
find_package(anotherProject CONFIG)
if(anotherProject_FOUND)
message(STATUS "Found project dependency: anotherProject")
else
# change WARNING to FATAL_ERROR if the dependency is NOT optional
message(WARNING "package anotherProject was not found")
endif()
On the differences between CONFIG and MODULE modes, check the documentation and this link.
Then assuming that your main project creates an executable, you could hook up the discovered dependency like this:
add_executable(myProject ${SOURCES})
[...]
if(anotherProject_FOUND)
target_link_libraries(myProject PUBLIC anotherProject)
endif()
This should take care of the required include files and definitions as well.
Now in the dependee project CMakeLists.txt you should do something like this:
set(PRJ_NAME "anotherProject")
string(TOLOWER ${PRJ_NAME} PRJ_NAME_LOWER)
set(ANOTHERPROJECT_EXPORT_NAME "${PRJ_NAME}")
install(TARGETS ${PRJ_NAME} EXPORT ${ANOTHERPROJECT_EXPORT_NAME}
RUNTIME DESTINATION .)
install(EXPORT ${ANOTHERPROJECT_EXPORT_NAME} DESTINATION "share/cmake")
This associates an export with a target and then installs the export.
Now, if you check that export file, it expects certain things to be found and included, that could be specific for your project. To make this as supple as possible, you can use the configure feature to generate them from a template and then install from the build directory.
So, in the project under a subdir named share/cmake you could have a file named config.cmake.in with contents:
include(${CMAKE_CURRENT_LIST_DIR}/#PRJ_NAME#.cmake)
In the main project's CMakeLists.txt you need to add the following for generating the file from that template:
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/share/cmake/config.cmake
${CMAKE_CURRENT_BINARY_DIR}/share/cmake/${PRJ_NAME_LOWER}-config.cmake)
install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/share/
DESTINATION share)
Notice that I used PRJ_NAME, because you could potentially reuse that to name the actual executable at the add_executable command. It mentally helps if the exported target has the same name with produced one.
This is a more versatile version to accommodate multiple subprojects of this tutorial.
TLDR: I would like to ask CMake to wait for ExternalProject_Add to complete before it attempts to move on to the next subdirectory and build a library that happens to use one of the files of the external project. In other words, I like to declare an external project as a dependency for a shared library.
More Description:
Suppose that my CMake project has two directories: thirdparty and src. My top-level CMakeLists.txt file has:
add_subdirectory(thirdparty)
add_subdirectory(src)
thirdparty/CMakeLists.txt contains multiple ExternalProject_Add commands. My original intention was to pull and build all these external projects and then move on to building my own libraries and executables in src. Unfortunately, this didn't go as I planned:
One of my external projects is called libsvm. And my src/CMakeLists.txt has the following:
set(Libsvm_SOURCE_FILES
${PROJECT_BINARY_DIR}/thirdparty/libsvm/src/libsvm/svm.cpp
)
include_directories(
${Libsvm_INCLUDE_DIR}
)
add_library(
mysvm
SHARED
${Libsvm_SOURCE_FILES}
)
Now the problem I am facing with is that CMake is unable to find ${Libsvm_SOURCE_FILES}, apparently because this step is being executed before the ExternalProject_Add in my thirdparty/CMakeLists.txt file is executed.
I would like to declare this external project as a dependency for this library.
Broader Question:
Is there a clean way to force CMake to finish everything in first subdirectory before moving on to the next? If not, do you recommend that I make any change in the hierarchy and organization of my CMakeLists files?
Thanks!
CMake expects every source file, passed to add_library or add_executable, to be existed unless it is marked as GENERATED. This property is automatically set for files listed as OUTPUT for add_custom_command. In other cases one need to set this property explicitly:
set_source_files_properties(${Libsvm_SOURCE_FILES} PROPERTIES GENERATED TRUE)
Consider the following CMakeLists.txt file:
add_subdirectory(execA)
add_subdirectory(libB)
install(TARGETS execA libB
RUNTIME DESTINATION bin
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib)
I get the following error:
install TARGETS given target "execA" which does not exist in this
directory
execA and libB have their own CMakeList.txt files and are located under project directory, as well as the build directory I'm running cmake (cmake ..):
project
|------ CMakeList.txt (the one with the code)
|----execA
| \- .cpp, .hpp and CMakelist.txt
|----libB
| \- .cpp, .hpp and CMakelist.txt
|---- lib
|---- bin
\---- build (where I´m commanding: $ cmake ..
How do I fix this error?
According to this bugreport, install(TARGETS) command flow accepts only targets created within the same directory.
So you need either move the add_library() call into the top-level directory, or split install(TARGETS) call into per-target ones, and move each of them into the corresponding subdirectory.
Since CMake 3.13 install(TARGETS) can work even with targets created in other directories.
install(TARGETS) can install targets that were created in other directories. When using such cross-directory install rules, running make install (or similar) from a subdirectory will not guarantee that targets from other directories are up-to-date.
Even though it would help seeing the CMakeLists.txt files contained in the subdirectories, I guess they contain add_executable and/or add_library statements to create your stuff.
Also, because of your example, I guess you are using the same name of your directories for your targets.
That said, you should know that symbols defined in a CMakeLists.txt file in a subdirectory are not visible by default within the context of the CMakeLists.txt file in the parent directory. Because of that, you should rather move your install statements within the CMakeLists.txt files within your subdirectories.
This should solve the problem, if my thoughts were right. Otherwise, I strongly suggest you to post in your question also the content of the other files above mentioned.
Anyway, the error is quite clear.
The file that contains the install statement for the target named X does not contain a target creation statement (add_executable and the others) that gives birth to that target, so it goes on saying that that target does not exist in that directory.
This still seems to be a pain point in CMake 3.11.
In our codebase, we have many targets defined in subdirectories and need to create an assortment of installers with different configurations and (potentially overlapping) combinations of targets.
Here's my solution:
Before calling add_subdirectory in your root CMakeLists.txt file, create a GLOBAL property with the names of the target(s) you want to include in your installer.
Wrap target creation functions (add_executable, etc.) in your own custom functions. Within those functions check if the target is present in the global property, and invoke install accordingly.
That approach allows you to centralize installer configuration.
Also: To support creation of multiple installers, we populate our global list along with other installer properties in separate .cmake files. When we invoke cmake, we pass the name of the installer configuration CMake file as a command-line argument. Our root CMakeLists.txt file simply calls include with that file.
I would like to have the following structure A -> B -> C, where:
C is boilerplate code, wrappers for third-party libraries, very
basic code etc.
B is the common classes, functions and data
structures specific to the project's domain.
A is the project itself.
I would like to make it easy to reuse C or B(+C) in future in my other projects. In addition, I have the following requirements:
As all three projects are in-progress, I would like to have an ability to build C, C+B and C+B+A in one shot.
I would prefer the static linkage over dynamic, so that C and C+B would be static libraries, and C+B+A would be the executable
I would like to keep cmake lists and config files simple and clean. Examples which I found in the official wiki and over the internet are pretty big and monstrous.
It would be great if it won't require changing more than a couple of lines if I'd change the locations of A, B or C in the filesystem.
All these three components are using google-test, but I'm not sure if it is important for the project layout.
I am pretty new to cmake and I don't even understand is it better to write XXXConfig.cmake or FindXXX.cmake files. Also, I am not sure, how should I pass relative paths from subcomponent to the parent component using X_INCLUDE_DIRS.
First I have to admit that I agree with #Tsyvarev. Your CMake environment should fit to your processes/workflow and should take project sizes and team structure into account. Or generally speaking the environment CMake will be used in. And this tends to be - in a positive way - very alive.
So this part of your question is difficult to answer and I'll concentrate on the technical part:
CMake has to know the location of the dependencies - relative or absolute - by
having a monolithic source tree (the one you don't want anymore)
CMake share library with multiple executables
CMake: How to setup Source, Library and CMakeLists.txt dependencies?
a common directory location for includes/libraries/binaries
Custom Directory for CMake Library Output
cmake install not installing libraries on windows
getting the paths via config files/variable definitions
How can I get cmake to find my alternative boost installation?
How to add_custom_command() for the CMake build process itself?
using registration in or installation from a database provided on the host
Making cmake library accessible by other cmake packages automatically
cmake wont run build_command in ExternalProject_Add correctly
To keep your CMake files as simple as possible I would recommend to group your CMake code into separate dedicated files:
Prefer toolchain files over if(SomeCompiler) statements
Move common/repeating code parts as function() bodies into a shared CMake include file
Move complex non-target specific code parts into their own (CMake) script files
Example Code
Since you have specifically asked for the find_package() variant, taking Use CMake-enabled libraries in your CMake project and the things listed above:
MyCommonCode.cmake
cmake_policy(SET CMP0022 NEW)
function(my_export_target _target _include_dir)
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Config.cmake"
"
include(\"\$\{CMAKE_CURRENT_LIST_DIR\}/${_target}Targets.cmake\")
set_property(
TARGET ${_target}
APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES \"${_include_dir}\"
)
"
)
export(
TARGETS ${_target}
FILE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Targets.cmake"
EXPORT_LINK_INTERFACE_LIBRARIES
)
export(PACKAGE ${_target})
endfunction(my_export_target)
C/CMakeLists.txt
include(MyCommonCode.cmake)
...
my_export_target(C "${CMAKE_CURRENT_SOURCE_DIR}/include")
B/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(C REQUIRED)
...
target_link_libraries(B C)
my_export_target(B "${CMAKE_CURRENT_SOURCE_DIR}/include")
A/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(B REQUIRED)
...
target_link_libraries(A B)
This keeps all 3 build environments separate, only sharing the relatively static MyCommonCode.cmake file. So in this approach I have so far not covered your first point, but would recommend the use of a external script to chain/trigger your build steps for A/B/C.