TLDR: I would like to ask CMake to wait for ExternalProject_Add to complete before it attempts to move on to the next subdirectory and build a library that happens to use one of the files of the external project. In other words, I like to declare an external project as a dependency for a shared library.
More Description:
Suppose that my CMake project has two directories: thirdparty and src. My top-level CMakeLists.txt file has:
add_subdirectory(thirdparty)
add_subdirectory(src)
thirdparty/CMakeLists.txt contains multiple ExternalProject_Add commands. My original intention was to pull and build all these external projects and then move on to building my own libraries and executables in src. Unfortunately, this didn't go as I planned:
One of my external projects is called libsvm. And my src/CMakeLists.txt has the following:
set(Libsvm_SOURCE_FILES
${PROJECT_BINARY_DIR}/thirdparty/libsvm/src/libsvm/svm.cpp
)
include_directories(
${Libsvm_INCLUDE_DIR}
)
add_library(
mysvm
SHARED
${Libsvm_SOURCE_FILES}
)
Now the problem I am facing with is that CMake is unable to find ${Libsvm_SOURCE_FILES}, apparently because this step is being executed before the ExternalProject_Add in my thirdparty/CMakeLists.txt file is executed.
I would like to declare this external project as a dependency for this library.
Broader Question:
Is there a clean way to force CMake to finish everything in first subdirectory before moving on to the next? If not, do you recommend that I make any change in the hierarchy and organization of my CMakeLists files?
Thanks!
CMake expects every source file, passed to add_library or add_executable, to be existed unless it is marked as GENERATED. This property is automatically set for files listed as OUTPUT for add_custom_command. In other cases one need to set this property explicitly:
set_source_files_properties(${Libsvm_SOURCE_FILES} PROPERTIES GENERATED TRUE)
Related
I tries to use FOREACH to generated several pb files. And make two list names PROTO_SRCS & PROTO_HDRS like below.
I can use it in the main CMakeLists. Like add_executable(a SHARED ${PROTO_SRCS} main.cpp).
But I can not use this param in subdirectories to make a library. when I type "cmake .." in main CMakelists build dir. It shown that "Cannot find source file: a.pb.cc".
main CMakeLists.txt
add_library(xxx SHARED ${PROTO_SRCS})
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/src/back back)
in src/back CMakeLists.txt
add_executable(yyy ${PROTO_SRCS})
and I can use message to show ${PROTO_SRCS} in subdir so it pass into successfully.
Please help me to point out the problem. Thx a lot
The issue is that in CMake versions older than 3.20 the GENERATED property of source files is only visible in the directory where it is set. Thus, when you add the protobuf-generated source files to a target defined in a different directory, CMake will no longer know that these are files generated during the build. Consequently, CMake will try to locate these files at configuration time, when they obviously do not exist yet.
Unfortunately, at the time of writing there is only a release candidate for CMake 3.20 and no official release yet. So depending on whether you need to coordinate with other coworkers or whether you're working on this project on your own it might not be feasible to use the release candidate.
If you can't use it, the alternative is to create an object library via add_library(protobuf_objs OBJECT ${PROTO_SRCS}) in the directory where you generate the files and to use target_sources(xxx PRIVATE $<TARGET_OBJECTS:protobuf_objs>) and target_sources(yyy PRIVATE $<TARGET_OBJECTS:protobuf_objs>) instead of adding the ${PROTO_SRCS} as source files to these targets directly.
I have a cross-compiler cmake project that depends on libraries from a separate project that happens to also use cmake:
/myProject/CMakeLists.txt (uses cross-compiler)
/anotherProject/CMakeLists.txt (platform-agnostic)
anotherProject can be built completely separately on its own. It has no knowledge of myProject at all.
Now, anotherProject has numerous modules that I need, like:
anotherProject/A/CMakeLists.txt (produces static lib A.a)
anotherProject/B/CMakeLists.txt (produces static lib B.a)
etc
When I build myProject, I want to build and link against anotherProject/A and anotherProject/B, to produce shared lib myproject.so. I'd like to leverage the existing cmake-ness of anotherProject if possible, as opposed to manually globbing its various source sets from myProject.
What's the correct way to achieve this with cmake? I feel like I'm missing something obvious.
It would be straightforward if, say, myProject were just a subdirectory under anotherProject, or if there were a top-level CMakeLists.txt that could reference both myProject and anotherProject; but neither is what I'm after. I know I could build anotherProject and export its libraries to a well-known location, and then reference the export directory from myProject - but I would like to avoid that setup as well.
A solution is to use CMake packages.
Basically, in anotherProject, you craft a CMake configuration file where you set variables to be used by myProject (eg. include directory, list of libraries, compilation flags...), or even targets.
Then, in myProject, you use the find_package() mechanism so that CMake finds this configuration file and imports the variables/targets in your current project.
There is a tutorial on the CMake wiki.
The only alternative setup that I can think of based on your requirements is to allow your main (dependent) project to discover the other (dependee) project using find_package.
In your main project CMakeLists.txt you should add something like this:
find_package(anotherProject CONFIG)
if(anotherProject_FOUND)
message(STATUS "Found project dependency: anotherProject")
else
# change WARNING to FATAL_ERROR if the dependency is NOT optional
message(WARNING "package anotherProject was not found")
endif()
On the differences between CONFIG and MODULE modes, check the documentation and this link.
Then assuming that your main project creates an executable, you could hook up the discovered dependency like this:
add_executable(myProject ${SOURCES})
[...]
if(anotherProject_FOUND)
target_link_libraries(myProject PUBLIC anotherProject)
endif()
This should take care of the required include files and definitions as well.
Now in the dependee project CMakeLists.txt you should do something like this:
set(PRJ_NAME "anotherProject")
string(TOLOWER ${PRJ_NAME} PRJ_NAME_LOWER)
set(ANOTHERPROJECT_EXPORT_NAME "${PRJ_NAME}")
install(TARGETS ${PRJ_NAME} EXPORT ${ANOTHERPROJECT_EXPORT_NAME}
RUNTIME DESTINATION .)
install(EXPORT ${ANOTHERPROJECT_EXPORT_NAME} DESTINATION "share/cmake")
This associates an export with a target and then installs the export.
Now, if you check that export file, it expects certain things to be found and included, that could be specific for your project. To make this as supple as possible, you can use the configure feature to generate them from a template and then install from the build directory.
So, in the project under a subdir named share/cmake you could have a file named config.cmake.in with contents:
include(${CMAKE_CURRENT_LIST_DIR}/#PRJ_NAME#.cmake)
In the main project's CMakeLists.txt you need to add the following for generating the file from that template:
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/share/cmake/config.cmake
${CMAKE_CURRENT_BINARY_DIR}/share/cmake/${PRJ_NAME_LOWER}-config.cmake)
install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/share/
DESTINATION share)
Notice that I used PRJ_NAME, because you could potentially reuse that to name the actual executable at the add_executable command. It mentally helps if the exported target has the same name with produced one.
This is a more versatile version to accommodate multiple subprojects of this tutorial.
I added the xgboost library as a git submodule of my project and I'm trying to add it to cmake as a subdirectory. Unfortunately it's not working. A simple hello world project with the following CMakeLists.txt replicates the error that I'm getting.
cmake_minimum_required(VERSION 3.2)
project(foo)
add_subdirectory(xgboost)
add_executable(${PROJECT_NAME} foo.cpp)
target_link_libraries(${PROJECT_NAME} xgboost)
After building the library there is nothing in the xgboost/lib directory so I get the following error.
clang: error: no such file or directory:
'/Users/.../myproject/xgboost/lib/libxgboost.dylib'
I think that the problem is generated in their CMakeLists file since they have two different targets. Maybe cmake is choosing the wrong target but I'm not familiar enough with cmake to figure it out. The following code is from xgboost's CMakeLists.
# Executable
add_executable(runxgboost $<TARGET_OBJECTS:objxgboost> src/cli_main.cc)
set_target_properties(runxgboost PROPERTIES
OUTPUT_NAME xgboost
)
set_output_directory(runxgboost ${PROJECT_SOURCE_DIR})
target_link_libraries(runxgboost ${LINK_LIBRARIES})
# Shared library
add_library(xgboost SHARED $<TARGET_OBJECTS:objxgboost>)
target_link_libraries(xgboost ${LINK_LIBRARIES})
set_output_directory(xgboost ${PROJECT_SOURCE_DIR}/lib)
#Ensure these two targets do not build simultaneously, as they produce outputs with conflicting names
add_dependencies(xgboost runxgboost)
My questions in order of importance are:
Is there any way to fix it without modifying xgboost's CMakeLists.txt file?
Is it reasonable to try to add xgboost to my project as a git submodule?
Is there any reason cmake is not instructing to build the library?
Note: There were several edits to this question since I tried to narrow down the problem and to provide more information.
(I would love to ask for few things beforehand in the comment section, but I have too low reputation to do so, so I will just give it a shot ;))
I have few suspects, and one of them is ${CMAKE_SOURCE_DIR} of the submodule's root CMakeLists.txt. Although the paths are set properly when you run that CMakeLists.txt alone, cmake gets confused the moment you add it as your subdirectory. Have you looked into another directories for your output binaries?
First I would suggest testing this hypothesis, and then I would suggest writing similar, but separate CMakeLists.txt file for xgboost library, and then substitute it in the project temporarily. Unfortunately the CMakeLists.txt filename is hardcoded and there is no possibility to have two files of that kind in one directory; so it seems that the answer to 1) is, that you rather have to change the file.
For the 2): as long as it does not require huge additional logic in your CMakeLists.txt, it makes sense. Other viable option is to create an install target, which you can use to install your xgboost library locally (using CMAKE_INSTALL_PREFIX(doc) variable), and then add the installation path to your CMAKE_LIBRARY_PATH(doc).
I'd like to add an external dependency to my project. The one I'm trying to add is the Leptonica library as a submodule.
My project has the following directory structure:
|root
CMakeLists.txt
|-bin
|-build
|-buildsystem
|-executable
|-leptonica
|--CMakeLists.txt
|--cmake
|---Configure.cmake
|-production
In my root CMakeLists.txt file I added ADD_SUBDIRECTORY(${ROOT_DIR}/leptonica)
Unfortunately, CMake is not searching for Configure.cmake in the proper directory:
CMake Error at leptonica/CMakeLists.txt:107 (include):
include could not find load file:
Configure
CMake Error: File
<root>/cmake/templates/LeptonicaConfig-version.cmake.in does not exist.
CMake Error at leptonica/CMakeLists.txt:113 (configure_file):
configure_file Problem configuring file
When I build the project by myself, everything goes fine. In my opinion, the problem is with CMAKE_SOURCE_DIR. When using add_subdirectory it has the value of ROOT CMake instead ROOT/leptonica, so it's searching the wrong paths - as you can see in Leptonica CMake, it's used to determinate paths of its files.
What should be the proper way to fix this - should I set CMAKE_SOURCE_DIR to ROOT/leptonica just before calling add_subdirectory and set it back when it's finished, or does some other, more elegant solutions exist?
Not every CMake project is suitable for inclusion via add_subdirectory.
Among those are projects which uses CMAKE_SOURCE_DIR or CMAKE_BINARY_DIR variables.
However, inclusion via ExternalProject_Add (optionally wrapped with execute_process) always works.
Modifying variable CMAKE_SOURCE_DIR (and CMAKE_BINARY_DIR too) is a bad idea: this variable should be changed only by CMake itself. Otherwise you may get weird errors.
Instead, you may replace (automatically, with some script) all references to the variable with another variable, which is not used in the project. This new variable you may safely set before stepping into the subproject.
${CMAKE_SOURCE_DIR} and ${CMAKE_BINARY_DIR} are set relative to the top-level CMakeLists.txt. If you need something relative to your current CMakeLists.txt (leptonica), use ${CMAKE_CURRENT_SOURCE_DIR} and ${CMAKE_CURRENT_BINARY_DIR}.
If you're having trouble finding a cmake file like LeptonicaConfig-version.cmake.in, try appending the appropriate directory to ${CMAKE_MODULE_DIR}.
list(APPEND ${CMAKE_MODULE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/cmake/templates)
I prefer to use ${CMAKE_CURRENT_SOURCE_DIR} over ${CMAKE_SOURCE_DIR} any day because using the latter will break your build if you try to integrate it into a super-build later. If I need to pass my current top-level directory to subdirectories, then I do the following and use that later down the chain.
set( LEPTONICA_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR} )
The layout of my project is as follow :
src/
include/
include1.h
include2.h
include3.h
lib/
lib1/
source1_lib1.c
source2_lib1.c
lib2/
source1_lib2.c
source2_lib2.c
source3_lib2.c
lib3/
source1_lib3.c
lib4/
source1_lib4.c
source2_lib4.c
module_A/ (this module will need lib1 and lib4)
source1_moduleA.c
source2_moduleA.c
module_B/ (this module will need lib2 and lib3)
source1_moduleB.c
source2_moduleB.c
source3_moduleB.c
module_C/ (this module will need lib1, lib2, lib3 and lib4)
source1_moduleC.c
module_D/ (this module will need lib1 and lib3)
source1_moduleD.c
source2_moduleD.c
source3_moduleD.c
source4_moduleD.c
The global solution can be made by any number of module_X (it depends on the customer)
My project CMake file located under "/src" includes a configuration file (it is defined by a customer needs). This configuration file indicates which modules must be built and packaged to the target customer.
Let's say I have a customer X and the modules he selected are module_A and module_D. In this case my build system should only builds lib_1, lib_3 and lib_4.
What I am looking for is a way to define target libraries without being built until I do reference them in one the CMakeLists files under module_X directories.
Oh my bad I missed to say A BIG THANKS FOR YOUR HELP
CMake can not do this out of the box. Any library that is added through an add_library call will be built.
But you can implement the behavior you want inside the CMake. Instead of calling add_library and add_executable directly you would call custom wrapper functions. The add_library wrapper simply stores the information required to call add_library, but does not call it directly. The add_executable wrapper iterates over all dependencies of the executable and makes the call to add_library if required.
You will have to design your own system for maintaining the state whether a particular library has already been added and what the parameters are for constructing the library target. Things get slightly more difficult when respecting transient dependencies as well (module_1 depends on lib_a which in turn depends on lib_b; but no executable depends on lib_b directly). But it is perfectly possible to build such a system with a few hundred lines of CMake code.
cmake ..
make module_A module_D