I am creating a project that is built using CMake requires some special packaging afterwards. I would like to ensure that the build is release and that every part has been built in release. I wrote the build type using the following code:
FUNCTION ( GetCMakeBuildType ret)
# Returns the generator string to get the current build type:
SET(${ret} "$<$<CONFIG:MinSizeRel>:MinSizeRel>$<$<CONFIG:Debug>:Debug>$<$<CONFIG:Release>:Release>$<$<CONFIG:RelWithDebInfo>:RelWithDebInfo>" PARENT_SCOPE)
ENDFUNCTION()
FUNCTION ( WriteBuildType )
GetCMakeBuildType(buildTypeString)
ADD_CUSTOM_TARGET(RecordBuildType ALL COMMAND echo ${buildTypeString} > ./target/build_type.txt)
ENDFUNCTION()
But I would also like to detect if ALL the libraries were built by determining if ALL_BUILD was the target or if they only built a single library while the rest is in debug. Can I detect if the user ran ALL_BUILD and write a flag to file to note that packaging is okay?
Related
I have a project that depends on Intel's oneTBB. My project is structured as follows:
external/
| - CMakeLists.txt
| - oneTBB/ (this is a git submodule)
| - ...
include/
lib/
include/
CMakeLists.txt
I currently get things to compile by manually building oneTBB and installing it inside a prefix directory located at external/oneTBB/prefix by running the following (bash) commands:
cd oneTBB
mkdir -p prefix
mkdir -p build
cd build
cmake -DCMAKE_INSTALL_PREFIX=../prefix -DTBB_TEST=OFF ..
cmake --build .
cmake --install .
I then simply include and link using this prefix. (I got this from following the oneTBB READMEs)
While this works without issue, I'm currently trying to clean up my CMake such that its easier to build on Windows as well. Ideally, I'm looking to get to a point where I can simply run:
mkdir build
cd build
cmake ..
cmake --build .
and my project will build itself and all dependencies.
I got this working with other dependencies such as glfw and eigen by simply adding (to the CMakeLists.txt in external/:
add_subdirectory(glfw)
add_subdirectory(eigen)
But adding add_subdirectory(oneTBB) throws a LOT of warnings, starting with:
CMake Warning at external/oneTBB/CMakeLists.txt:116 (message):
You are building oneTBB as a static library. This is highly discouraged
and such configuration is not supported. Consider building a dynamic
library to avoid unforeseen issues.
-- TBBBind build targets are disabled due to unsupported environment
-- Configuring done
CMake Warning (dev) at external/oneTBB/src/tbb/CMakeLists.txt:15 (add_library):
Policy CMP0069 is not set: INTERPROCEDURAL_OPTIMIZATION is enforced when
enabled. Run "cmake --help-policy CMP0069" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
INTERPROCEDURAL_OPTIMIZATION property will be ignored for target 'tbb'.
This warning is for project developers. Use -Wno-dev to suppress it.
I have no need to build oneTBB as a static library.
Am I doing something wrong in my attempt? Really all I need is for oneTBB to be built as a dynamic library and placed somewhere I can link it to (without installing it on the system overall)
Similar question:
I am also including the METIS library which depends on GKlib. Currently I'm doing this in a similar way to what I did for oneTBB where I manually build each using the following script:
# Setup the GKlib library:
cd GKlib
mkdir -p prefix
mkdir -p build
cd build
cmake -DCMAKE_INSTALL_PREFIX=../prefix ..
cmake --build . -j
cmake --install .
cd ../../
# Setup the METIS library:
cd METIS
mkdir -p prefix
make config prefix=../prefix gklib_path=../GKlib/prefix #(GKLib path is done from root, install path done relative to build)
make install -j
cd ../
When I try to add them using:
add_subdirectory(GKlib)
add_subdirectory(METIS)
it throws errors that METIS cannot find GKlib:
CMake Error at external/METIS/CMakeLists.txt:50 (add_subdirectory):
add_subdirectory given source "build/xinclude" which is not an existing
directory.
While I recognize this is a separate issue, I figured to include it here as it is related to my issues with add_subdirectory()
Many projects expect that you invoke CMake on them separately instead of adding them into an existing project with add_subdirectory. While there might be a way to make add_subdirectory work with oneTBB, there is an easier way.
CMake has a function that allows you to download, build, and install external projects at build time: ExternalProject_Add.
Here's an example for spdlog, taken straight from one of my own projects:
# project_root/thirdparty/spdlog/CMakeLists.txt
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UPPER)
ExternalProject_Add(spdlog-project
GIT_REPOSITORY https://github.com/gabime/spdlog
GIT_TAG edc51df1bdad8667b628999394a1e7c4dc6f3658
GIT_SUBMODULES_RECURSE ON
GIT_REMOTE_UPDATE_STRATEGY CHECKOUT
INSTALL_DIR "${CMAKE_CURRENT_BINARY_DIR}/install"
LIST_SEPARATOR |
CMAKE_CACHE_ARGS
"-DCMAKE_CXX_FLAGS_${CMAKE_BUILD_TYPE_UPPER}:STRING=${CMAKE_CXX_FLAGS_${CMAKE_BUILD_TYPE_UPPER}}"
"-DCMAKE_C_FLAGS_${CMAKE_BUILD_TYPE_UPPER}:STRING=${CMAKE_C_FLAGS_${CMAKE_BUILD_TYPE_UPPER}}"
"-DCMAKE_EXE_LINKER_FLAGS_${CMAKE_BUILD_TYPE_UPPER}:STRING=${CMAKE_EXE_LINKER_FLAGS_${CMAKE_BUILD_TYPE_UPPER}}"
"-DCMAKE_SHARED_LINKER_FLAGS_${CMAKE_BUILD_TYPE_UPPER}:STRING=${CMAKE_SHARED_LINKER_FLAGS_${CMAKE_BUILD_TYPE_UPPER}}"
"-DCMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE}"
"-DCMAKE_INSTALL_PREFIX:STRING=<INSTALL_DIR>"
"-DSPDLOG_BUILD_EXAMPLE:BOOL=OFF"
)
add_library(ext-spdlog INTERFACE)
add_dependencies(ext-spdlog spdlog-project)
ExternalProject_Get_property(spdlog-project INSTALL_DIR)
target_include_directories(ext-spdlog SYSTEM INTERFACE "${INSTALL_DIR}/include")
target_link_directories(ext-spdlog INTERFACE "${INSTALL_DIR}/lib")
target_link_libraries(ext-spdlog INTERFACE spdlog$<$<CONFIG:Debug>:d>)
After that, my project simply links against the created library target:
target_link_libraries(my_project ext-spdlog)
To adapt this for oneTBB, you have to switch out the repository URL and commit hash, and add your own CMake definitions (i.e. "-DTBB_TEST=OFF"). Depending on how oneTBB has its include and library directories set up, you may also have to change the target_include_directories and/or target_link_directories lines. I haven't looked this up yet, but I'm sure you can figure it out.
This works regardless of whether the external project is built as a static or shared library. You shouldn't use git submodules, though - instead, let CMake do the downloading. (It'll only download and build once; subsequent builds will not re-build the external project if it's already built and up-to-date.)
I have no need to build oneTBB as a static library. Am I doing something wrong in my attempt? Really all I need is for oneTBB to be built as a dynamic library and placed somewhere I can link it to (without installing it on the system overall)
All your diagnostic messages indicate that it's actually being configured to be built as a static library, and additional clues point to the probability that you've set BUILD_SHARED_LIBS to false in the scope where you add_subdirectory(oneTBB).
CMake Warning at external/oneTBB/CMakeLists.txt:116 (message):
You are building oneTBB as a static library. This is highly discouraged
and such configuration is not supported. Consider building a dynamic
library to avoid unforeseen issues.
If you look in oneTBB's CMakeLists.txt file, you'll the following:
if (NOT DEFINED BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON)
endif()
if (NOT BUILD_SHARED_LIBS)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
message(WARNING "You are building oneTBB as a static library. This is highly discouraged and such configuration is not supported. Consider building a dynamic library to avoid unforeseen issues.")
endif()
And then right after that, you get
-- TBBBind build targets are disabled due to unsupported environment
The corresponding section of oneTBB's CMakeLists.txt file is:
if (TBB_FIND_PACKAGE OR TBB_DIR)
...
else()
if (APPLE OR NOT BUILD_SHARED_LIBS)
message(STATUS "TBBBind build targets are disabled due to unsupported environment")
else()
add_subdirectory(src/tbbbind)
endif()
...
Both of these clues indicate that in the variable scope at which you add_subdirectory(oneTBB), BUILD_SHARED_LIBS is set to a falsy value.
Set BUILD_SHARED_LIBS it to a truthy value (Ex. 1, TRUE, YES, ON, etc.) before doing add_subdirectory(oneTBB) and then restore the previous value afterward.
Ex.
set(BUILD_SHARED_LIBS_TEMP "${BUILD_SHARED_LIBS}")
set(BUILD_SHARED_LIBS YES)
add_subdirectory(oneTBB)
set(BUILD_SHARED_LIBS "${BUILD_SHARED_LIBS_TEMP}")
unset(BUILD_SHARED_LIBS_TEMP)
So, I've found really strange behaviour in CMake creating dependency on target_link_library..
It's hard to explain in one sentence, so here is a list of requirements (I hope this all will make sence in the end)
your project must have custom build type defined through CMAKE_CONFIGURATION_TYPES ('Tools' in this example)
you must have at least 3 targets:
executable (or simply main target) (test-exe in this example)
interface library which link to main target (this could be something other than INTERFACE library, but the next target must be linked to it via interface property only) (test-env in this example)
static library which links to the interface library with specific generator expression, which is depends on custom build type (something like that 'target_link_libraries(test-env INTERFACE $<$CONFIG:Tools:test-lib>)') (test-lib in this example)
Here is the code of the CMakeLists.txt file to make it little bit clearer:
cmake_minimum_required(VERSION 3.19.0 FATAL_ERROR)
project(multiconfiguration-test LANGUAGES CXX)
set(CMAKE_CONFIGURATION_TYPES Debug Release Tools)
set(CMAKE_CXX_FLAGS_TOOLS ${CMAKE_CXX_FLAGS_DEBUG})
set(CMAKE_EXE_LINKER_FLAGS_TOOLS ${CMAKE_EXE_LINKER_FLAGS_DEBUG})
set(CMAKE_STATIC_LINKER_FLAGS_TOOLS ${CMAKE_STATIC_LINKER_FLAGS_DEBUG})
add_library(test-env INTERFACE)
# EXCLUDE_FROM_ALL used to not build this target by default
add_library(test-lib STATIC EXCLUDE_FROM_ALL "test-lib.cpp")
target_link_libraries(test-env INTERFACE $<$<CONFIG:Tools>:test-lib>)
add_executable(test-exe "test-exe.cpp")
target_link_libraries(test-exe PRIVATE test-env)
(Files test-exe.cpp and test-lib.cpp are trivial, test-lib.cpp has a simple function, and test-exe.cpp declares this function and then calls it from main.)
When you would try to build this project with visual studio 2019/2017 generators (with "Tools" as configuration type of course: cmake --build . --config Tools), you will have next error:
LINK : fatal error LNK1104: cannot open file 'Tools\test-lib.lib' [<none_important_path_to_the_project>\test-exe.vcxproj]
And, what is important, you will not see in the terminal target test-lib being build.
So, what happened is target test-exe knows it must be linked against test-lib, but the build system doesn't know that target test-exe is dependent on target test-lib.
And now the most strange thing! If we will link this library like that: target_link_libraries(test-env INTERFACE $<$<CONFIG:Debug>:test-lib>) (so build type must be Debug), and still build project with Tools as a build type, you will see in the terminal that target test-lib is now building! (yes we have link error because test-exe can't find the function which is defined in test-lib, but this is at least expected)
So, what actually happens, the link flag of the target test-exe is correctly depends on the generator expression BUT, the actual build dependency, somehow, transforms any custom build type in this generator expression to the Debug.
Again this only happens with requirements I pointed up above, so it's not only the fault of generator expression, it's also connected to the interface dependency as well..
If we can't break any of the requirements, one possible solution will be to add direct dependency of test-lib target to test-env (like that: add_dependecies(test-env test-lib)), but this is not perfect, because even if we will use test-lib only then where is Tools as build type, we still will build this library each time, which can be undesired behavior.
I'm not really asking for solution here (but if you have one please share), I'm asking for the explanation why this even happening? Is this a bug or a really strange feature?
EDIT Small update I've found just now:
It must not even be the custom build type. The same bug can be encountered even with Release build type, so the final code can look as simple as this:
cmake_minimum_required(VERSION 3.19.0 FATAL_ERROR)
project(multiconfiguration-test LANGUAGES CXX)
add_library(test-env INTERFACE)
add_library(test-lib STATIC EXCLUDE_FROM_ALL "test-lib.cpp")
target_link_libraries(test-env INTERFACE $<$<CONFIG:Release>:test-lib>)
add_executable(test-exe "test-exe.cpp")
target_link_libraries(test-exe PRIVATE test-env)
and be build with next command: cmake --build . --config Release
Looks like a bug with the Visual Studio generator to me. I've just tested the Ninja Multi-Config generator both on Linux and on Windows and there cmake --build <build-dir> --config Release works just fine.
My configuration has CMake 3.6, Visual Studio 2015 and latest Google test from GitHub. I add my unit tests through one of my cmake functions addGtest and do the build. After this I can run the test from my RUN_TESTS target or using ctrl + F5 in VS and works as expected.
The final goal is to run the unit tests at build time using the CMake dependency management. For now, as a first step, I have enhanced my function to create a custom_target (included the entire function, in case there are unforeseen issues in the working part), but not build it:
function (addGtest)
# vvvv this part works as explained vvvv #
set (optBOOLS)
set (optSINGLES EXE)
set (optLISTS DLL_LIST)
cmake_parse_arguments (myARGS
"${optBOOLS}" "${optSINGLES}" "${optLISTS}" ${ARGN})
# addExecutable is a function that adds target executables
set(myARGS_DLL_LIST gtest_main gtest "${myARGS_DLL_LIST}")
addExecutable (EXE ${myARGS_EXE} DLL_LIST ${myARGS_DLL_LIST} ${myARGS_UNPARSED_ARGUMENTS})
add_test (NAME ${myARGS_EXE} COMMAND ${myARGS_EXE} WORKING_DIRECTORY
${CMAKE_INSTALL_PREFIX}/$<$<CONFIG:Release>:Release>$<$<CONFIG:Debug>:Debug>/bin
) # so it can be run using ctest
# ^^^^ this part works as explained ^^^^ #
add_custom_target (${myARGS_EXE}.tgt DEPENDS ${myARGS_EXE}
COMMAND ${myARGS_EXE} --gtest_output="xml:${myARGS_EXE}.xml"
WORKING_DIRECTORY ${CMAKE_INSTALL_PREFIX}/$<$<CONFIG:Release>:Release>$<$<CONFIG:Debug>:Debug>/bin
)
endfunction (addGtest)
As expected when I perform the build, a new target, say, utMyTest.tgt is added to VS, but it is not built. Now when I build this new target by hand in VS, I expect that the test will be run. But it doesn't and gives the following error:
1> The filename, directory name, or volume label syntax is incorrect.
I tried providing full path to the COMMAND option, removing double quotes around --gtest_output value, but to no avail. On the other hand when I cd to the working directory in a command line window and invoke the exe, it works fine!!
The first question is how do I fix it to run the test by building this new target? After that, I plan to add_custom_target (${myARGS_EXE}.run) and add_dependencies (${myARGS_EXE}.run ${myARGS_EXE}.tgt). Would this then run the test whenever the exe changes? Or should I do something else? Thank you for your help.
Could not add so much details in the comment, hence this answer.
1. Answer to the original problem
Since I needed the configuration dependent path in the WORKING_DIRECTORY option of the add_custom_target command, but cannot pass generator expressions to it, the idea is to use the CMAKE_CFG_INTDIR variable so:
add_custom_target (${myARGS_EXE}.tgt
DEPENDS ${myARGS_EXE}
COMMAND ${myARGS_EXE} --gtest_output=xml:${myARGS_EXE}.xml
WORKING_DIRECTORY ${CMAKE_INSTALL_PREFIX}/${CMAKE_CFG_INTDIR}/bin
)
Now, when you build the above target, the unit test is run in the WORKING_DIRECTORY which is not entirely desirable, since that is the install directory for libs and exes. It would be really nice to ...
2. Run the unit test from its build directory
While, at the same time, picking up the DLL paths from within Visual Studio, and storing the Gtest generated .xml file in the build directory. This is the solution:
In CMake version 3.10 CMAKE_MSVCIDE_RUN_PATH property was added. In the project wide CMakeLists.txt, set(CMAKE_MSVCIDE_RUN_PATH ${CMAKE_INSTALL_PREFIX}/${CMAKE_CFG_INTDIR}/bin) - thanks to this solution #3, we can appendPATH to point to our install directory. And then replace the above add_custom_target command with this:
add_custom_command (
TARGET ${myARGS_EXE} POST_BUILD
COMMAND ${myARGS_EXE} --gtest_output=xml:${myARGS_EXE}.xml
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_CFG_INTDIR}
)
This solution avoids the mess of creating additional targets. Clearly only when myARGS_EXE is built, the unit test is run. Obviously myARGS_EXE's transitive dependency on other DLLs is covered also.
If you have other elegant solutions, please post.
I am on Visual Studio 2013, CMake 3.5.1, Windows 10. I am trying to copy some files via CMake like below:
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/Release)
Is it possible to replace "Release" with a variable that represents the configuration like:
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/${Variable})
I attempted
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/${CMAKE_BUILD_TYPE})
but CMAKE_BUILD_TYPE is an empty string when I use message to print it out, I also attempted
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/$<CONFIGURATION>)
but for some reason file command cannot decipher $<CONFIGURATION> whereas command like
add_custom_target(run COMMAND ${CMAKE_BINARY_DIR}/bin/$<CONFIGURATION>/Test.exe)
can. What is the right way to extract whether visual studio is currently built in Release or Debug in CMake?
The file command is executed during CMake runtime, not during build time (i.e. VS runtime).
This also means, that the generator expressions (e.g. $<CONFIG>) can not be used, as these are evaluated during build time.
(Hint: As long as there is no explicit reference to the use of generator expressions for a particular command in the CMake docu, they are not supported by that command).
The reason, why ${CMAKE_BUILD_TYPE} is empty, is due to the reason that you probably haven't specified it on the invocation of CMake:
cmake -DCMAKE_BUILD_TYPE=Debug ..
However, using that, would mean that the build files are only generated for the Debug configuration. That's obviously not what you want.
To solve your problem: Using generator expressions is the right way, as you've already figured out with the use of add_custom_target (or add_custom_command).
You can use custom commands as dependencies for other "real" targets and you can specify post-/pre-build and pre-link commands for a specific target via add_custom_command.
As the docu states for the COMMAND argument of add_custom_command:
Arguments to COMMAND may use generator expressions. References to target names in generator expressions imply target-level dependencies, but NOT file-level dependencies. List target names with the DEPENDS option to add file-level dependencies.
To copy a file after a successful build of a target:
add_custom_command(TARGET myTarget POST_BUILD
COMMAND "${CMAKE_COMMAND}" -E copy_if_different "${IMAGE1}" "${CMAKE_BINARY_DIR}/bin/$<CONFIG>/"
COMMAND "${CMAKE_COMMAND}" -E copy_if_different "${IMAGE2}" "${CMAKE_BINARY_DIR}/bin/$<CONFIG>/"
)
I apologize if this has been asked already but I can't find online the right way to get this thing to work.
I have a cmake project Foo which depends on another cmake project Bar. Goal: whenever Bar is reinstalled (changing only the libraries, not the headers), then Foo should re-link (without rebuilding, of course).
So, in the CMakeLists.txt in the top folder of project Foo (which has only one target, an executable) I have the cmake command
FIND_PACKAGE(Bar REQUIRED)
Which, correctly, finds project Bar at configure time. In the part where I create the target, I have
LINK_DIRECTORIES (${BAR_LIBRARY_DIRS} )
ADD_EXECUTABLE(foo.exe main.cpp)
TARGET_LINK_LIBRARIES(foo.exe ${BAR_LIBRARIES})
The two variables in there are defined in BarConfig.cmake, which is the one that FIND_PACKAGE(Bar) looks for, and contains only the following instructions
SET(BAR_INCLUDE_DIRS "${CMAKE_CURRENT_LIST_DIR}/../../../include")
SET(BAR_LIBRARY_DIRS "${CMAKE_CURRENT_LIST_DIR}/../../../lib")
SET(BAR_LIBRARIES bar)
I printed on screen those variables and they contain what you expect them to contain (/path/to/folder/include, /path/to/folder/lib and bar).
So, I read somewhere that cmake cannot add a dependency to a library which is not specified with its complete path. One should write
TARGET_LINK_LIBRARIES (foo.exe full-path-to-bar-libraries)
In that case, it works. But it's unsatisfactory. First, cause the path can change. But you can read it from a variable, you might say. True. But, second, even in that case, if the project Bar contains a number of libraries that is not known, one would have to sweat to create the correct string to add there...
However, I also read that if that library is also built and installed with cmake, it should work automatically. As a matter of fact, I have another project pair, A depends on B, both built with cmake. In that case the dependence works. Unfortunately, the project B is HUGE, and defines TONS of cmake macros, and I can't identify the part where it sets up the right variables.
Do you have any idea of how to get Foo to re-link (without rebuilding) every time that the library Bar is reinstalled? I would like to avoid to use full path.
Thanks
Edit: to be more clear: if library Bar set up a variable BAR_LIBRARIES containing all its libraries with the full path, then TARGET_LINK_LIBRARIES would work. However, most likely BAR_LIBRARIES would contain 'bar', rather than '/some/path/libbar.a'. I'd like cmake to be able, given the directories provided with LINK_DIRECTORIES and the library names provided with TARGET_LINK_LIBRARIES to put the two pieces together. E.g. if BAR_LIBRARY_DIRS contains '/folder1/;/folder2/' and BAR_LIBRARIES contains 'bar1;bar2', I'd like cmake to establish a dependency on libbar1.a and libbar2.a, found in any of the following:
/folder1/libbar1.a
/folder2/libbar1.a
/folder1/libbar2.a
/folder2/libbar2.a
And relink if any of the ones upon which a dependency has been created is changed since the last linking.
and contains only the following instructions
You don't need to set all this variables manually, just use CMakePackageConfigHelpers module:
configure_package_config_file(
"./FooConfig.cmake.in"
"${foo_config}"
INSTALL_DESTINATION ${CONFIG_INSTALL_DESTINATION}
PATH_VARS CONFIG_INSTALL_DESTINATION
)
install(
FILES "${foo_config}"
DESTINATION ${CONFIG_INSTALL_DESTINATION}
)
and install targets:
install(
TARGETS foo DESTINATION ${LIB_INSTALL_DESTINATION} EXPORT FooTargets
)
install(
EXPORT FooTargets NAMESPACE Foo:: DESTINATION ${CONFIG_INSTALL_DESTINATION}
)
Usage in other project:
find_package(Foo CONFIG REQUIRED)
target_link_libraries(boo Foo::foo)
That's all. Work like a charm (Makefile generator example):
> cmake -HFoo -B_builds/Foo -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="`pwd`/_install"
> cmake --build _builds/Foo/ --target install
> cmake -HBoo -B_builds/Boo -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="`pwd`/_install" -DCMAKE_VERBOSE_MAKEFILE=ON
> cmake --build _builds/Boo
One more time (check no relink):
> cmake --build _builds/Boo
Modify target Foo and reinstall:
> echo "void some(){}" >> Foo/foo.cpp
> cmake --build _builds/Foo/ --target install
Now build Boo:
> cmake --build _builds/Boo
...
Linking CXX executable boo.exe
...
Relink!