I have the following situation with CMake:
it has to build two applications:
a. generator
b. something_else
The generator is nothing fancy, a few CPP files linked together
The something_else is:
a. a few "normal" CPP files
b. generated CPP/h files which I have to link in. These CPP files are generated by the generator
The generator is configured in the configure phase with some choices, depending on these options the content of the generated files is different.
And here I get an error: If I specify all the files (generated and not) in the add_application of something_else the configure phase chokes on it since it cannot find the generated files... obviously because they were not generated yet, since the generator was not built and executed yet.
So the question: Is this possible using CMake? If yes, how?
Yes, it is possible. You do that by providing a custom command to generate the files, so that CMake learns how to generate them. Here's an example:
add_executable(generator gen1.cpp gen2.cpp)
add_custom_command(
OUTPUT generated_file1.cpp generated_file2.cpp
COMMAND generator -options --go here
COMMENT "Running generator"
VERBATIM
)
add_executable(something_else
fixed1.cpp
fixed2.cpp
generated_file1.cpp
generated_file2.cpp
)
This way, CMake will know the files are generated, and will introduce proper dependencies as well - typing make something_else in a clean build dir will build generator, then run it, then build something_else.
You need to tell CMake that the files are generated:
set_source_files_properties(someautogeneratedfile.cpp PROPERTIES GENERATED TRUE)
set_source_files_properties(someothergeneratedfile.h PROPERTIES GENERATED TRUE)
Now you just need to make sure that you run your generator before you run your something_else step. There are ways to manage that too, check out
add_custom_command(...)
add_custom_target(...)
add_dependencies(...)
Related
In my C++ project I'm using a source generator to embed some resources into the binary.
I use CMake to build my project and my code works but had some issues.
I am pretty sure that what I want to accomplish is possible but I didn't find any answer online.
The current problems I have are:
The generator runs every time, even if the input files did not change.
This is not too big of a deal because it is really fast, but I hopped there was a better way to do it
While using Ninja the generator runs at every build (as described above) without rebuilding every time.
I think that Ninja sees that the file has not changed and does not build it again,
but when I make changes in the resources change it still uses the old version.
It takes another build to "realize" that the generated file has changed and rebuild it
While using Make the code rebuilds every time, even when the generated file does not change, resulting in wasted build time
In both cases (looking at the output) the generator runs before the compiler.
This situation is not unsustainable but I was wondering if a better solution was possible.
Here's a code snippet from my CMakeLists.txt
add_subdirectory(Generator)
file(GLOB RESOURCES Resources/*)
add_custom_command(
OUTPUT src/Resources/Generated.hpp src/Resources/Generated.cpp
COMMAND Generator ${RESOURCES}
DEPENDS ${RESOURCES}
DEPENDS Generator
)
add_custom_target(Generated DEPENDS src/Resources/Generated.hpp src/Resources/Generated.cpp)
add_dependencies(${PROJECT_NAME} Generated)
Here's a minimal reproducible example, sorry for not having it before.
EDIT: I implemented the functional solution from the correct answer in the fix branch, on the same repo. It might be useful for future reference :)
This one is interesting, because there are multiple errors and stylistic issues, which partially overlap each other.
First off:
file(GLOB_RECURSE SRC src/*.cpp src/*.hpp)
add_executable(${PROJECT_NAME} ${SRC})
While convenient in the beginning, globbing your sources is not a good idea. At some point you will have a testme.cpp in there that should not be built with the rest, or a conditionally_compiled.cpp that should only be compiled if a certain option is set. You end up compiling sources that you really did not intended to.
In this case, the file src/Generated.hpp from your git repository. That file is supposed to be generated, not checked out from repo. How did it even get in there?
add_custom_command(
OUTPUT src/Generated.hpp
COMMAND ${PROJECT_SOURCE_DIR}/generator.sh
${PROJECT_SOURCE_DIR}/Resources/data.txt
> ${PROJECT_SOURCE_DIR}/src/Generated.hpp
DEPENDS Resources/data.txt
DEPENDS ${PROJECT_SOURCE_DIR}/generator.sh
)
Do you see the output redirection there? You wrote to ${PROJECT_SOURCE_DIR}. That is not a good idea. Your source tree should never have anything compiled or generated in it. Because these things end up being committed with the rest... like it happened to you.
Next issue:
add_custom_target(Generated DEPENDS src/Generated.hpp)
This creates a make target Generated. Try it: make Generated. You keep getting the following output:
[100%] Generating src/Generated.hpp
[100%] Built target Generated
Obviously it does not realize that Generated.hpp is already up-to-date. Why not?
Let's look at your custom command again:
add_custom_command(
OUTPUT src/Generated.hpp
COMMAND ${PROJECT_SOURCE_DIR}/generator.sh
${PROJECT_SOURCE_DIR}/Resources/data.txt
> ${PROJECT_SOURCE_DIR}/src/Generated.hpp
DEPENDS Resources/data.txt
DEPENDS ${PROJECT_SOURCE_DIR}/generator.sh
)
What if I told you that your OUTPUT is never actually generated?
Quoting from CMake docs on add_custom_command, emphasis mine:
OUTPUT
Specify the output files the command is expected to produce. If an output name is a relative path it will be interpreted relative to the build tree directory corresponding to the current source directory.
So your output claims to be to the binary tree, but your command's redirection is to the source tree... no wonder the generator keeps getting re-run.
Having your header generated in the wrong location does not give you a compiler error, because that header from your source tree gets picked up by your GLOB_RECURSE. As that one keeps getting re-generated, your executable keeps getting recompiled as well.
Try this from your build directory:
mkdir src && touch src/Generated.hpp && make Generated
Output:
[100%] Built target Generated
You see that make has nothing to do for the Generated target, because it now sees an OUTPUT of your custom command that is newer than its dependencies. Of course, that touched file isn't the generated one; we need to bring it all together.
Solution
Don't write to your source tree.
Since ${PROJECT_BINARY_DIR}/src does not exist, you need to either create it, or live with the created files on the top dir. I did the latter for simplicity here. I also removed unnecessary ${PROJECT_SOURCE_DIR} uses.
add_custom_command(
OUTPUT Generated.hpp
COMMAND ${PROJECT_SOURCE_DIR}/generator.sh \
${PROJECT_SOURCE_DIR}/Resources/data.txt \
> Generated.hpp
DEPENDS Resources/data.txt
DEPENDS generator.sh
)
Don't glob, keep control over what actually gets compiled:
add_executable( ${PROJECT_NAME} src/main.cpp )
Add the binary tree to your target's include path. After add_executable:
target_include_directories( ${PROJECT_NAME} PRIVATE ${PROJECT_BINARY_DIR} )
That's it, things work as expected now.
Description
I have a code generator that takes an XML input file and outputs a c++ header and source file. These auto-generated files are then compiled with static source files to produce a library. Simplified cmake file.
add_library(subdirectory/${MODULE_NAME} ${STATIC_SOURCES})
# Invoke auto-coder
add_custom_command(
OUTPUT ${GEN_HEADER} ${GEN_SOURCE}
COMMAND ${CMAKE_COMMAND} ${CMAKE_SOURCE_DIR}/my_autocoder ${SOURCE_XML}
DEPENDS ${SOURCE_XML}
)
# Add auto-generated source dependencies
target_sources(${MODULE_NAME}
PUBLIC ${GEN_HEADER}
PRIVATE ${GEN_SOURCE}
)
Problem
A generated header file ${GEN_HEADER_A} includes other generated headers files e.g. ${GEN_HEADER_B}. When running make at the top level everything builds correctly. However when building the library in isolation, it errors because ${GEN_HEADER_B} does not exist.
Question
How do I add all dependencies like ${GEN_HEADER_A} on ${GEN_HEADER_B} without manually specifying each dependency?
Attempts
The IMPLICIT_DEPENDS feature of add_custom_command seems to have the functionality I want. However, my attempts at using it have not resulted in the code generator being invoked for ${GEN_HEADER_B}. My assumption is this is because I already have a rule to build ${MODULE_NAME} when I added the library add_library(subdirectory/${MODULE_NAME} ${STATIC_SOURCES})
add_custom_command(
OUTPUT ${MODULE_NAME}
COMMAND ...
IMPLICIT_DEPENDS ${GEN_HEADER}
)
I do see ${GEN_HEADER_B} in the CXX.includecache under ${GEN_HEADER_A}
If you want file A to be (re)created when file B is built, you should specify this dependency explicitely.
Dependencies discovered by scanning (either automatic in add_library/add_executable commands, or via IMPLICIT_DEPENDS option of add_custom_command) are only checked for modification. CMake doesn't attempt to (re)create dependencies discovered by scanning.
I'm trying to generate Qt help files during build.
.qhp (Qt help project), contains list of HTML files located in /html folder adjacent to .qhp file.
It works fine, if I change .qhp file. But if I change only HTML files the build is not started.
This is part of my CMakeLists.txt:
add_custom_command(
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/help.qch
COMMAND qhelpgenerator ${CMAKE_CURRENT_SOURCE_DIR}/help.qhp -o ${CMAKE_CURRENT_BINARY_DIR}/help.qch
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/help.qhp
)
ADD_CUSTOM_TARGET(${TARGET_NAME}
ALL DEPENDS
${CMAKE_CURRENT_BINARY_DIR}/help.qch
SOURCES
${CMAKE_CURRENT_SOURCE_DIR}/uav_help.qhp
)
Q: How can I add dependencies to HTML files (w/o creating list of them in CMakeLists.txt), if I don't want to add them to the project?
Q2: Are there other ways to organize automatic .qch generation during build?
List these HTML's in DEPENDS of add_custom_command() call too, obviously.
You can also look at KDE's Extra CMake Modules project, which contains ECMAddQch macro.
I am trying to add a custom target with CMake that executes one command for each given .cpp file. The command should only be re-executed when the source file itself or one of the included source files changes. AFAIK to achieve this I need a list of all the included files and add them to the DEPENDS option of the add_custom_command() calls that belong to my custom target.
So is there a built-in way to get that list of included files?
I know about the IMPLICIT_DEPENDS option of the add_custom_command() function but it only works for Makefile generators. I would like to make this work for all generators.
Thank you for your time
Edit:
As requested I will post some cmake code to show what I want to achieve.
I want to add a custom target, that runs clang-tidy on all the given .cpp files. When incrementally building the custom target the clang-tidy commands should be re-run whenever a .cpp file or one of its directly or indirectly included header files is changed. Just like re-runs of the compiler are handled.
# ----------------------------------------------------------------------------------------
# mainTargetName The name of the target that shall be analyzed
# files A list of all the main targets .cpp files
#
function( addStaticAnalysisTarget mainTargetName files )
set(targetName runStaticAnalysis_${mainTargetName})
set(command "clang-tidy-4.0 -checks=* -p ${CMAKE_BINARY_DIR}")
foreach( file ${files} )
get_filename_component( baseName ${file} NAME_WE)
set(stampFile ${CMAKE_CURRENT_BINARY_DIR}/analyze_${baseName}.stamp )
set(fullFile ${CMAKE_CURRENT_SOURCE_DIR}/${file})
set(commandWithFile "${command} ${fullFile}")
separate_arguments_for_platform( commandList ${commandWithFile})
add_custom_command(
OUTPUT ${stampFile}
DEPENDS "${fullFile}"
IMPLICIT_DEPENDS CXX "${fullFile}"
COMMAND ${commandList}
COMMAND cmake -E touch "${stampFile}" # without creating a file as a touch-stone the command will always be re-run.
WORKING_DIRECTORY ${CPPCODEBASE_ROOT_DIR}
COMMENT "${commandWithFile}"
VERBATIM
)
list(APPEND stampFiles ${stampFile})
endforeach()
set_source_files_properties(${stampFiles} PROPERTIES GENERATED TRUE) # make the stamp files known to cmake as generated files.
add_custom_target(
${targetName}
DEPENDS ${stampFiles}
)
endfunction()
The problem with that is, that it does not seem to work. When I change included files clang-tidy is not re-run for the affected files.
I used the "Unix Makefile" generator for this example so it should work at least with make. Any hints why it doesn't?
My hopes where that I could achieve the desired behavior for all generators by somehow getting the file-dependencies at cmake time and then adding them to the ''''DEPENDS'''' list. But the dependency scanning must be done each time the command is run, so it can not be done at cmake time. This means that the scanning must be implemented by cmake which it currently is not.
A guy with similar problems:
https://gitlab.kitware.com/cmake/cmake/issues/16830
Edit 2:
I think the problem that the IMPLICIT_DEPENDS option was not working was because I did not use correct filenames. I changed that in the code snipped, but I have not yet tested if it works in the project.
I think the answer to my question is ...
No, you can not use cmakes dependency scanner in the cmake code.
That makes sense, because this problem can not be solved at cmake time, because the dependencies of a .cpp file may change without cmake being re-run.
The problem must be solved within cmake itself at make time. This is done when using the IMPLICIT_DEPENDS option.
Also, I tried to solve a Problem that I did not really have, because at this point I can only run clang-tidy on linux anyways. However, clang-tidy may become available on windows as well and then I may have the problem again.
To sum the comments up:
Tambre stated that CMake is not a compiler and therefore can not do that.
I think this is wrong. According to this article, CMake can parse cpp include dependencies because make has no such dependency searcher itself. That was news to me, but I mostly live on Windows so I am not that familiar with make. It could also be possible that in the meantime make was extended to do its own dependency searching. Also this explains why the IMPLICIT_DEPENDS option is only available for make.
Florian pointed out that it is not necessary to create an own custom target for running clang-tidy. Instead, one can use the CXX_CLANG_TIDY target property to run clang-tidy for each file after compiling it. This means however, that static-analysis can not be separated from the build which could lead to inacceptable buildtimes.
There is the cmake -E cmake_depends command line, that could be used to retrieve dependencies at cmake time. But as stated above, I erroneously thought that I needed the dependencies at cmake time, while I needed them at runtime.
The IMPLICIT_DEPENDS options did not work because I had an error in my cmake code.
I have a custom tool that processes a given list of IDL files and produces a number of .cpp and .h files as output. I want to add those files to the list of things to compile in my CMakeLists, and also model the dependencies those files have on the IDL.
To keep things simple, I will state that any change to any of the IDL files should trigger a regeneration of all cpp/h.
I have a custom command that takes care of running the generator tool and listing all the IDL files as dependencies.
My issue is getting the subsequent list of cpp/h files into cmake at build-time. It is not possible to infer from the name of the IDL files what cpp files will be generated. My generator tool will, however, output the list of generated files to a text file.
So my question is: how do I instruct cmake to "read from this text file and add the contents as extra source and header files to be compiled", also bearing in mind that the said text file only exists during a certain point of the build?
CMake needs to be able to infer the names of all .cpp files participating in the build at configure time. It is not possible to add files afterwards without re-running CMake.
One possible approach would be to use a two-phase CMake build: Instead of building the generated source files directly from your main project, you create a separate CMake project for building just the generated sources.
Then in your main CMake project you add a custom target that runs after the code generation and invokes CMake to both configure and build the generated files project.
The disadvantage here is that the generated files no longer appear as part of the main project. Also some trickery is required if you don't want to rebuild the generated sources every time - custom targets are always considered out-of-date, so you might want to use a script here that only runs CMake on the subproject if the generated files changed.
This is a few years late but this works just fine:
#run whatever tool that generates the cpp files
execute_process(COMMAND "./your_tool.sh")
#read files from files.txt and make a cmake 'list' out of them
file(READ "files.txt" SOURCES)
#found this technique to build the cmake list here:
#http://public.kitware.com/pipermail/cmake/2007-May/014236.html
#maybe there is a better way...
STRING(REGEX REPLACE ";" "\\\\;" SOURCES "${SOURCES}")
STRING(REGEX REPLACE "\n" ";" SOURCES "${SOURCES}")
#at this point you have your source files inside ${SOURCES}
#build a static library...?
add_library(mylib STATIC ${SOURCES})
There is a function that build the list directly from file:
file(STRINGS <filename> <variable> [<options>...])
source: https://cmake.org/cmake/help/v3.11/command/file.html