I'm using cmake with C++ project. I want to use precompiled headers in GCC.
I want to run cmake once and then, when running make, I want this actions to happen:
Run my custom tool (python script) on whole source tree. It generates precompiled.h headers in some subdirectories. It's purpose is to scan *.h and *.cpp for #include's and move them to common precompiled.h files.
generate GCC's precompiled headers (g++ -c precompiled.h -o precompiled.h.gch) ONLY when precompiled.h file has changed after step 1.
build outdated targets (after step 2. because I want to use .gch file)
I've managed to add dependencies, so whenever I build any target, my python script's target is executed (this is ugly, because now every target has to depend on single global_init target). Also I can modify my python script, so it doesn't modify precompiled.h when it is not necessary.
But I don't know what to do next. Is there any way to tell cmake that it should run my custom script and AFTER THAT determine if precompiled.h.gch must be created? Now it is always build (this file contains many #include's, so it takes some time). Basically:
Execute python script -> precompiled.h MAY BE updated
Examine precompiled.h timestamp and if it's newer -> build precompiled.h.gch
Standard compilation steps
I've looked in many places, but cmake's (and make's) way of calculating dependencies seems too weak to accomplish this task.
In this question:
lazy c++
compilation always happens after lazycpp is executed. This would not be optimal in my case.
The following CMake commands may serve as a starting point for your project:
add_custom_target(scanner COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/scanner.py"
COMMENT "Scanning include files ..."
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}")
This command covers step 1. It invokes your python script which may update the existing precompiled.h files in CMAKE_CURRENT_SOURCE_DIR.
add_custom_command(OUTPUT precompiled.h.gch
COMMAND ${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS}
-c "${CMAKE_CURRENT_SOURCE_DIR}/precompiled.h" -o precompiled.h.gch
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/precompiled.h"
IMPLICIT_DEPENDS CXX "${CMAKE_CURRENT_SOURCE_DIR}/precompiled.h")
add_custom_target(generate_precompiled DEPENDS
"${CMAKE_CURRENT_BINARY_DIR}/precompiled.h.gch")
add_dependencies(generate_precompiled scanner)
These commands cover step 2. The precompiled header is generated with a custom command which depends on precompiled.h and implicitly on other headers that precompiled.h includes.
precompiled.h.gch is generated in the CMAKE_CURRENT_BINARY_DIR directory.
Because the precompiled header generation requires precompiled.h to be updated by the python script, we add a target level dependency on the target scanner.
include_directories(BEFORE ${CMAKE_CURRENT_BINARY_DIR})
add_executable(main main.cpp)
set_target_properties(main PROPERTIES COMPILE_FLAGS "-include precompiled.h -H")
add_dependencies(main generate_precompiled)
These commands add standard compilation steps which generate an executable main. The precompiled header is included as a default header with gcc's -include switch. The CMAKE_CURRENT_BINARY_DIR which contains precompiled.h.gch is added as an include directory. gcc will pick up precompiled.h.gch from there when precompiled.h is included (see Using Precompiled Headers).
Finally the target dependency on generate_precompiled ensures that the precompiled header is updated if necessary.
Related
I have a CMakeLists.txt in which I want to generate several source files (namely, versiondata.cpp and version.rc.inc, included by res.rc) that depends on the general environment (current git HEAD, gcc -v output, CMakeCache.txt itself, and so on).
If it depended just on some files, I would generate it using an add_custom_command directive with the relevant DEPENDS and OUTPUT clauses; however, it's a bit tricky to pinpoint exactly its file dependencies; ideally, I'd want to run my script every time I call make, updating the files only if needed; if the generated files have actually been touched, then the targets depending from them should be rebuilt (the script is careful not to overwrite the files if they would have the same content as before).
My first attempt was using an add_custom_command with a fake main output, like this:
add_custom_command(OUTPUT versiondata.cpp.fake versiondata.cpp version.rc.inc
COMMAND my_command my_options
COMMENT "Generating versiondata.cpp"
)
# ...
# explicitly set the dependencies of res.rc, as they are not auto-deduced
set_source_files_properties(res.rc PROPERTIES OBJECT_DEPENDS "${PROJECT_BINARY_DIR}/version.rc.inc;${PROJECT_SOURCE_DIR}/other_stuff.ico")
# ...
add_executable(my_executable WIN32 ALL main.cpp versiondata.cpp res.rc)
versiondata.cpp.fake is never really generated, so the custom command is always run. This worked correctly, but always rebuilt my_executable, as CMake for some reasons automatically touches the output files (if generated) even though my script left them alone.
Then I thought I might make it work using an add_custom_target, that is automatically "never already satisfied":
add_custom_target(versiondata BYPRODUCTS versiondata.cpp version.rc.inc
COMMAND my_command my_options
COMMENT "Generating versiondata.cpp"
)
# ...
# explicitly set the dependencies of res.rc, as they are not auto-deduced
set_source_files_properties(res.rc PROPERTIES OBJECT_DEPENDS "${PROJECT_BINARY_DIR}/version.rc.inc;${PROJECT_SOURCE_DIR}/other_stuff.ico")
# ...
add_executable(my_executable WIN32 ALL main.cpp versiondata.cpp res.rc)
The idea here is that the versiondata target should be "pulled in" from the targets that depend on its BYPRODUCTS, and should be always executed. This seems to work on CMake 3.20, and the BYPRODUCTS seem to have some effect because if I remove the dependencies from my_executable my script doesn't get called.
However, on CMake 3.5 I get
make[2]: *** No rule to make target 'version.rc.inc', needed by 'CMakeFiles/my_executable.dir/res.rc.res'. Stop.
and if I remove the explicit dependency from version.rc.inc it doesn't get generated at all
[ 45%] Building RC object CMakeFiles/my_executable.dir/res.rc.res
/co/my_executable/res.rc:386:26: fatal error: version.rc.inc: No such file or directory
#include "version.rc.inc"
^
compilation terminated.
/opt/mingw32-dw2/bin/i686-w64-mingw32-windres: preprocessing failed.
CMakeFiles/my_executable.dir/build.make:5080: recipe for target 'CMakeFiles/my_executable.dir/res.rc.res' failed
make[2]: *** [CMakeFiles/my_executable.dir/res.rc.res] Error 1
so I suspect that the fact that this works in 3.20 is just by chance.
Long story short: is there some way to make this work as I wish?
In CMake there are two types of dependencies:
Target-level dependency, between targets.
A target can be build only after unconditional building of all targets it depends on.
File-level dependency, between files.
If some file is older than one of its dependencies, the file will be regenerated using corresponded COMMAND.
The key factor is that checking for timestamp of dependent files is performed strictly after building of dependent targets.
For correct regeneration of versiondata.cpp file and executable based on it, one need both dependencies:
Target-level, which would ensure that versiondata custom target
will be built before the executable.
add_dependencies(my_executable versiondata)
File-level, which will ensure that the executable will be rebuilt whenever
file versiondata.cpp will be updated.
This dependency is created automatically by listing versiondata.cpp
among the sources for the executable.
Now about BYPRODUCTS.
Even without explicit add_dependencies, your code works on CMake 3.20 because BYPRODUCTS generates needed target-level dependency automatically.
This could be deduced from the description of DEPENDS option in add_custom_target/add_custom_command:
Changed in version 3.16: A target-level dependency is added if any dependency is a byproduct of a target or any of its build events in the same directory to ensure the byproducts will be available before this target is built.
and noting, that add_executable effectively depends on every of its source files.
Because given comment for DEPENDS is applicable only for CMake 3.16 and later,
in older CMake versions BYPRODUCTS does not create target-level dependency automatically, and one need to resort to explicit add_dependencies.
I am trying to add a custom target with CMake that executes one command for each given .cpp file. The command should only be re-executed when the source file itself or one of the included source files changes. AFAIK to achieve this I need a list of all the included files and add them to the DEPENDS option of the add_custom_command() calls that belong to my custom target.
So is there a built-in way to get that list of included files?
I know about the IMPLICIT_DEPENDS option of the add_custom_command() function but it only works for Makefile generators. I would like to make this work for all generators.
Thank you for your time
Edit:
As requested I will post some cmake code to show what I want to achieve.
I want to add a custom target, that runs clang-tidy on all the given .cpp files. When incrementally building the custom target the clang-tidy commands should be re-run whenever a .cpp file or one of its directly or indirectly included header files is changed. Just like re-runs of the compiler are handled.
# ----------------------------------------------------------------------------------------
# mainTargetName The name of the target that shall be analyzed
# files A list of all the main targets .cpp files
#
function( addStaticAnalysisTarget mainTargetName files )
set(targetName runStaticAnalysis_${mainTargetName})
set(command "clang-tidy-4.0 -checks=* -p ${CMAKE_BINARY_DIR}")
foreach( file ${files} )
get_filename_component( baseName ${file} NAME_WE)
set(stampFile ${CMAKE_CURRENT_BINARY_DIR}/analyze_${baseName}.stamp )
set(fullFile ${CMAKE_CURRENT_SOURCE_DIR}/${file})
set(commandWithFile "${command} ${fullFile}")
separate_arguments_for_platform( commandList ${commandWithFile})
add_custom_command(
OUTPUT ${stampFile}
DEPENDS "${fullFile}"
IMPLICIT_DEPENDS CXX "${fullFile}"
COMMAND ${commandList}
COMMAND cmake -E touch "${stampFile}" # without creating a file as a touch-stone the command will always be re-run.
WORKING_DIRECTORY ${CPPCODEBASE_ROOT_DIR}
COMMENT "${commandWithFile}"
VERBATIM
)
list(APPEND stampFiles ${stampFile})
endforeach()
set_source_files_properties(${stampFiles} PROPERTIES GENERATED TRUE) # make the stamp files known to cmake as generated files.
add_custom_target(
${targetName}
DEPENDS ${stampFiles}
)
endfunction()
The problem with that is, that it does not seem to work. When I change included files clang-tidy is not re-run for the affected files.
I used the "Unix Makefile" generator for this example so it should work at least with make. Any hints why it doesn't?
My hopes where that I could achieve the desired behavior for all generators by somehow getting the file-dependencies at cmake time and then adding them to the ''''DEPENDS'''' list. But the dependency scanning must be done each time the command is run, so it can not be done at cmake time. This means that the scanning must be implemented by cmake which it currently is not.
A guy with similar problems:
https://gitlab.kitware.com/cmake/cmake/issues/16830
Edit 2:
I think the problem that the IMPLICIT_DEPENDS option was not working was because I did not use correct filenames. I changed that in the code snipped, but I have not yet tested if it works in the project.
I think the answer to my question is ...
No, you can not use cmakes dependency scanner in the cmake code.
That makes sense, because this problem can not be solved at cmake time, because the dependencies of a .cpp file may change without cmake being re-run.
The problem must be solved within cmake itself at make time. This is done when using the IMPLICIT_DEPENDS option.
Also, I tried to solve a Problem that I did not really have, because at this point I can only run clang-tidy on linux anyways. However, clang-tidy may become available on windows as well and then I may have the problem again.
To sum the comments up:
Tambre stated that CMake is not a compiler and therefore can not do that.
I think this is wrong. According to this article, CMake can parse cpp include dependencies because make has no such dependency searcher itself. That was news to me, but I mostly live on Windows so I am not that familiar with make. It could also be possible that in the meantime make was extended to do its own dependency searching. Also this explains why the IMPLICIT_DEPENDS option is only available for make.
Florian pointed out that it is not necessary to create an own custom target for running clang-tidy. Instead, one can use the CXX_CLANG_TIDY target property to run clang-tidy for each file after compiling it. This means however, that static-analysis can not be separated from the build which could lead to inacceptable buildtimes.
There is the cmake -E cmake_depends command line, that could be used to retrieve dependencies at cmake time. But as stated above, I erroneously thought that I needed the dependencies at cmake time, while I needed them at runtime.
The IMPLICIT_DEPENDS options did not work because I had an error in my cmake code.
I'm developing a C++ project which is going to be enclosed on a bigger one.
I've seen that on the bigger project (is a Qt application and it's being generated from qmake) I am able to compile a single file from the linux command line, just entering the relative path to the specific file as an argument to make.
On the other hand, I'm using CMake for my own project. When I modify some code for a compilation unit and I have to modify its header file, I have to wait a long time to compile its dependencies and then its own source file. But there are some situations in which I would prefer to check whether the source code in the *.cc file is compilable without errors.
Is there a way to generate a Makefile from CMake the way qmake does this? Switching to qmake is not an option anymore.
You do not have to add extra custom targets to your CMake scripts, as the Makefiles generated by CMake already contain .o targets for each .cc file. E.g. if you have a source file called mySourceFile.cc, there will be a Makefile in your build directory that defines a target called <Some Path>/mySourceFile.cc.o. If you cd into your build directory, you can use grep or ack-grep to locate the Makefile that defines this target, then cd into that Makefile's directory and build it.
E.g. suppose the command ack-grep mySourceFile.cc.o prints something like:
foo/bar/Makefile
119:x/y/z/mySourceFile.o: x/y/z/mySourceFile.cc.o
123:x/y/z/mySourceFile.cc.o:
124: # recipe for building target
Then you can build mySourceFile.cc.o by doing:
cd foo/bar && make x/y/z/mySourceFile.cc.o
CMake doesn't have a generic built-in way of doing this (it's an open issue), but if you're using the Ninja generator, you can can use a special Ninja syntax for building just the direct outputs of a given source file. For example, to compile just foo.o you would use:
ninja /path/to/foo.cpp^
Not out-of-the box. CMake does not expose those "internal" makefile rules in the main makefile.
You can do this only if you consider what kind of file structure CMake uses internally. You can e.g. for compiling a single .obj files using CMake generated makefiles call
make -f CMakeFiles/myProg.dir/build.make CMakeFiles/myProg.dir/main.cc.obj
when you have something like
cmake_minimum_required(VERSION 3.1)
project(myProg CXX)
file(WRITE "main.cc" "int main()\n{\nreturn 0;\n}")
add_executable(myProg main.cc)
To build src/foo.cpp alone:
cmake --build . --target src/foo.cpp.o
No, CMake does not offer built-in support to compile single files.
You have to add a target for each object file, maybe by a function iterating over all files of a directory.
Others have suggested ways to find the target name (ending in .cpp.o) from the .cpp filename, but if you already know the name of a target that will trigger compilation of the .cpp file and you're using ninja this suggestion should be easier.
First build the target:
ninja TriggersCppCompilationLib
Assuming your file was changed or was not yet built, ninja will print the full target name. When you see the name come up, hit enter so it is not overwritten. Then simply copy the name from the terminal (e.g. using tmux copy mode).
I'd like to add some object files to a CMake static library, but they have a custom extension.
Here's what I've tried:
set(SRCS testfile.cxx jsobj.js)
add_library(testlib STATIC ${SRCS})
When made, CMake invokes ar testfile.cxx.o (ie the other file is completely ignored). How do I get it included in the archive? Here are some other tricks I've tried:
list(APPEND CMAKE_CXX_SOURCE_FILE_EXTENSIONS js)
list(APPEND CMAKE_C_SOURCE_FILE_EXTENSIONS js) # no luck
add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/jsobj.js.o
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/jsobj.js
${CMAKE_CURRENT_BINARY_DIR}/jsobj.js.o
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/jsobj.js.o) # still no luck
(In case you're interested, I'm using the emscripten compiler, which can accept C/C++ files as source input, and JavaScript files are essentially "precompiled objects". I want to find a way to get CMake to add them to the ar commandline, that's all!)
For the record, this is how I solved my problem in a hacky way: "proper" solutions would be gladly accepted.
I made up a new file extension for my special pre-compiled objects, "jso", then added it to the list of input files CMake understands:
list(APPEND CMAKE_CXX_SOURCE_FILE_EXTENSIONS jso)
Then, I add my object files with the extension ".jso" to the CMake sources for inclusion in a static library target.
Finally, I hacked the compiler by setting CC=mycc, where mycc is a Python script which checks if the input has the extension ".jso": if not, it simply re-invokes the standard compiler; otherwise it copies the input to the output with no changes at all, so that mycc -c input.jso -o output.jso.o is just a file copy.
This isn't pretty, but it picks up all the dependencies perfectly for incremental builds. I can't pretend it's pretty, but doing things the way CMake likes seems to work. Here, we're just pretending all inputs are source files, even if they're actually already compiled.
I have a custom tool that processes a given list of IDL files and produces a number of .cpp and .h files as output. I want to add those files to the list of things to compile in my CMakeLists, and also model the dependencies those files have on the IDL.
To keep things simple, I will state that any change to any of the IDL files should trigger a regeneration of all cpp/h.
I have a custom command that takes care of running the generator tool and listing all the IDL files as dependencies.
My issue is getting the subsequent list of cpp/h files into cmake at build-time. It is not possible to infer from the name of the IDL files what cpp files will be generated. My generator tool will, however, output the list of generated files to a text file.
So my question is: how do I instruct cmake to "read from this text file and add the contents as extra source and header files to be compiled", also bearing in mind that the said text file only exists during a certain point of the build?
CMake needs to be able to infer the names of all .cpp files participating in the build at configure time. It is not possible to add files afterwards without re-running CMake.
One possible approach would be to use a two-phase CMake build: Instead of building the generated source files directly from your main project, you create a separate CMake project for building just the generated sources.
Then in your main CMake project you add a custom target that runs after the code generation and invokes CMake to both configure and build the generated files project.
The disadvantage here is that the generated files no longer appear as part of the main project. Also some trickery is required if you don't want to rebuild the generated sources every time - custom targets are always considered out-of-date, so you might want to use a script here that only runs CMake on the subproject if the generated files changed.
This is a few years late but this works just fine:
#run whatever tool that generates the cpp files
execute_process(COMMAND "./your_tool.sh")
#read files from files.txt and make a cmake 'list' out of them
file(READ "files.txt" SOURCES)
#found this technique to build the cmake list here:
#http://public.kitware.com/pipermail/cmake/2007-May/014236.html
#maybe there is a better way...
STRING(REGEX REPLACE ";" "\\\\;" SOURCES "${SOURCES}")
STRING(REGEX REPLACE "\n" ";" SOURCES "${SOURCES}")
#at this point you have your source files inside ${SOURCES}
#build a static library...?
add_library(mylib STATIC ${SOURCES})
There is a function that build the list directly from file:
file(STRINGS <filename> <variable> [<options>...])
source: https://cmake.org/cmake/help/v3.11/command/file.html