CMake run custom command before build? - c++

Example source of a binary I want to run before each build, once per add_executable:
#include <stdio.h>
int main(int argc, char *argv[]) {
for(int i=0; i<argc; ++i)
printf("argv[%d] = %s\n", i, argv[i]);
fclose(fopen("foo.hh", "a"));
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.5)
project(foo_proj)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
set(SOURCE_FILES main.cpp)
# ---- this line changes ----
add_executable(foo_proj ${SOURCE_FILES})
Attempts:
add_custom_target(create_foo_hh COMMAND /tmp/bin/create_foo_hh)
add_dependencies(${SOURCE_FILES} create_foo_hh)
Error:Cannot add target-level dependencies to non-existent target "main.cpp".
The add_dependencies works for top-level logical targets created by the add_executable, add_library, or add_custom_target commands. If you want to add file-level dependencies see the DEPENDS option of the add_custom_target and add_custom_command commands.
execute_process(COMMAND /tmp/bin/create_foo_hh main.cpp)
No error, but foo.hh isn't created.
How do I automate the running of this command?

execute_process() is invoked at configuration time.
You can use add_custom_command():
add_custom_command(
OUTPUT foo.hh
COMMAND /tmp/bin/create_foo_h main.cpp
DEPENDS ${SOURCE_FILES} /tmp/bin/create_foo_hh main.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
add_executable(foo_proj ${SOURCE_FILES} foo.hh)
That way, foo.hh is a dependency of foo_proj: and your command will be invoked when building foo_proj. It depends on ${SOURCE_FILES} and /tmp/bin/create_foo_hh main.cpp so that it is generated again if one of those files changes.
Regarding paths, add_custom_command() is configured to run in the current build directory to generate the file there, and include_directories() is used to add the build directory to the include dirs.

You probably don't want the custom target to depend on your source files (because they aren't targets themselves and are therefore never "run"), but on the target you create with them:
target_add_dependencies(foo_proj create_foo_hh)

I think that the cleanest is to add two new project() (targets) and then add the resulting file to your final executable. This is how cmake can build a valid dependency tree so when your source files change they get recompiled, the command run, as necessary to get everything up to date.
Build Executable
First, as you do in your example, I create an executable from some .cpp file:
(example extracted from the as2js project)
project(unicode-characters)
add_executable(${PROJECT_NAME}
unicode_characters.cpp
)
target_include_directories(${PROJECT_NAME}
PUBLIC
${ICU_INCLUDE_DIRS}
${SNAPDEV_INCLUDE_DIRS}
)
target_link_libraries(${PROJECT_NAME}
${ICU_LIBRARIES}
${ICU_I18N_LIBRARIES}
)
As we can see,m we can add specific include paths (-I) and library paths (-L). It is specific to that one target so you can have a set of paths that is different from the one used with your other executables.
Generate Additional File
Next, you create a custom command to run your executable like so:
project(unicode-character-types)
set(UNICODE_CHARACTER_TYPES_CI ${PROJECT_BINARY_DIR}/${PROJECT_NAME}.ci)
add_custom_command(
OUTPUT
${UNICODE_CHARACTER_TYPES_CI}
COMMAND
unicode-characters >${UNICODE_CHARACTER_TYPES_CI}
WORKING_DIRECTORY
${PROJECT_BINARY_DIR}
DEPENDS
unicode-characters
)
add_custom_target(${PROJECT_NAME}
DEPENDS
${UNICODE_CHARACTER_TYPES_CI}
)
Notice a couple of things:
I set a variable (UNICODE_CHARACTER_TYPES_CI) because I am going to reference that one file multiple times
a. Notice how I put the destination in the binary (cmake output folder) using the ${PROJECT_BINARY_DIR}/... prefix. This is best to avoid generating those files in your source tree (and possibly ending up adding that file to your source tracking system like svn or git).
b. An important aspect of the add_custom_command() is the DEPENDS section which includes the name of your special command, the one we defined in the previous step.
The add_custom_target() is what allows cmake to find your target and execute the corresponding command whenever one of the source files (a.k.a. dependency) changes; notice the DEPENDS definition.
Use the Output
Finally, here is the main project (a library in my case) that makes use of the file we generated in the step above.
Notice that I reference that file using the variable I defined in the previous step. That way, when I feel like changing that name, I can do it by simply editing that one variable.
project(as2js)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/version.h.in
${CMAKE_CURRENT_BINARY_DIR}/version.h
)
add_library(${PROJECT_NAME} SHARED
compiler/compiler.cpp
...
parser/parser_variable.cpp
${UNICODE_CHARACTER_TYPES_CI}
file/database.cpp
...
)
(Note: the ... represent a list of files, shorten for display here as these are not important, the link above will take you to the file with the complete list.)
By having the filename inside the list of files defined in the add_library() (or the add_executable() in your case), you create a dependency which will find your custom_target(), because of the filename defined in the OUTPUT section of the add_custom_command()¹.
¹ It is possible to defined multiple outputs for an add_custom_command(). For example, some of my generators output a .cpp and a .h. In that case, I simply define both files in the OUTPUT section.
Results
Important points about the final results with this solution:
the output files of your generator are saved in the binary output path instead of your current working directory
the Makefile generated by cmake includes all the necessary targets/dependencies which means changing any of the input files regenerate everything as expected (even if you just update a comment)
if the generator fails, the build fails as expected
the files are generated by the build step (make time) instead of the generation step (cmake time, like the execute_process() would do)

Related

Forcing cmake to call external executable before reading source files [duplicate]

Example source of a binary I want to run before each build, once per add_executable:
#include <stdio.h>
int main(int argc, char *argv[]) {
for(int i=0; i<argc; ++i)
printf("argv[%d] = %s\n", i, argv[i]);
fclose(fopen("foo.hh", "a"));
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.5)
project(foo_proj)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
set(SOURCE_FILES main.cpp)
# ---- this line changes ----
add_executable(foo_proj ${SOURCE_FILES})
Attempts:
add_custom_target(create_foo_hh COMMAND /tmp/bin/create_foo_hh)
add_dependencies(${SOURCE_FILES} create_foo_hh)
Error:Cannot add target-level dependencies to non-existent target "main.cpp".
The add_dependencies works for top-level logical targets created by the add_executable, add_library, or add_custom_target commands. If you want to add file-level dependencies see the DEPENDS option of the add_custom_target and add_custom_command commands.
execute_process(COMMAND /tmp/bin/create_foo_hh main.cpp)
No error, but foo.hh isn't created.
How do I automate the running of this command?
execute_process() is invoked at configuration time.
You can use add_custom_command():
add_custom_command(
OUTPUT foo.hh
COMMAND /tmp/bin/create_foo_h main.cpp
DEPENDS ${SOURCE_FILES} /tmp/bin/create_foo_hh main.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
add_executable(foo_proj ${SOURCE_FILES} foo.hh)
That way, foo.hh is a dependency of foo_proj: and your command will be invoked when building foo_proj. It depends on ${SOURCE_FILES} and /tmp/bin/create_foo_hh main.cpp so that it is generated again if one of those files changes.
Regarding paths, add_custom_command() is configured to run in the current build directory to generate the file there, and include_directories() is used to add the build directory to the include dirs.
You probably don't want the custom target to depend on your source files (because they aren't targets themselves and are therefore never "run"), but on the target you create with them:
target_add_dependencies(foo_proj create_foo_hh)
I think that the cleanest is to add two new project() (targets) and then add the resulting file to your final executable. This is how cmake can build a valid dependency tree so when your source files change they get recompiled, the command run, as necessary to get everything up to date.
Build Executable
First, as you do in your example, I create an executable from some .cpp file:
(example extracted from the as2js project)
project(unicode-characters)
add_executable(${PROJECT_NAME}
unicode_characters.cpp
)
target_include_directories(${PROJECT_NAME}
PUBLIC
${ICU_INCLUDE_DIRS}
${SNAPDEV_INCLUDE_DIRS}
)
target_link_libraries(${PROJECT_NAME}
${ICU_LIBRARIES}
${ICU_I18N_LIBRARIES}
)
As we can see,m we can add specific include paths (-I) and library paths (-L). It is specific to that one target so you can have a set of paths that is different from the one used with your other executables.
Generate Additional File
Next, you create a custom command to run your executable like so:
project(unicode-character-types)
set(UNICODE_CHARACTER_TYPES_CI ${PROJECT_BINARY_DIR}/${PROJECT_NAME}.ci)
add_custom_command(
OUTPUT
${UNICODE_CHARACTER_TYPES_CI}
COMMAND
unicode-characters >${UNICODE_CHARACTER_TYPES_CI}
WORKING_DIRECTORY
${PROJECT_BINARY_DIR}
DEPENDS
unicode-characters
)
add_custom_target(${PROJECT_NAME}
DEPENDS
${UNICODE_CHARACTER_TYPES_CI}
)
Notice a couple of things:
I set a variable (UNICODE_CHARACTER_TYPES_CI) because I am going to reference that one file multiple times
a. Notice how I put the destination in the binary (cmake output folder) using the ${PROJECT_BINARY_DIR}/... prefix. This is best to avoid generating those files in your source tree (and possibly ending up adding that file to your source tracking system like svn or git).
b. An important aspect of the add_custom_command() is the DEPENDS section which includes the name of your special command, the one we defined in the previous step.
The add_custom_target() is what allows cmake to find your target and execute the corresponding command whenever one of the source files (a.k.a. dependency) changes; notice the DEPENDS definition.
Use the Output
Finally, here is the main project (a library in my case) that makes use of the file we generated in the step above.
Notice that I reference that file using the variable I defined in the previous step. That way, when I feel like changing that name, I can do it by simply editing that one variable.
project(as2js)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/version.h.in
${CMAKE_CURRENT_BINARY_DIR}/version.h
)
add_library(${PROJECT_NAME} SHARED
compiler/compiler.cpp
...
parser/parser_variable.cpp
${UNICODE_CHARACTER_TYPES_CI}
file/database.cpp
...
)
(Note: the ... represent a list of files, shorten for display here as these are not important, the link above will take you to the file with the complete list.)
By having the filename inside the list of files defined in the add_library() (or the add_executable() in your case), you create a dependency which will find your custom_target(), because of the filename defined in the OUTPUT section of the add_custom_command()¹.
¹ It is possible to defined multiple outputs for an add_custom_command(). For example, some of my generators output a .cpp and a .h. In that case, I simply define both files in the OUTPUT section.
Results
Important points about the final results with this solution:
the output files of your generator are saved in the binary output path instead of your current working directory
the Makefile generated by cmake includes all the necessary targets/dependencies which means changing any of the input files regenerate everything as expected (even if you just update a comment)
if the generator fails, the build fails as expected
the files are generated by the build step (make time) instead of the generation step (cmake time, like the execute_process() would do)

How can I use cmake to compile multiple source files and generate multiple executable files named after these file names

Currently, I need help either creating a CMakeList.txt, or simply figuring out the cmake command for the following.
I have some source files in the same directory, called A.cpp, B.cpp, C.cpp, D.cpp. I need to compile these such that the executables are named as A, B, C, D, respectively.
I want to use CMake to automatically traverse the directory and generate the corresponding executable file instead of adding the corresponding executable file in CMakeList.txt every time I add a file.
This is a bit of an odd request. Usually I'd suggest manually writing add_executable where needed since it's more maintainable.
In CMake, there isn't really a good way to collect all files in a directory. You can use file(GLOB ...) to grab all files; but this gets done at configure time, and if you introduce new sources then CMake won't detect the new source and won't reconfigure automatically or build the new sources without explicitly being told to reconfigure.
If you're able to discretely list each source, it would be better. But otherwise what you are requesting can be done with a combination of a foreach through each source file using get_filename_component to get the file name and passing it to add_executable
set(source_files src/a.cpp src/b.cpp src/c.cpp ...)
# Loop through each source file
foreach(source_file IN LISTS source_files)
# Get the name of the file without the extension (e.g. 'a' from src/a.cpp'
get_filename_component(target_name ${source_file} NAME_WE)
# Create an executable with the above name, building the above source
add_executable("${target_name}" "${source_file}"
endforeach()
If discretely listing the source files isn't possible, you can use file(GLOB...) or file(GLOB_RECURSE):
file(GLOB source_files "src/*.cpp")
But again; this prevents automatically detecting when new sources get added, and I don't recommend it.

How do I add files generated during the build to my include directory?

I'm trying to use a Python script to generate some header and source files for my C++ code and move them to the appropriate source directories, as the subsequent C++ build requires them.
Here's what the current iteration of my code looks like :
# Generate the Parameters and ParamParser files.
add_custom_command(
OUTPUT include/waveparameters.h
include/waveparser.h
src/waveparser.cpp
DEPENDS ${PROJECT_SOURCE_DIR}/scripts/wave.json
COMMAND python3 ${CMAKE_SOURCE_DIR}/scripts/genParams.py ${CMAKE_CURRENT_SOURCE_DIR}/scripts/wave.json
COMMENT "Generating custom Parameters files"
VERBATIM USES_TERMINAL
)
add_custom_target(PAR_GENERATION
DEPENDS include/waveparameters.h
include/waveparser.h
src/waveparser.cpp
)
set(MULTIGRID_INCLUDE_FILES
include/waveparameters.h
include/firstorderwave.h
include/waveparser.h
)
set(MULTIGRID_SOURCE_FILES
src/multiGridTest.cpp
src/firstorderwave.cpp
src/waveparser.cpp
)
set(SOURCE_FILES ${MULTIGRID_INCLUDE_FILES} ${MULTIGRID_SOURCE_FILES})
add_executable(MultiGridTest ${SOURCE_FILES})
target_include_directories(MultiGridTest PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/include)
target_include_directories(MultiGridTest PRIVATE ${CMAKE_SOURCE_DIR}/include)
target_link_libraries(MultiGridTest oops ${EXTRA_LIBS} m)
The current code generation works just fine.
However, no matter what directory I specify for the output (including ${CMAKE_CURRENT_SOURCE_DIR}/include or ${CMAKE_SOURCE_DIR}/MultiGrid/include and related paths), the generated .h and .cpp files always get moved to the directory my binary is generated in :
${CMAKE_SOURCE_DIR}/build/MultiGrid
rather than :
${CMAKE_SOURCE_DIR}MultiGrid/include` and `${CMAKE_SOURCE_DIR}/MultiGrid/src
I figured it out a while ago using Tsyvarev's advice in the comments. The solution was to add WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} as an argument to add_custom_command.

cmake: Copy files to dependent targets (specifically for use in OpenCL)

I am fiddling around with OpenCL, where files are typically compiled at runtime. The typical structure of a project looks like this:
foo/
cl/foo.cl
inc/foo.hpp
src/foo.cpp
CMakeLists.txt
where foo.cpp will have a command to build the OpenCL kernel. Something like:
buildKernelFromFile('cl/foo.cl'); // Note the relative path
To facilitate this, I have been setting up the CMakeLists.txt file to copy the .cl files to a cl/ folder in the binary directory:
set(kernels
foo.cl)
foreach (kernel IN ITEMS ${kernels})
configure_file(${CMAKE_CURRENT_LIST_DIR}/cl/${kernel} ${CMAKE_CURRENT_BINARY_DIR}/cl/${kernel} COPYONLY)
endforeach ()
The issue is that when the foo project is an executable, then this approach is fine since the .cl files will be copied to the same location as the executable. However, when the foo project is a library, this approach no longer works because the relative path is to the binary directory of the library, not the binary directory of the executable using the library.
So what I would like is a way to somehow tell CMake to copy the .cl files to the binary directory of any dependent targets. Specifically, if I build the OpenCL project like this
add_library(foo foo.cpp)
and then in a separate CMakeLists.txt file, link to that project:
add_executable(something main.cpp)
target_link_library(something PRIVATE foo)
I need to copy my .cl files to the build folder CMAKE_CURRENT_BINARY_DIR of the something target.
One thing I would like to avoid is having to save the .cl files as string literals and then using them like an include. (Ignore this comment if it does not make sense to you).
EDIT:
In the meantime, in the OpenCL library's CMakeLists.txt file (foo/CMakeLists.txt), I simply declared a function
# Copy the OpenCL kernels to the build folder
set(KERNEL_DIR ${CMAKE_CURRENT_LIST_DIR}/cl PARENT_SCOPE)
function(copy_kernels)
set(kernels
foo.cl)
foreach (kernel IN ITEMS ${kernels})
configure_file(${KERNEL_DIR}/${kernel} ${CMAKE_CURRENT_BINARY_DIR}/cl/${kernel} COPYONLY)
endforeach ()
endfunction()
which should be called by dependent projects. Then in the dependent project, in addition to linking to the foo target, you need to call this function
copy_kernels()
target_link_library(something PRIVATE foo)
I would like to figure out how to get rid of that copy_kernels() call, but I don't know cmake well enough to do that.
Frustratingly there isn't a neat way of doing this in CMake. As you say, configure_file should be exactly what we need, but it's not tied to a target and runs at configuration time (so you'd have to re-run cmake every time a kernel changed). The best I've been able to manage is this:
file(GLOB KERNELS cl/*.cl)
foreach(CurrentKernel IN LISTS KERNELS)
cmake_path(GET CurrentKernel FILENAME FileName)
add_custom_command(
TARGET foo PRE_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${CurrentKernel} ${CMAKE_CURRENT_BINARY_DIR}/cl/${FileName}
COMMENT "Copying kernel: ${CurrenKernel}")
endforeach()

CMake -- Add all sources in subdirectory to cmake project

As a follow up to this question:
Add Source in a subdirectory to a cmake project
What is the best way (perhaps using the FILE directive?) to select all the .cpp and .h files in the subdirectory and add them to the SOURCE variable defined in the parent directory?
Example from answer to question above:
set(SOURCE
${SOURCE}
${CMAKE_CURRENT_SOURCE_DIR}/file1.cpp
${CMAKE_CURRENT_SOURCE_DIR}/file2.cpp
PARENT_SCOPE
)
set(HEADERS
${HEADERS}
${CMAKE_CURRENT_SOURCE_DIR}/file1.hpp
${CMAKE_CURRENT_SOURCE_DIR}/file2.hpp
PARENT_SCOPE
)
Is it possible to do something like this?
FILE(GLOB SUB_SOURCES *.cpp)
set(SOURCE
${SOURCE}
${CMAKE_CURRENT_SOURCE_DIR}/${SUB_SOURCES}
PARENT_SCOPE
)
What is the best way (using CMake) to compile all the sources in a directory and a subdirectory into a single output file (not multiple libraries?)
I think what you are looking for is the aux_source_directory command.
aux_source_directory Find all source files in a directory.
aux_source_directory( )
Collects the names of all the source files in the specified directory
and stores the list in the provided. This command is
intended to be used by projects that use explicit template
instantiation. Template instantiation files can be stored in a
"Templates" subdirectory and collected automatically using this
command to avoid manually listing all instantiations.
It is tempting to use this command to avoid writing the list of source
files for a library or executable target. While this seems to work,
there is no way for CMake to generate a build system that knows when a
new source file has been added. Normally the generated build system
knows when it needs to rerun CMake because the CMakeLists.txt file is
modified to add a new source. When the source is just added to the
directory without modifying this file, one would have to manually
rerun CMake to generate a build system incorporating the new file.
Your CMakeLists.txt within the subdirectory could look like this:
aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} SUB_SOURCES)
set(SOURCE
${SOURCE}
${SUB_SOURCES}
PARENT_SCOPE
)
The recommended practice is however, as you see from the documentation, to list the files individually within CMakeLists.txt as changes to the CMakeLists.txt file triggers running cmake.
I hope this was helpful and to the point.