Create CMake target that copies files/dirs but excludes some items - c++

I am trying to generate a CMake target for a C++ project using add_custom_target that copies the contents of the directory in which the CMakeLists.txt resides into ${CMAKE_BINARY_DIR}, but excludes a given list of files.
While it looks like a quite easy task, I'm facing problems here. My attempts so far:
1) Generate a list of all files/dirs, remove the items to be excluded and use the CMake copy command:
file(GLOB files_to_copy LIST_DIRECTORIES TRUE *)
list(REMOVE_ITEM files_to_copy
${CMAKE_CURRENT_SOURCE_DIR}/file_to_exclude.txt
${CMAKE_CURRENT_SOURCE_DIR}/dir_to_exclude
# ...
)
add_custom_target(my-target
COMMAND ${CMAKE_COMMAND} -E copy ${files_to_copy} ${CMAKE_BINARY_DIR}
)
Problems:
Removing items this way works on a string comparison level, which could lead to problems when using symbolic links, for example
The copy command line tool apparently supports copying directories, however it doesn't seem to work on my machine, therefore directories do not get copied.
2) Use the file command to copy the files, excluding some files
file(COPY "${CMAKE_CURRENT_SOURCE_DIR}/" DESTINATION ${CMAKE_BINARY_DIR}
PATTERN "file_to_exclude.txt" EXCLUDE
PATTERN "dir_to_exclude" EXCLUDE
)
Problems:
This does exactly what I want, except that the command is not bound to a target and therefore is always executed. But I only need the copy operation to be a target.
Is there any possibility to bind the file(COPY ...) command to a target? Or any other straightforward solution to achieve what I want?
Please note that I only want to use CMake built-in tools and not execute any OS-specific shell commands (working on a cross-platform project).

CMake scripting commands work only in CMake context and are executed immediately.
But a COMMAND in add_custom_command (or add_custom_target) is executed in the context of a build tool (e.g. Makefile), not in CMake context.
However, you may put CMake scripting commands into separate CMake script, and call this script from the COMMAND. This solution has the same platform-independent properties as CMakeLists.txt itself.
You may parameterize separate script either:
With configure_file CMake command.
By passing -D parameters to CMake when call the script.
The first approach is quite simple: you write the script as would you write CMakeLists.txt itself. But it generates additional files for every parametrization set.
The second approach is useful for multi-purpose (or multi-usable) scripts, as it doesn't create additional copy of the script for every usage. But it requires some design of the parameters.
Using 'configure_file'
copy_sources.cmake.in (as if commands are written in CMakeLists.txt):
file(COPY "${CMAKE_CURRENT_SOURCE_DIR}/" DESTINATION ${CMAKE_BINARY_DIR}
PATTERN "file_to_exclude.txt" EXCLUDE
PATTERN "dir_to_exclude" EXCLUDE
)
CMakeLists.txt:
# Instantiate the parameterized script in the binary directory.
configure_file(copy_sources.cmake.in copy_sources.cmake)
add_custom_target(my-target
COMMAND ${CMAKE_COMMAND} -P ${CMAKE_BINARY_DIR}/copy_sources.cmake
)
Using '-D' parameters
copy_sources.cmake:
# Expect the script to be called from the source directory.
# This saves one parameter.
#
# DEST_DIR denotes the directory for copy to.
file(COPY "${CMAKE_SOURCE_DIR}/" DESTINATION ${DEST_DIR}
PATTERN "file_to_exclude.txt" EXCLUDE
PATTERN "dir_to_exclude" EXCLUDE
)
CMakeLists.txt:
add_custom_target(my-target
COMMAND ${CMAKE_COMMAND}
-DDEST_DIR=${CMAKE_BINARY_DIR} # all '-D' options should preceed '-P' one
-P ${CMAKE_CURRENT_SOURCE_DIR}/copy_sources.cmake
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} # The script assumes proper current directory
)

Related

CMake: Platform independent binary stripping for release builds

How can you add binary stripping to CMake for gcc and clang as a post-processing step but only in release builds? MSVC strips by default so it does not have to be handled.
One of the problems is that clang does not have a -s compilation flag but gcc does so doing it this way does not work.
Another idea is to use the strip command. The -s switch again exists on Linux but not on XCode (this is Apple's development toolchain).
So the final choice is to use the strip command without any arguments besides the binary itself which appears to be a decent generic solution. How can this be used in CMake?
In CMake add_custom_command can execute post build commands such as strip. The PROJECT_NAME variable is defined as your target binary e.g.
# Set the project name
set(PROJECT_NAME "MyProject")
project(${PROJECT_NAME})
Place the following code after add_executable of your CMakeLists.txt:
# Strip binary for release builds
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND $<$<CONFIG:release>:${CMAKE_STRIP} ${PROJECT_NAME}>)
CMAKE_STRIP is the file path to the platform's strip utility (e.g. /usr/bin/strip on Linux). The $<$<CONFIG:cfg>:str> generator expression will expand to str when building the configuration cfg, and to an empty string otherwise. In this scenario, this directly means "call strip when building in Release, and do nothing otherwise". Note that the CONFIG generator is case insensitive on your build type, and will work when using multi-config generators.
While using add_custom_command, I was having issues with the COMMAND generator expression containing spaces. The ARGS option can be used while still toggling the inclusion of the custom command through a generator expression for the COMMAND option:
add_custom_command(
TARGET "${TARGET}" POST_BUILD
DEPENDS "${TARGET}"
COMMAND $<$<CONFIG:release>:${CMAKE_STRIP}>
ARGS --strip-all $<TARGET_FILE:${TARGET}>
)
This results in strip --strip-all myapp being called when config is release, but no command being called when it's not, despite the args being specified in a separate option.

Copy out plain .o files with cmake

I'm trying to get cmake (on linux) to create some static object (.o) files and install them to an external directory. To that end I have a list, object_sources, containing the project path of the sources, and put this in the top level CMakeLists.txt:
set(local_objects "")
foreach(file ${object_sources})
get_filename_component(cur ${file} NAME_WE)
add_library("${cur}.o" OBJECT ${file})
list(APPEND local_objects "${cur}.o")
# To confirm these variables are what I think they are:
message("---> ${cur} | ${file}")
endforeach()
install(
# Also tried FILES, see below.
TARGETS ${local_objects}
CONFIGURATIONS Debug
# Also tried nothing instead of 'OBJECTS' (same outcome)
# and LIBRARY (cmake fails on error).
OBJECTS
DESTINATION "/some/external/dir"
)
With regard to using FILES, it cannot work because in fact a foo.o is never created, so there is no point in me trying to get the correct project path.
When I get to the actual make command in the cmake build directory, it spits out Built target foo.o, etc., during the process. However, as mentioned, no foo.o is ever created anywhere, and the install creates an objects-Debug directory in /some/external/dir containing directories for each target using its name (e.g., foo.o/) and inside that some more nested directories at the bottom of which is the object I want, with the wrong name: foo.cpp.o.
Cmake can structure its build tree however it wants and name its temporaries whatever it likes, but I would really like to be able to ask for a final product object foo.o (not foo.cpp.o) and have it installed that way.
Is that possible? I'm beginning to think this is all very contrary to the (implicit) purposes of cmake, and that I should keep these kinds of tasks separate in a shell script or makefile (it takes about three lines in either), and just use cmake for the normative libs and executables.
Or: I believe I could get this to work if I created each object as a static archive (lib.a) instead, but part of the issue here is that would require modifying the build scripts of something else, or else extracting them via a script (in which case I might as well do the whole thing in the script).
I'm not aware of a good way to do this in CMake, but maybe the least fussy would be to use an OBJECT library. This would allow you to get the object files for a target using generator expressions, more specifically the TARGET_OBJECTS expression. This is a minimal example with a single foo.c file containing an empty main:
cmake_minimum_required(VERSION 3.11)
project(foo C)
set(SOURCES foo.c)
add_library(foo_obj OBJECT ${SOURCES})
add_executable(foo $<TARGET_OBJECTS:foo_obj>)
set(OBJS $<TARGET_OBJECTS:foo_obj>)
message(STATUS "these are my objects: ${OBJS}") # generator expression printed here, not evaluated yet
add_custom_target(print_foo_objs
COMMAND ${CMAKE_COMMAND} -E echo $<TARGET_OBJECTS:foo_obj>)
set(OBJ_ROOT_DIR $<TARGET_PROPERTY:foo,BINARY_DIR>)
add_custom_target(copy_foo_objs
COMMAND ${CMAKE_COMMAND} -E make_directory ${OBJ_ROOT_DIR}/myobjects/
COMMAND ${CMAKE_COMMAND} -E copy_if_different $<TARGET_OBJECTS:foo_obj> ${OBJ_ROOT_DIR}/myobjects/
COMMAND_EXPAND_LISTS) # UPDATE
Please note that as a generator expression, you will not have access to it during configuration (hence, that message will contain the generator expression as it was textually declared), but only during the generation phase.
If you still need to rename the object files (i.e. from foo.c.o to foo.o) you can maybe call a custom external script to do that for you.
Moreover, there might be issues with cleanup, etc. which will force you to add more custom targets and/or commands (and associated dependencies) to deal with it. Another downside of this approach.
UPDATE: COMMAND_EXPAND_LISTS allows the expansion of lists in the generator expression and it was added in CMake 3.8 and later (see here).

Copying assets directory from source to build directory in CMake

I am trying to write a game for practice in C++. I am using CMake for this, and my project gets built in a separate build directory automatically by my IDE. However, my assets folder is not being copied over to the new directory.
Currently, I am attempting to resolve this with the following:
add_custom_command(TARGET ${PROJECT_NAME}
POST_BUILD
COMMAND cp -r ${PROJECT_SOURCE_DIR}/assets ${CMAKE_BINARY_DIR}
)
However, this seems to have absolutely no effect on the outcome and the assets directory is still not present in my build directory.
How can I actually make it copy this asset directory to my build location? Is there a smarter way to point my program to my assets location (perhaps one that will also work with assets in /usr/share?)
After quite a while of more searching, I managed to find the file(COPY {files} DESTINATION {directory}) command:
file(COPY assets DESTINATION ${CMAKE_BINARY_DIR})
Two ways I've found, both ensuring that it copies on target build and stays up to date even if the target doesn't need to re-build.
The first uses the CMake command line tool copy_directory. It's unclear from the documentation whether this copy always happens, regardless if files in the assets directory were updated or not, or if it only copies when the asset files don't exist or an asset file was updated:
add_custom_target(copy_assets
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_LIST_DIR}/assets ${CMAKE_CURRENT_BINARY_DIR}/assets
)
add_dependencies(mytarget copy_assets)
The second requires a small additional CMake script file, but it allows us to use the file(COPY ...) CMake comamnd, which states in the documentation that "copying preserves input file timestamps, and optimizes out a file if it exists at the destination with the same timestamp."
First, create the CMake script file copy-assets.cmake in the same directory as your CMakeLists.txt, with the contents being (similar to Ethan McTague's answer)
file(COPY ${CMAKE_CURRENT_LIST_DIR}/assets DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
And then in your CMakeLists.txt, add the following:
add_custom_target(copy_assets
COMMAND ${CMAKE_COMMAND} -P ${CMAKE_CURRENT_LIST_DIR}/copy-assets.cmake
)
add_dependencies(mytarget copy_assets)
The solution above using add_custom_target is suitable for most cases, but has a few problems.
Does not support dependent generator expressions (e.g. $<TARGET_FILE_DIR:tgt>)
Re-copies every build (slow for numerous/large files)
Is not particularly readable, especially for copying from multiple locations
Instead, I've been using file GENERATE
set(ASSETS
assetfile
...)
foreach(ASSET ${ASSETS})
file(GENERATE OUTPUT ${ASSET} INPUT ${ASSET})
endforeach()
Note I haven't needed generator expressions yet, but it looks like the latest version of CMake (3.19) adds generator expression support to this function.
Here is what I do with vulkan shaders. Simply replace the compilation step with a copy command:
macro(add_shaders)
cmake_parse_arguments("MY" "TARGET" "SOURCES" ${ARGN})
foreach(src ${MY_SOURCES})
set(OUTF "${CMAKE_CURRENT_BINARY_DIR}/shaders/${src}.spv")
get_filename_component(PARENT_DIR "${OUTF}" DIRECTORY)
add_custom_command(
OUTPUT "${OUTF}"
COMMAND ${CMAKE_COMMAND} -E make_directory "${PARENT_DIR}"
COMMAND "${Vulkan_GLSLC_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/${src}" -o "${OUTF}"
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/${src}"
VERBATIM
)
# src in order to get listed in the IDE, OUTF to declare the build relationship
target_sources("${MY_TARGET}" PRIVATE "${src}" "${OUTF}")
endforeach()
endmacro()
add_shaders(TARGET my_target SOURCES asset1 asset2 asset3 ...)
You'll have to specify each asset on its own (or loop over a list). Including wildcards into a build is a bad idea [tm]. This will also only copy/process stuff if it has changed.

How to get current configuration (Release/Debug) in CMake for Visual Studio

I am on Visual Studio 2013, CMake 3.5.1, Windows 10. I am trying to copy some files via CMake like below:
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/Release)
Is it possible to replace "Release" with a variable that represents the configuration like:
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/${Variable})
I attempted
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/${CMAKE_BUILD_TYPE})
but CMAKE_BUILD_TYPE is an empty string when I use message to print it out, I also attempted
file(COPY ${IMAGES} DESTINATION ${CMAKE_BINARY_DIR}/bin/$<CONFIGURATION>)
but for some reason file command cannot decipher $<CONFIGURATION> whereas command like
add_custom_target(run COMMAND ${CMAKE_BINARY_DIR}/bin/$<CONFIGURATION>/Test.exe)
can. What is the right way to extract whether visual studio is currently built in Release or Debug in CMake?
The file command is executed during CMake runtime, not during build time (i.e. VS runtime).
This also means, that the generator expressions (e.g. $<CONFIG>) can not be used, as these are evaluated during build time.
(Hint: As long as there is no explicit reference to the use of generator expressions for a particular command in the CMake docu, they are not supported by that command).
The reason, why ${CMAKE_BUILD_TYPE} is empty, is due to the reason that you probably haven't specified it on the invocation of CMake:
cmake -DCMAKE_BUILD_TYPE=Debug ..
However, using that, would mean that the build files are only generated for the Debug configuration. That's obviously not what you want.
To solve your problem: Using generator expressions is the right way, as you've already figured out with the use of add_custom_target (or add_custom_command).
You can use custom commands as dependencies for other "real" targets and you can specify post-/pre-build and pre-link commands for a specific target via add_custom_command.
As the docu states for the COMMAND argument of add_custom_command:
Arguments to COMMAND may use generator expressions. References to target names in generator expressions imply target-level dependencies, but NOT file-level dependencies. List target names with the DEPENDS option to add file-level dependencies.
To copy a file after a successful build of a target:
add_custom_command(TARGET myTarget POST_BUILD
COMMAND "${CMAKE_COMMAND}" -E copy_if_different "${IMAGE1}" "${CMAKE_BINARY_DIR}/bin/$<CONFIG>/"
COMMAND "${CMAKE_COMMAND}" -E copy_if_different "${IMAGE2}" "${CMAKE_BINARY_DIR}/bin/$<CONFIG>/"
)

CMAKE: performing a file copy AFTER a custom command has run

I have a build script fragment that looks as follows:
foreach(...)
...
add_custom_command( OUTPUT ${fn_c} ${fn_s} ${fn_p_c} {fn_p_h}
COMMAND ${PROTOBUF_PROTOC_EXECUTABLE} --proto_path=${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/${proto_var} --cpp_out=. --plugin=protoc-gen-RBLRPC=${CMAKE_BINARY_DIR}/tools/protoc-gen-RBLRPC --RBLRPC_out=.
DEPENDS ${proto_var}
)
if(${M_S_} OR ${M_C_})
set(MARSHALL_RPC_FILES ${MARSHALL_RPC_FILES} ${fn_p_c})
message(status "copy marshall -------------------")
file(COPY ${CMAKE_CURRENT_BINARY_DIR}/${fn_c}
${CMAKE_CURRENT_BINARY_DIR}/${fn_s}
${CMAKE_CURRENT_BINARY_DIR}/${fn_p_h} DESTINATION ${CMAKE_SOURCE_DIR}/include/rpc/marshall)
endif()
...
endforeach(...)
The copied files are not generated untill the custom comand is executed, however cmake attempts to copy the files upon first pass over the script. I'd welcome any suggestions to solve this problem , without drastically changing my scrips.
Don't use file(COPY...) function, but add the following command to your add_custom_command:
COMMAND ${CMAKE_COMMAND} copy ${CMAKE_CURRENT_BINARY_DIR}/${fn_c}
${CMAKE_SOURCE_DIR}/include/rpc/marshall
But for what you're going to do, I would suggest you keep your source tree clean, add directly use the generated files from the build directory. That would break, for instance, if you want to make two different build tree from a single source tree.
edit :
CMAKE_COMMAND is documented in the variable section of the online man-page documentation, (search for CMAKE_COMMAND and not ${CMAKE_COMMAND}.
On the command line CMAKE -E will show you a list of portable commands useable.