cmake: Copy files to dependent targets (specifically for use in OpenCL) - c++

I am fiddling around with OpenCL, where files are typically compiled at runtime. The typical structure of a project looks like this:
foo/
cl/foo.cl
inc/foo.hpp
src/foo.cpp
CMakeLists.txt
where foo.cpp will have a command to build the OpenCL kernel. Something like:
buildKernelFromFile('cl/foo.cl'); // Note the relative path
To facilitate this, I have been setting up the CMakeLists.txt file to copy the .cl files to a cl/ folder in the binary directory:
set(kernels
foo.cl)
foreach (kernel IN ITEMS ${kernels})
configure_file(${CMAKE_CURRENT_LIST_DIR}/cl/${kernel} ${CMAKE_CURRENT_BINARY_DIR}/cl/${kernel} COPYONLY)
endforeach ()
The issue is that when the foo project is an executable, then this approach is fine since the .cl files will be copied to the same location as the executable. However, when the foo project is a library, this approach no longer works because the relative path is to the binary directory of the library, not the binary directory of the executable using the library.
So what I would like is a way to somehow tell CMake to copy the .cl files to the binary directory of any dependent targets. Specifically, if I build the OpenCL project like this
add_library(foo foo.cpp)
and then in a separate CMakeLists.txt file, link to that project:
add_executable(something main.cpp)
target_link_library(something PRIVATE foo)
I need to copy my .cl files to the build folder CMAKE_CURRENT_BINARY_DIR of the something target.
One thing I would like to avoid is having to save the .cl files as string literals and then using them like an include. (Ignore this comment if it does not make sense to you).
EDIT:
In the meantime, in the OpenCL library's CMakeLists.txt file (foo/CMakeLists.txt), I simply declared a function
# Copy the OpenCL kernels to the build folder
set(KERNEL_DIR ${CMAKE_CURRENT_LIST_DIR}/cl PARENT_SCOPE)
function(copy_kernels)
set(kernels
foo.cl)
foreach (kernel IN ITEMS ${kernels})
configure_file(${KERNEL_DIR}/${kernel} ${CMAKE_CURRENT_BINARY_DIR}/cl/${kernel} COPYONLY)
endforeach ()
endfunction()
which should be called by dependent projects. Then in the dependent project, in addition to linking to the foo target, you need to call this function
copy_kernels()
target_link_library(something PRIVATE foo)
I would like to figure out how to get rid of that copy_kernels() call, but I don't know cmake well enough to do that.

Frustratingly there isn't a neat way of doing this in CMake. As you say, configure_file should be exactly what we need, but it's not tied to a target and runs at configuration time (so you'd have to re-run cmake every time a kernel changed). The best I've been able to manage is this:
file(GLOB KERNELS cl/*.cl)
foreach(CurrentKernel IN LISTS KERNELS)
cmake_path(GET CurrentKernel FILENAME FileName)
add_custom_command(
TARGET foo PRE_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${CurrentKernel} ${CMAKE_CURRENT_BINARY_DIR}/cl/${FileName}
COMMENT "Copying kernel: ${CurrenKernel}")
endforeach()

Related

How can i link a custom library with Cmake? [duplicate]

We use CMake for generating the Visual Studio files of our sources in our SVN. Now my tool requires some DLL files to be in the same folder as the executable. The DLL files are in a folder alongside the source.
How can I change my CMakeLists.txt such that the generated Visual Studio project will either have already the particular DLL files in the release/debug folders or will copy them upon compilation?
I'd use add_custom_command to achieve this along with cmake -E copy_if_different.... For full info run
cmake --help-command add_custom_command
cmake -E
So in your case, if you have the following directory structure:
/CMakeLists.txt
/src
/libs/test.dll
and your CMake target to which the command applies is MyTest, then you could add the following to your CMakeLists.txt:
add_custom_command(TARGET MyTest POST_BUILD # Adds a post-build event to MyTest
COMMAND ${CMAKE_COMMAND} -E copy_if_different # which executes "cmake - E copy_if_different..."
"${PROJECT_SOURCE_DIR}/libs/test.dll" # <--this is in-file
$<TARGET_FILE_DIR:MyTest>) # <--this is out-file path
If you just want the entire contents of the /libs/ directory copied, use cmake -E copy_directory:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs"
$<TARGET_FILE_DIR:MyTest>)
If you need to copy different dlls depending upon the configuration (Release, Debug, eg) then you could have these in subdirectories named with the corresponding configuration: /libs/Release, and /libs/Debug. You then need to inject the configuration type into the path to the dll in the add_custom_command call, like this:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I put these lines in my top-level CMakeLists.txt file. All the libraries and executables compiled by CMake will be placed in the top level of the build directory so that the executables can find the libraries and it is easy to run everything.
set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
Note that this doesn't solve the OP's problem of copying precompiled binaries from the project's source directory.
I've had this problem today when tried to make a Windows build of my program. And I ended up doing some research myself since all these answers didn't satisfy me. There were three main issues:
I wanted debug builds to be linked with debug versions of libraries
and release builds to be linked with release builds of libraries,
respectively.
In addition to that, I wanted correct versions of DLL files
(Debug/Release) to be copied to output directories.
And I wanted to achieve all this without writing complex and fragile scripts.
After browsing some CMake manuals and some multiplatform projects at github I've found this solution:
Declare your library as a target with "IMPORTED" attribute, reference its debug and release .lib and .dll files.
add_library(sdl2 SHARED IMPORTED GLOBAL)
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_RELEASE "${SDL_ROOT_PATH}/lib/SDL2.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_RELEASE "${SDL_ROOT_PATH}/bin/SDL2.dll")
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_DEBUG "${SDL_ROOT_PATH}/lib/SDL2d.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_DEBUG "${SDL_ROOT_PATH}/bin/SDL2d.dll")
Link this target with your project as usual
target_link_libraries(YourProg sdl2 ...)
Make custom build step to copy dll file to its destination if it has been altered somehow since previous build
add_custom_command ( TARGET YourProg POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE:sdl2> $<TARGET_FILE_DIR:YourProg>
)
For Windows users, there is a new generator expression $<TARGET_RUNTIME_DLLS:tgt> in CMake 3.21+ and you could use this official snippet for copying all of the DLLs that a target depends on.
find_package(foo REQUIRED)
add_executable(exe main.c)
target_link_libraries(exe PRIVATE foo::foo foo::bar)
add_custom_command(TARGET exe POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe>
COMMAND_EXPAND_LISTS
)
Moving files during build using install
I had this issue trying to follow the CMake official tutorial on Step 9. This was the location of the file I wanted to move:
src
|_build
|_Debug
- `MathFunctions.dll`
This was the location I wanted the file to be in:
src
|_build
|_install
|_bin
- `MathFunctions.dll`
Since this DLL was generated as a shared library, all I did was to include this line in the CMakeLists.txt in the subdirectory that contained the source code for the library src/Mathfunctions/CMakeLists.txt
install(FILES ${PROJECT_BINARY_DIR}/$<CONFIG>/MathFunctions.dll
DESTINATION bin)
Thanks to your answers I could think on this one. Is just one line, so I think is ok. The $<CONFIG> can have two values Debug or Release Depending on how the project is built, as the original question required.
You can also use the command find_library:
find_library(<some_var> NAMES <name_of_lib> PATHS "<path/to/lib>")
With a defined EXECUTABLE_PATH, for instance:
set(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin)
you could move the .dll files that your executable need, with
file(COPY ${<some_var>}
DESTINATION ${EXECUTABLE_OUTPUT_PATH})
An addendum to the accepted answer, added as a separate answer so I get code formatting:
If you are building your dlls in the same project, they will usually be in Release, Debug, etc. directories. You'll have to use the Visual Studio environment variables to correctly copy them. e.g.:
"${PROJECT_BINARY_DIR}/your_library/\$\(Configuration\)/your_library.dll"
for the source and
"${CMAKE_CURRENT_BINARY_DIR}/\$\(Configuration\)/your_library.dll"
for the destination. Note the escaping!
You can't use the CMake CMAKE_BUILD_TYPE variable for the configuration since it's resolved at VS project generation time and will always be whatever the default is.
This is useful for one of them
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/lib CACHE
PATH "Directory where all the .lib files are dumped." FORCE)
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/bin CACHE
PATH "Directory where .exe and .dll files are dumped." FORCE)
1. The most correct way: TARGET_RUNTIME_DLLS (CMake >= 3.21)
install(FILES $<TARGET_RUNTIME_DLLS:your_exe_here> TYPE BIN)
For this to work, your dependencies' CMake modules have to be well-written. In other words, they use CMake 3 targets with all their target properties set up correctly. If they set up everything right, all transitively-linked DLLs will be automagically gathered up and installed alongside your exe.
A big difference between this and the straight-up copy-the-files approach is that, because it goes through install(), CMake will actually know about them as files to-be-installed. CPack will know about your DLLs and include them in any installer you generate with it. CMake will automatically adjust what type of DLL (release vs debug) to match your target exe.
This is where CMake will be headed more in the future, and the way you should prefer if you have a choice.
2. The second most correct way: RUNTIME_DEPENDENCIES (CMake >= 3.21)
install(TARGETS your_exe_here
RUNTIME ARCHIVE LIBRARY RUNTIME FRAMEWORK BUNDLE PUBLIC_HEADER RESOURCE)
install(TARGETS your_exe_here
COMPONENT your_exe_here
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
DIRECTORIES $<TARGET_FILE_DIR:your_exe_here>)
The key here is RUNTIME_DEPENDENCIES.
Internally, RUNTIME_DEPENDENCIES calls file(GET_RUNTIME_DEPENDENCIES), which scans your executable binary, tries very hard to exactly replicate what actual dependency resolution would look like, and write down all the DLLs mentioned along the way. These are passed back up to install().
What this means is that this doesn't depend on your dependencies' CMake modules having their target properties set up correctly. Your actual executable binary is scanned. Everything will get picked up.
3. The third most correct way: install(DIRECTORY)
install(
DIRECTORY "${DIR_CONTAINING_YOUR_DLLS}"
TYPE BIN
FILES_MATCHING REGEX "[^\\\\/.]\\.[dD][lL][lL]$"
)
To use, put the DLLs appropriate for your build in $DIR_CONTAINING_YOUR_DLLS.
The trick here is that, unlike install(FILES), install(DIRECTORY) doesn't care what specific files are in the directory until install time. That means now we have all of configure time and compile time to get a list of your DLLs and stuff them in $DIR_CONTAINING_YOUR_DLLS. As long as the DLL files are in $DIR_CONTAINING_YOUR_DLLS by install time, install(DIRECTORY) will pick them up.
If you choose this method, it's becomes your responsibility to match DLLs to your build config. (Consider: static vs dynamic, debug vs release, import lib version vs DLL version, libs with optional multithreading, forgetting to remove DLLs you don't need anymore.)
If you choose this method, you might want to automate DLL finding and matching using something like what vcpkg's applocal.ps1 does.
Hint for vcpkg
If you use vpckg with VCPKG_APPLOCAL_DEPS enabled, vcpkg will locate and copy your DLLs into your $CMAKE_RUNTIME_OUTPUT_DIRECTORY for you, but without going through install(). You need to use the install(DIRECTORY) trick to get CMake to pick them up.
(Internally, vcpkg uses dumpbin, llvm-objdump, and objdump to scan your executable binary to get these filenames.)
You probably need to add custom target and make it depend on one of your executable targets.
To copy file using above function use:
COMMAND ${CMAKE_PROGRAM} -E copy_if_different ${CMAKE_BINARY_DIR}/path/to/file.dll ${CMAKE_BINARY_DIR}/where/to/put/file.dll`
The following command from the currently top rated answer depends on the output being put in /libs/.
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I use the below command to do the copy, which works for me everywhere. Note that I'm only copying the output dll here, not the entire directory. Also note that I'm hard coding a destination /bin/ directory, which is specific to this project. But I wanted to share the $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}> syntax, which I think is neat:
add_custom_command(TARGET ${CMAKE_PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>
${CMAKE_CURRENT_SOURCE_DIR}/bin/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>)
I'm a CMake beginner, but still I wanted to shared my experience. In my case I needed a post-install copy so that all my binaries are in.
In the case of third-party binary that can be imported within CMake, the following works for me:
find_package( dependency REQUIRED )
if( MSVC )
# If done properly and if the dependency has a correct config file, IMPORTED_LOCATION_RELEASE should be defined
get_target_property( DEP_SHARED_LIB_PATH dependency IMPORTED_LOCATION_RELEASE )
# Create a bin directory in the install folder
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_INSTALL_PREFIX}/bin/)
# Copy the shared lib file
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy ${DEP_SHARED_LIB_PATH} ${CMAKE_INSTALL_PREFIX}/bin/)
endif()
Obviously IMPORTED_LOCATION_RELEASE can have variants depending on how the shared library was built / installed. Could be IMPORTED_LOCATION_DEBUG.
Maybe there's a better way to get that property name, I don't know.

Automatically copy dlls to output directory CPP Using Poco [duplicate]

We use CMake for generating the Visual Studio files of our sources in our SVN. Now my tool requires some DLL files to be in the same folder as the executable. The DLL files are in a folder alongside the source.
How can I change my CMakeLists.txt such that the generated Visual Studio project will either have already the particular DLL files in the release/debug folders or will copy them upon compilation?
I'd use add_custom_command to achieve this along with cmake -E copy_if_different.... For full info run
cmake --help-command add_custom_command
cmake -E
So in your case, if you have the following directory structure:
/CMakeLists.txt
/src
/libs/test.dll
and your CMake target to which the command applies is MyTest, then you could add the following to your CMakeLists.txt:
add_custom_command(TARGET MyTest POST_BUILD # Adds a post-build event to MyTest
COMMAND ${CMAKE_COMMAND} -E copy_if_different # which executes "cmake - E copy_if_different..."
"${PROJECT_SOURCE_DIR}/libs/test.dll" # <--this is in-file
$<TARGET_FILE_DIR:MyTest>) # <--this is out-file path
If you just want the entire contents of the /libs/ directory copied, use cmake -E copy_directory:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs"
$<TARGET_FILE_DIR:MyTest>)
If you need to copy different dlls depending upon the configuration (Release, Debug, eg) then you could have these in subdirectories named with the corresponding configuration: /libs/Release, and /libs/Debug. You then need to inject the configuration type into the path to the dll in the add_custom_command call, like this:
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I put these lines in my top-level CMakeLists.txt file. All the libraries and executables compiled by CMake will be placed in the top level of the build directory so that the executables can find the libraries and it is easy to run everything.
set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
Note that this doesn't solve the OP's problem of copying precompiled binaries from the project's source directory.
I've had this problem today when tried to make a Windows build of my program. And I ended up doing some research myself since all these answers didn't satisfy me. There were three main issues:
I wanted debug builds to be linked with debug versions of libraries
and release builds to be linked with release builds of libraries,
respectively.
In addition to that, I wanted correct versions of DLL files
(Debug/Release) to be copied to output directories.
And I wanted to achieve all this without writing complex and fragile scripts.
After browsing some CMake manuals and some multiplatform projects at github I've found this solution:
Declare your library as a target with "IMPORTED" attribute, reference its debug and release .lib and .dll files.
add_library(sdl2 SHARED IMPORTED GLOBAL)
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_RELEASE "${SDL_ROOT_PATH}/lib/SDL2.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_RELEASE "${SDL_ROOT_PATH}/bin/SDL2.dll")
set_property(TARGET sdl2 PROPERTY IMPORTED_IMPLIB_DEBUG "${SDL_ROOT_PATH}/lib/SDL2d.lib")
set_property(TARGET sdl2 PROPERTY IMPORTED_LOCATION_DEBUG "${SDL_ROOT_PATH}/bin/SDL2d.dll")
Link this target with your project as usual
target_link_libraries(YourProg sdl2 ...)
Make custom build step to copy dll file to its destination if it has been altered somehow since previous build
add_custom_command ( TARGET YourProg POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE:sdl2> $<TARGET_FILE_DIR:YourProg>
)
For Windows users, there is a new generator expression $<TARGET_RUNTIME_DLLS:tgt> in CMake 3.21+ and you could use this official snippet for copying all of the DLLs that a target depends on.
find_package(foo REQUIRED)
add_executable(exe main.c)
target_link_libraries(exe PRIVATE foo::foo foo::bar)
add_custom_command(TARGET exe POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe>
COMMAND_EXPAND_LISTS
)
Moving files during build using install
I had this issue trying to follow the CMake official tutorial on Step 9. This was the location of the file I wanted to move:
src
|_build
|_Debug
- `MathFunctions.dll`
This was the location I wanted the file to be in:
src
|_build
|_install
|_bin
- `MathFunctions.dll`
Since this DLL was generated as a shared library, all I did was to include this line in the CMakeLists.txt in the subdirectory that contained the source code for the library src/Mathfunctions/CMakeLists.txt
install(FILES ${PROJECT_BINARY_DIR}/$<CONFIG>/MathFunctions.dll
DESTINATION bin)
Thanks to your answers I could think on this one. Is just one line, so I think is ok. The $<CONFIG> can have two values Debug or Release Depending on how the project is built, as the original question required.
You can also use the command find_library:
find_library(<some_var> NAMES <name_of_lib> PATHS "<path/to/lib>")
With a defined EXECUTABLE_PATH, for instance:
set(EXECUTABLE_OUTPUT_PATH ${PROJECT_BINARY_DIR}/bin)
you could move the .dll files that your executable need, with
file(COPY ${<some_var>}
DESTINATION ${EXECUTABLE_OUTPUT_PATH})
An addendum to the accepted answer, added as a separate answer so I get code formatting:
If you are building your dlls in the same project, they will usually be in Release, Debug, etc. directories. You'll have to use the Visual Studio environment variables to correctly copy them. e.g.:
"${PROJECT_BINARY_DIR}/your_library/\$\(Configuration\)/your_library.dll"
for the source and
"${CMAKE_CURRENT_BINARY_DIR}/\$\(Configuration\)/your_library.dll"
for the destination. Note the escaping!
You can't use the CMake CMAKE_BUILD_TYPE variable for the configuration since it's resolved at VS project generation time and will always be whatever the default is.
This is useful for one of them
SET(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/lib CACHE
PATH "Directory where all the .lib files are dumped." FORCE)
SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${PROJECT_SOURCE_DIR}/bin CACHE
PATH "Directory where .exe and .dll files are dumped." FORCE)
1. The most correct way: TARGET_RUNTIME_DLLS (CMake >= 3.21)
install(FILES $<TARGET_RUNTIME_DLLS:your_exe_here> TYPE BIN)
For this to work, your dependencies' CMake modules have to be well-written. In other words, they use CMake 3 targets with all their target properties set up correctly. If they set up everything right, all transitively-linked DLLs will be automagically gathered up and installed alongside your exe.
A big difference between this and the straight-up copy-the-files approach is that, because it goes through install(), CMake will actually know about them as files to-be-installed. CPack will know about your DLLs and include them in any installer you generate with it. CMake will automatically adjust what type of DLL (release vs debug) to match your target exe.
This is where CMake will be headed more in the future, and the way you should prefer if you have a choice.
2. The second most correct way: RUNTIME_DEPENDENCIES (CMake >= 3.21)
install(TARGETS your_exe_here
RUNTIME ARCHIVE LIBRARY RUNTIME FRAMEWORK BUNDLE PUBLIC_HEADER RESOURCE)
install(TARGETS your_exe_here
COMPONENT your_exe_here
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
DIRECTORIES $<TARGET_FILE_DIR:your_exe_here>)
The key here is RUNTIME_DEPENDENCIES.
Internally, RUNTIME_DEPENDENCIES calls file(GET_RUNTIME_DEPENDENCIES), which scans your executable binary, tries very hard to exactly replicate what actual dependency resolution would look like, and write down all the DLLs mentioned along the way. These are passed back up to install().
What this means is that this doesn't depend on your dependencies' CMake modules having their target properties set up correctly. Your actual executable binary is scanned. Everything will get picked up.
3. The third most correct way: install(DIRECTORY)
install(
DIRECTORY "${DIR_CONTAINING_YOUR_DLLS}"
TYPE BIN
FILES_MATCHING REGEX "[^\\\\/.]\\.[dD][lL][lL]$"
)
To use, put the DLLs appropriate for your build in $DIR_CONTAINING_YOUR_DLLS.
The trick here is that, unlike install(FILES), install(DIRECTORY) doesn't care what specific files are in the directory until install time. That means now we have all of configure time and compile time to get a list of your DLLs and stuff them in $DIR_CONTAINING_YOUR_DLLS. As long as the DLL files are in $DIR_CONTAINING_YOUR_DLLS by install time, install(DIRECTORY) will pick them up.
If you choose this method, it's becomes your responsibility to match DLLs to your build config. (Consider: static vs dynamic, debug vs release, import lib version vs DLL version, libs with optional multithreading, forgetting to remove DLLs you don't need anymore.)
If you choose this method, you might want to automate DLL finding and matching using something like what vcpkg's applocal.ps1 does.
Hint for vcpkg
If you use vpckg with VCPKG_APPLOCAL_DEPS enabled, vcpkg will locate and copy your DLLs into your $CMAKE_RUNTIME_OUTPUT_DIRECTORY for you, but without going through install(). You need to use the install(DIRECTORY) trick to get CMake to pick them up.
(Internally, vcpkg uses dumpbin, llvm-objdump, and objdump to scan your executable binary to get these filenames.)
You probably need to add custom target and make it depend on one of your executable targets.
To copy file using above function use:
COMMAND ${CMAKE_PROGRAM} -E copy_if_different ${CMAKE_BINARY_DIR}/path/to/file.dll ${CMAKE_BINARY_DIR}/where/to/put/file.dll`
The following command from the currently top rated answer depends on the output being put in /libs/.
add_custom_command(TARGET MyTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PROJECT_SOURCE_DIR}/libs/$<CONFIGURATION>"
$<TARGET_FILE_DIR:MyTest>)
I use the below command to do the copy, which works for me everywhere. Note that I'm only copying the output dll here, not the entire directory. Also note that I'm hard coding a destination /bin/ directory, which is specific to this project. But I wanted to share the $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}> syntax, which I think is neat:
add_custom_command(TARGET ${CMAKE_PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>
${CMAKE_CURRENT_SOURCE_DIR}/bin/$<TARGET_FILE_NAME:${CMAKE_PROJECT_NAME}>)
I'm a CMake beginner, but still I wanted to shared my experience. In my case I needed a post-install copy so that all my binaries are in.
In the case of third-party binary that can be imported within CMake, the following works for me:
find_package( dependency REQUIRED )
if( MSVC )
# If done properly and if the dependency has a correct config file, IMPORTED_LOCATION_RELEASE should be defined
get_target_property( DEP_SHARED_LIB_PATH dependency IMPORTED_LOCATION_RELEASE )
# Create a bin directory in the install folder
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_INSTALL_PREFIX}/bin/)
# Copy the shared lib file
add_custom_command(TARGET BGS POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy ${DEP_SHARED_LIB_PATH} ${CMAKE_INSTALL_PREFIX}/bin/)
endif()
Obviously IMPORTED_LOCATION_RELEASE can have variants depending on how the shared library was built / installed. Could be IMPORTED_LOCATION_DEBUG.
Maybe there's a better way to get that property name, I don't know.

Forcing cmake to call external executable before reading source files [duplicate]

Example source of a binary I want to run before each build, once per add_executable:
#include <stdio.h>
int main(int argc, char *argv[]) {
for(int i=0; i<argc; ++i)
printf("argv[%d] = %s\n", i, argv[i]);
fclose(fopen("foo.hh", "a"));
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.5)
project(foo_proj)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
set(SOURCE_FILES main.cpp)
# ---- this line changes ----
add_executable(foo_proj ${SOURCE_FILES})
Attempts:
add_custom_target(create_foo_hh COMMAND /tmp/bin/create_foo_hh)
add_dependencies(${SOURCE_FILES} create_foo_hh)
Error:Cannot add target-level dependencies to non-existent target "main.cpp".
The add_dependencies works for top-level logical targets created by the add_executable, add_library, or add_custom_target commands. If you want to add file-level dependencies see the DEPENDS option of the add_custom_target and add_custom_command commands.
execute_process(COMMAND /tmp/bin/create_foo_hh main.cpp)
No error, but foo.hh isn't created.
How do I automate the running of this command?
execute_process() is invoked at configuration time.
You can use add_custom_command():
add_custom_command(
OUTPUT foo.hh
COMMAND /tmp/bin/create_foo_h main.cpp
DEPENDS ${SOURCE_FILES} /tmp/bin/create_foo_hh main.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
add_executable(foo_proj ${SOURCE_FILES} foo.hh)
That way, foo.hh is a dependency of foo_proj: and your command will be invoked when building foo_proj. It depends on ${SOURCE_FILES} and /tmp/bin/create_foo_hh main.cpp so that it is generated again if one of those files changes.
Regarding paths, add_custom_command() is configured to run in the current build directory to generate the file there, and include_directories() is used to add the build directory to the include dirs.
You probably don't want the custom target to depend on your source files (because they aren't targets themselves and are therefore never "run"), but on the target you create with them:
target_add_dependencies(foo_proj create_foo_hh)
I think that the cleanest is to add two new project() (targets) and then add the resulting file to your final executable. This is how cmake can build a valid dependency tree so when your source files change they get recompiled, the command run, as necessary to get everything up to date.
Build Executable
First, as you do in your example, I create an executable from some .cpp file:
(example extracted from the as2js project)
project(unicode-characters)
add_executable(${PROJECT_NAME}
unicode_characters.cpp
)
target_include_directories(${PROJECT_NAME}
PUBLIC
${ICU_INCLUDE_DIRS}
${SNAPDEV_INCLUDE_DIRS}
)
target_link_libraries(${PROJECT_NAME}
${ICU_LIBRARIES}
${ICU_I18N_LIBRARIES}
)
As we can see,m we can add specific include paths (-I) and library paths (-L). It is specific to that one target so you can have a set of paths that is different from the one used with your other executables.
Generate Additional File
Next, you create a custom command to run your executable like so:
project(unicode-character-types)
set(UNICODE_CHARACTER_TYPES_CI ${PROJECT_BINARY_DIR}/${PROJECT_NAME}.ci)
add_custom_command(
OUTPUT
${UNICODE_CHARACTER_TYPES_CI}
COMMAND
unicode-characters >${UNICODE_CHARACTER_TYPES_CI}
WORKING_DIRECTORY
${PROJECT_BINARY_DIR}
DEPENDS
unicode-characters
)
add_custom_target(${PROJECT_NAME}
DEPENDS
${UNICODE_CHARACTER_TYPES_CI}
)
Notice a couple of things:
I set a variable (UNICODE_CHARACTER_TYPES_CI) because I am going to reference that one file multiple times
a. Notice how I put the destination in the binary (cmake output folder) using the ${PROJECT_BINARY_DIR}/... prefix. This is best to avoid generating those files in your source tree (and possibly ending up adding that file to your source tracking system like svn or git).
b. An important aspect of the add_custom_command() is the DEPENDS section which includes the name of your special command, the one we defined in the previous step.
The add_custom_target() is what allows cmake to find your target and execute the corresponding command whenever one of the source files (a.k.a. dependency) changes; notice the DEPENDS definition.
Use the Output
Finally, here is the main project (a library in my case) that makes use of the file we generated in the step above.
Notice that I reference that file using the variable I defined in the previous step. That way, when I feel like changing that name, I can do it by simply editing that one variable.
project(as2js)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/version.h.in
${CMAKE_CURRENT_BINARY_DIR}/version.h
)
add_library(${PROJECT_NAME} SHARED
compiler/compiler.cpp
...
parser/parser_variable.cpp
${UNICODE_CHARACTER_TYPES_CI}
file/database.cpp
...
)
(Note: the ... represent a list of files, shorten for display here as these are not important, the link above will take you to the file with the complete list.)
By having the filename inside the list of files defined in the add_library() (or the add_executable() in your case), you create a dependency which will find your custom_target(), because of the filename defined in the OUTPUT section of the add_custom_command()¹.
¹ It is possible to defined multiple outputs for an add_custom_command(). For example, some of my generators output a .cpp and a .h. In that case, I simply define both files in the OUTPUT section.
Results
Important points about the final results with this solution:
the output files of your generator are saved in the binary output path instead of your current working directory
the Makefile generated by cmake includes all the necessary targets/dependencies which means changing any of the input files regenerate everything as expected (even if you just update a comment)
if the generator fails, the build fails as expected
the files are generated by the build step (make time) instead of the generation step (cmake time, like the execute_process() would do)

Copy out plain .o files with cmake

I'm trying to get cmake (on linux) to create some static object (.o) files and install them to an external directory. To that end I have a list, object_sources, containing the project path of the sources, and put this in the top level CMakeLists.txt:
set(local_objects "")
foreach(file ${object_sources})
get_filename_component(cur ${file} NAME_WE)
add_library("${cur}.o" OBJECT ${file})
list(APPEND local_objects "${cur}.o")
# To confirm these variables are what I think they are:
message("---> ${cur} | ${file}")
endforeach()
install(
# Also tried FILES, see below.
TARGETS ${local_objects}
CONFIGURATIONS Debug
# Also tried nothing instead of 'OBJECTS' (same outcome)
# and LIBRARY (cmake fails on error).
OBJECTS
DESTINATION "/some/external/dir"
)
With regard to using FILES, it cannot work because in fact a foo.o is never created, so there is no point in me trying to get the correct project path.
When I get to the actual make command in the cmake build directory, it spits out Built target foo.o, etc., during the process. However, as mentioned, no foo.o is ever created anywhere, and the install creates an objects-Debug directory in /some/external/dir containing directories for each target using its name (e.g., foo.o/) and inside that some more nested directories at the bottom of which is the object I want, with the wrong name: foo.cpp.o.
Cmake can structure its build tree however it wants and name its temporaries whatever it likes, but I would really like to be able to ask for a final product object foo.o (not foo.cpp.o) and have it installed that way.
Is that possible? I'm beginning to think this is all very contrary to the (implicit) purposes of cmake, and that I should keep these kinds of tasks separate in a shell script or makefile (it takes about three lines in either), and just use cmake for the normative libs and executables.
Or: I believe I could get this to work if I created each object as a static archive (lib.a) instead, but part of the issue here is that would require modifying the build scripts of something else, or else extracting them via a script (in which case I might as well do the whole thing in the script).
I'm not aware of a good way to do this in CMake, but maybe the least fussy would be to use an OBJECT library. This would allow you to get the object files for a target using generator expressions, more specifically the TARGET_OBJECTS expression. This is a minimal example with a single foo.c file containing an empty main:
cmake_minimum_required(VERSION 3.11)
project(foo C)
set(SOURCES foo.c)
add_library(foo_obj OBJECT ${SOURCES})
add_executable(foo $<TARGET_OBJECTS:foo_obj>)
set(OBJS $<TARGET_OBJECTS:foo_obj>)
message(STATUS "these are my objects: ${OBJS}") # generator expression printed here, not evaluated yet
add_custom_target(print_foo_objs
COMMAND ${CMAKE_COMMAND} -E echo $<TARGET_OBJECTS:foo_obj>)
set(OBJ_ROOT_DIR $<TARGET_PROPERTY:foo,BINARY_DIR>)
add_custom_target(copy_foo_objs
COMMAND ${CMAKE_COMMAND} -E make_directory ${OBJ_ROOT_DIR}/myobjects/
COMMAND ${CMAKE_COMMAND} -E copy_if_different $<TARGET_OBJECTS:foo_obj> ${OBJ_ROOT_DIR}/myobjects/
COMMAND_EXPAND_LISTS) # UPDATE
Please note that as a generator expression, you will not have access to it during configuration (hence, that message will contain the generator expression as it was textually declared), but only during the generation phase.
If you still need to rename the object files (i.e. from foo.c.o to foo.o) you can maybe call a custom external script to do that for you.
Moreover, there might be issues with cleanup, etc. which will force you to add more custom targets and/or commands (and associated dependencies) to deal with it. Another downside of this approach.
UPDATE: COMMAND_EXPAND_LISTS allows the expansion of lists in the generator expression and it was added in CMake 3.8 and later (see here).

CMake run custom command before build?

Example source of a binary I want to run before each build, once per add_executable:
#include <stdio.h>
int main(int argc, char *argv[]) {
for(int i=0; i<argc; ++i)
printf("argv[%d] = %s\n", i, argv[i]);
fclose(fopen("foo.hh", "a"));
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.5)
project(foo_proj)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
set(SOURCE_FILES main.cpp)
# ---- this line changes ----
add_executable(foo_proj ${SOURCE_FILES})
Attempts:
add_custom_target(create_foo_hh COMMAND /tmp/bin/create_foo_hh)
add_dependencies(${SOURCE_FILES} create_foo_hh)
Error:Cannot add target-level dependencies to non-existent target "main.cpp".
The add_dependencies works for top-level logical targets created by the add_executable, add_library, or add_custom_target commands. If you want to add file-level dependencies see the DEPENDS option of the add_custom_target and add_custom_command commands.
execute_process(COMMAND /tmp/bin/create_foo_hh main.cpp)
No error, but foo.hh isn't created.
How do I automate the running of this command?
execute_process() is invoked at configuration time.
You can use add_custom_command():
add_custom_command(
OUTPUT foo.hh
COMMAND /tmp/bin/create_foo_h main.cpp
DEPENDS ${SOURCE_FILES} /tmp/bin/create_foo_hh main.cpp
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
add_executable(foo_proj ${SOURCE_FILES} foo.hh)
That way, foo.hh is a dependency of foo_proj: and your command will be invoked when building foo_proj. It depends on ${SOURCE_FILES} and /tmp/bin/create_foo_hh main.cpp so that it is generated again if one of those files changes.
Regarding paths, add_custom_command() is configured to run in the current build directory to generate the file there, and include_directories() is used to add the build directory to the include dirs.
You probably don't want the custom target to depend on your source files (because they aren't targets themselves and are therefore never "run"), but on the target you create with them:
target_add_dependencies(foo_proj create_foo_hh)
I think that the cleanest is to add two new project() (targets) and then add the resulting file to your final executable. This is how cmake can build a valid dependency tree so when your source files change they get recompiled, the command run, as necessary to get everything up to date.
Build Executable
First, as you do in your example, I create an executable from some .cpp file:
(example extracted from the as2js project)
project(unicode-characters)
add_executable(${PROJECT_NAME}
unicode_characters.cpp
)
target_include_directories(${PROJECT_NAME}
PUBLIC
${ICU_INCLUDE_DIRS}
${SNAPDEV_INCLUDE_DIRS}
)
target_link_libraries(${PROJECT_NAME}
${ICU_LIBRARIES}
${ICU_I18N_LIBRARIES}
)
As we can see,m we can add specific include paths (-I) and library paths (-L). It is specific to that one target so you can have a set of paths that is different from the one used with your other executables.
Generate Additional File
Next, you create a custom command to run your executable like so:
project(unicode-character-types)
set(UNICODE_CHARACTER_TYPES_CI ${PROJECT_BINARY_DIR}/${PROJECT_NAME}.ci)
add_custom_command(
OUTPUT
${UNICODE_CHARACTER_TYPES_CI}
COMMAND
unicode-characters >${UNICODE_CHARACTER_TYPES_CI}
WORKING_DIRECTORY
${PROJECT_BINARY_DIR}
DEPENDS
unicode-characters
)
add_custom_target(${PROJECT_NAME}
DEPENDS
${UNICODE_CHARACTER_TYPES_CI}
)
Notice a couple of things:
I set a variable (UNICODE_CHARACTER_TYPES_CI) because I am going to reference that one file multiple times
a. Notice how I put the destination in the binary (cmake output folder) using the ${PROJECT_BINARY_DIR}/... prefix. This is best to avoid generating those files in your source tree (and possibly ending up adding that file to your source tracking system like svn or git).
b. An important aspect of the add_custom_command() is the DEPENDS section which includes the name of your special command, the one we defined in the previous step.
The add_custom_target() is what allows cmake to find your target and execute the corresponding command whenever one of the source files (a.k.a. dependency) changes; notice the DEPENDS definition.
Use the Output
Finally, here is the main project (a library in my case) that makes use of the file we generated in the step above.
Notice that I reference that file using the variable I defined in the previous step. That way, when I feel like changing that name, I can do it by simply editing that one variable.
project(as2js)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/version.h.in
${CMAKE_CURRENT_BINARY_DIR}/version.h
)
add_library(${PROJECT_NAME} SHARED
compiler/compiler.cpp
...
parser/parser_variable.cpp
${UNICODE_CHARACTER_TYPES_CI}
file/database.cpp
...
)
(Note: the ... represent a list of files, shorten for display here as these are not important, the link above will take you to the file with the complete list.)
By having the filename inside the list of files defined in the add_library() (or the add_executable() in your case), you create a dependency which will find your custom_target(), because of the filename defined in the OUTPUT section of the add_custom_command()¹.
¹ It is possible to defined multiple outputs for an add_custom_command(). For example, some of my generators output a .cpp and a .h. In that case, I simply define both files in the OUTPUT section.
Results
Important points about the final results with this solution:
the output files of your generator are saved in the binary output path instead of your current working directory
the Makefile generated by cmake includes all the necessary targets/dependencies which means changing any of the input files regenerate everything as expected (even if you just update a comment)
if the generator fails, the build fails as expected
the files are generated by the build step (make time) instead of the generation step (cmake time, like the execute_process() would do)