I'm trying to write a CMakeLists.txt which copies as a post build event required dlls from external libraries to the folder in which the executable is located after building. The OS I'm working on is Win7 and VS2010.
There a quite a lot of external libraries, so I do not want to list them individually within the CMakeLists.txt. My current solution is to use file globbing and create a post build event for each library:
FILE(GLOB files "${LIBRARY_DIR}/lib/$(ConfigurationName)/*dll")
MESSAGE("FILE LIST IS: ${files}")
FOREACH(file ${files})
ADD_CUSTOM_COMMAND(
TARGET mylib
POST_BUILD
COMMENT "Copying dll ${file}..."
COMMAND ${CMAKE_COMMAND} -E copy ${file} "${CMAKE_BINARY_DIR}/$(ConfigurationName)/"
)
ENDFOREACH()
Basically, the code snipped above works file if I replace $(ConfigurationName) with Release or Debug. However, I'd like to take the libraries from the corresponding directory dependent on Release or Debug build mode. In the code snipped above, $(ConfigurationName) does not get substituted by the visual studio build mode. I guess this is due to that this is a VS2010 variable which isn't known at cmake generation time).
Does somebody have a smart idea how to resolve this issue?
ADD_CUSTOM_COMMAND understands generator expressions, so you would be able to use $<CONFIG> rather than $(Configuration) or $(ConfigurationName) there. For the FILE command, however, this won't work. More importantly though, the file globbing approach isn't going to work for the first time you build your project if those DLL's are built as part of the same project, since none of the DLL's would be there when you first run CMake in such an arrangement.
You may want to look into BundleUtilities as a more robust way to solve your problem. Despite its name, it supports all platforms, not just Mac. You would need to invoke the fixup_bundle command as a post-build or install step to achieve what you describe. There are a few variations of this approach already described online (e.g. here and here).
You can also have a look at the source for the DeployQt4.cmake file which should be included in the Modules subdirectory of your CMake installation for ideas.
Lets suppose you have _TARGET_NAME filled with your your project name, and ADDITIONAL_DLL_PATH with a list of folders containing external dlls, Then the following code will copy any external dependencies found using dumpbin on windows.
foreach(CONF_TYPE ${CMAKE_CONFIGURATION_TYPES})
string(TOUPPER ${CONF_TYPE} CONF_TYPE_U)
get_property(TARGET_LOCATION_${CONF_TYPE_U} TARGET ${_TARGET_NAME} PROPERTY LOCATION_${CONF_TYPE_U})
get_filename_component(TARGE_DIR ${TARGET_LOCATION_${CONF_TYPE_U}} PATH)
install(CODE "
if (\"\${BUILD_TYPE}\" STREQUAL \"${CONF_TYPE}\")
include(BundleUtilities)
fixup_bundle(\"${TARGET_LOCATION_${CONF_TYPE_U}}\" \"\" \"${ADDITIONAL_DLL_PATH}\")
endif()"
COMPONENT Runtime)
endforeach()
There are downsides to this solution though, Like the need to build INSTALL target whenever you need to copy those files (which might not be many times), and not detecting delay loaded dlls.
Related
I have a subproject in my project for generating code used in the project, however i want to include it using ExternalProject so it can be built and ran regardless of the toolchain i use for the main project.
It mostly works except i can't figure out how to use it in add_custom_command, since i want it to use the target rule specified in the docs:
If COMMAND specifies an executable target name (created by the add_executable() command), it will automatically be replaced by the location of the executable created at build time [...]
Simple example of my project setup that replicates the issue:
https://godbolt.org/z/of8G4c4Gf
Replacing the ExternalProject_Add with a simple add_subdirectory makes it work, but wouldn't do what i want in the case of using a different toolchain.
ExternalProject_Add will add the codegen target. However, CMake has no idea what that target is doing and what output it will provide, as that info is now hidden away in the external CMake run. So the outer CMake run has no idea about the codegen binary produced by that step and where it will be located. You need to provide the full path to the executable to add_custom_command manually.
ExternalProject_Add will build codegen and place the resulting binary in a subdirectory inside your build directory. The exact location is toolchain dependent, so it can be tricky to reconstruct the full target path. The most robust way to solve this is:
Add an install step to the codegen project. This will allow you to portably specify which path the installed binary will end up in relative to the install root directory.
Set the INSTALL_DIR option on ExternalProject_Add to a subdirectory inside your PROJECT_BINARY_DIR.
Hardcode the full path to the installed codegen binary in the custom command of the outer CMake project. To work on multiple platforms, you may need to make use of the CMAKE_EXECUTABLE_SUFFIX variable.
How can I use FetchContent in a way that source files are not duplicated, but binaries are still compiled once per build configuration (Debug, Release, etc.)?
When using cmake's FetchContent with minimal options, e.g.
FetchContent_Declare(
<name>
GIT_REPOSITORY <git_url>
GIT_TAG <git_tag>
)
FetchContent_MakeAvailable(<name>)
a directory called _deps is created in the build directory, with three subdirs: <name>-src, <name>-build and <name>-subbuild. So both the source and binary files are written into a subdir of the build directory.
However, if several build directories exist, e.g. for Debug and Release builds, this is carried out for each, leading to unnecessary source file duplication. Also, all other libraries in my project are in <source_root>/lib, and the method of retrieval doesn't seem to be a good reason to split them up.
So I added
set(FETCHCONTENT_BASE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/fetchContent)
inside the CMakeLists.txt in <source_root>/lib.
But now the binaries are only built once, with whichever flags happen to be used at that moment. So if I build everything in Debug mode first, and Release mode afterwards, I get a Release executable with libs that were built with Debug flags.
It seems like I'm missing something obvious here, since my case is hardly special. Maybe FetchContent_Populate is the way to go? But the cmake doc advises against using it. Please advise.
I am working on a C++ application in Visual Studio, using the Visual Studio's CMake project template.
To build my application I only need the header and the external library and I can link it like this:
# CMakeList.txt : CMake project for myapp, include source and define project specific logic here.
cmake_minimum_required (VERSION 3.8)
# Add source to this project's executable.
add_executable (myapp "myapp.cpp" "myapp.h")
# Add TIFF library
set(TIFF_INCLUDE_DIR "C:\\libs\\tiff\\out\\install\\x64-Debug\\include")
set(TIFF_LIBRARY "C:\\libs\\tiff\\out\\install\\x64-Debug\\lib\\tiffd.lib")
find_package(TIFF REQUIRED)
target_link_libraries(myapp PRIVATE TIFF::TIFF)
So far, so good. I can use the tiff library to open, read, write tiff files, etc., and IntelliSense is catching up with declarations in the header files. But, I would like IntelliSense to also be aware of the full source code of the TIFF library. For example, if I am in myapp.cpp and I ctrl+click on TIFFOpen it opens the header corresponding to TIFFOpen, but when I ctrl+click TIFFOpen in the header file, it doesn't go to the corresponding source file, which is the normal behaviour for source files in myapp. This is understandable, since I never told Visual Studio where to find the source files of the external library.
CMake doesn't need to know where the source files of the external libraries are, since it won't build the external library, therefore I guess I don't/shouldn't change anything in CMakeLists.txt
One option (I haven't tried yet, but I'm fairly sure it would work), would be to just include the entire tiff library as a sub-project of myapp. I do have some problems with this solution though:
The external library is not conceptually an integral part of the project, and I don't plan to modify the external library. This is more of a principle issue.
Simply having it as a subfolder in my project makes it a risk of changing something I didn't intend to change.
I don't want to rebuild the external library when I do a rebuild all. I know Visual Studio / CMake is smart enough to figure out that nothing changed and doesn't rebuild, but I would rather have Visual Studio / CMake not even try.
The way I see it, I have to set the directory with the source files somewhere in Visual Studio settings, but still related to the project. My best guess is that the .vs/ProjectSettings.json is the file I need to edit somehow, but honestly, I have no clue.
Alternatively, maybe I could write some command in CMakeLists.txt that doesn't do anything, but triggers the IntelliSense to look in the folder with source files. Again, I have no clue how should I go about this.
In a nutshell, I want IntelliSense to see all source files of an external library, the same way it sees the source files of myapp, without including all source files of the external library as a sub-project of myapp. How should I go about it, if even possible?
If relevant, I use Visual Studio 2019 Community and the CMake it comes with it (3.15).
Regarding your last comment, writing code in comment section is inconvenient so I'll just post it here although not an answer.
libtiff-4.pc is for pkg-config, not cmake, and find_package() can't deal with it directly on Windows and would take some work if you really want to. It might be easier to just write everything manually. Remember to set the tiff.lib and tiffd.lib according to your configuration. You can use CMAKE_BUILD_TYPE variable and if() command such as:
# set build type to release if not specified
if(NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "")
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build Type" FORCE)
endif()
# switch lib to link according to build type
if(CMAKE_BUILD_TYPE STREQUAL "Release")
# for release
set(TIFF_LIBRARY "<your_path_to_installed_lib>/tiff.lib")
else()
# for debug. If you have other configs like relwithdebinfo you can add more
set(TIFF_LIBRARY "<your_path_to_installed_lib>/tiffd.lib")
endif()
Also, remove find_package() and use target_link_libraries() and target_inlude_directories():
set(TIFF_INCLUDE_DIR "<your_path_to_installed_headers>/include")
target_link_libraries(myapp PRIVATE ${TIFF_LIBRARY})
target_include_directories(myapp PRIVATE ${TIFF_INCLUDE_DIR})
You can also skip setting TIFF_LIBRARY and TIFF_INCLUDE_DIR and pass the string directly if you like.
I use Visual Studio a lot and it's my favorite IDE. But package management with cmake on Windows is not as smooth as Linux. Always remember to set environment variables after compiling and installing external libraries.
Normally find_package() will look for a system environment variable named <libname>_DIR (for example TIFF_DIR, which is not found in your case), which is used to store path to installed lib, then look for <libname>Config.cmake and <libname>ConfigVersion.cmake in that folder (and would fail for TIFF since it doesn't have them).
It also searches other places, check https://cmake.org/cmake/help/v3.15/command/find_package.html?highlight=find_package for details.
So for those libs with cmake exported files, add a variable with correct name and value. And that's only for compiling.
If you want your application to run after compiling, you also need to add the path of installed lib's binaries (usually *.dll) to system's Path variable. In your case you should find something like tiff.dll and tiffd.dll after compiling and installation, add that folder to Path and you are good to go.
Actually, showing the source in the IDE is easy enough.
Just add the source with add_library(tiff_for_ide EXCLUDE_FROM_ALL ${SOURCES}) but don't link with it in the main program. You'll want to use a different target name for the library. For this you'd need the source in your project directly or as a submodule (if using Git, something else if available by you VCS).
Other options are using ExternalModule_Add; or FetchContent_MakeAvailable with the add_library as above, to avoid adding third party deps into the repository directly.
Now, just cross your fingers and hope that the IDE is not intelligent enough to restrict searching for sources that are linked against the target which compiles the current file. Or that it's intelligent enough to detect this situation but fallback to searching the whole project files anyway when it's linking against a binary.
I have the following situation:
I'm working on an application that depends on a number of third party libs, among them wxwidgets
I build the application for multiple target configurations (x86, arm, Linux, Windows) using Linux as my build host system
Due to the above mentioned multiple target configurations, I have chosen to build those third-party libs from source, using CMake's ExternalProject_Add function.
The third-party libs are built 'on-demand' at a location separate from my application's CMAKE_BINARY_DIR so that I can wipe the build tree for my application without having to rebuild the third-party libs (takes a looooong time).
The location of the third-party libs is different depending on what target configuration I build them for (obviously).
I'm quite new to CMake and the problem I currently face is this:
The source files in my application can't find the wx include files and I need to set the correct linker flags to be able to link my application against wxwidgets.
This seems to be handled by a utility 'wx-config' that provides exactly that info as output when run with either the --cppflags or --libs flag. I can not however, figure out how to catch that output and append it to the include dirs and linked libraries I setup from my CMakeLists.txt files.
So basically what I want is.
Build wxwidgets (if it doesn't exist) for the current target configuration
Run wx-config --cppflags and --libs to find out the correct include dirs and linker flags for the current target configuration
Use the info from step 2 when building targets that are my own application
So far I've tried something like this:
# Set a target-configuration-specific location
set(wxwidgetsTop ${MYPROJECT_EXTERNAL_DIR}/wxwidgets/wxwidgets_${MYPROJECT_CURRENT_TARGET_CFG})
# Build the project
ExternalProject_Add( wxWidgetsExternal
PREFIX ${wxwidgetsTop}
URL ${MYPROJECT_EXTERNAL_DIR}/tarballs/wxWidgets-3.0.2.tar.bz2
SOURCE_DIR ${wxwidgetsTop}/src/wxwidgets
CONFIGURE_COMMAND ${configure_cmdline}
BUILD_COMMAND make -j${MYPROJECT_NCPU}
INSTALL_COMMAND make install
)
# Create a wxwidgets target to be used as a dependency from other code
add_library(wxWidgets IMPORTED STATIC GLOBAL)
add_dependencies(wxWidgets wxWidgetsExternal)
# (non-working) attempt to get the correct include dirs and linker
# flags for wxwidgets
add_custom_command(TARGET wxWidgetsExternal
POST_BUILD
COMMAND ${INSTALL_DIR}/bin/wx-config ARGS --cppflags
COMMENT "Running wx-config"
)
but the above does not provide a way to actually use the result from the custom command to append the cppflags and linker options when building the targets that make up my application.
What is a good way to achieve what I want?
I see three different ways of doing this:
Method 1: use find_package
Use wxWidgets as a standalone requirement for your project, and expect the devs to install it before building your project. In your CMakeLists.txt you will need to call find_package(wxWidgets), like this:
find_package(wxWidgets COMPONENTS net gl core base)
if(wxWidgets_FOUND)
include(${wxWidgets_USE_FILE})
# and for each of your dependent executable/library targets:
target_link_libraries(<YourTarget> ${wxWidgets_LIBRARIES})
endif()
This has the advantage of not rebuilding the lib if you rebuild your project, however it requires some work for your user (they need to handle the installation of wxWidgets by hand) and for you (you need to setup include paths / compile definitions / ... by hand).
Method 2: embed wxWidgets
The second option is to bundle wxWidgets in your repo (svn external or git submodule) and usually (re)write the CMakeLists.txt of this lib to be target-oriented. Then, in your top-most CMakeLists.txt, you can do the following:
# for example, if you just need core and net:
target_link_librairies(my_app PUBLIC wxWidgetsCore wxWidgetsNet)
# No need to manually setup include dirs, etc...
To make a CMakeLists.txt target-oriented, you define include directories and other compilation properties for a target, not a directory. Example:
# When defining wxWidgetsCore, for example
add_library(wxWidgetsCore ...)
target_include_directories(wxWidgetsCore PUBLIC someDir)
target_compile_definitions(wxWidgetsCore PUBLIC -pedantic)
target_link_libraries(wxWidgetsCore PUBLIC someLib)
The drawback of this approach is that rebuilding your project will trigger a rebuild of wxWidgets. However, it is possible to trick this by not using "rebuild" but "clean just my app, then build". Here is some insight on how to achieve this.
Method 3: some sort of hybrid
The big drawback of method 2 leads to the third approach: don't put wxWidgets in your project, but create a CMakeLists.txt that will "import" the lib. The idea: you ask your user for the directory where wxWidgets is installed, then this script will setup everything for your project. First, put the CMakeLists.txt here:
/your-project-root
/thirdparty
/wxWidgets
CMakeLists.txt
/dir-where-wxwidgets-is-installed
...
Now, you define an imported target:
# When defining wxWidgetsCore, for example
set(WX_INCLUDE_DIR ${USER_SPECIFIED_WX_ROOT}/include)
add_library(wxWidgetsCore IMPORTED GLOBAL)
set_property(TARGET wxWidgetsCore APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES ${WX_INCLUDE_DIR})
See INTERFACE_INCLUDE_DIRECTORIES and INTERFACE_LINK_LIBRARIES. You need your user to have build wxWidgets somewhere in his system, but from your point of view you just do target_link_libraries(your_app PUBLIC wxWidgets...), as in method 2. The advantage is that this approach is interchangeable with method 2 transparently, and you don't put the whole dependency in your project.
Setting cppflags and linker flags has to be done at CMake time, but you are trying to run wx-config at build time and you are not capturing its output anyway, so your add_custom_command() isn't doing anything useful other than printing things to the build tool's output.
Ideally, you would use the FindwxWidgets module CMake already provides. It requires wxWidgets to already be built (but see further below). Have a look at the CMake documentation for it and see if that at least sounds like what you are trying to achieve manually by using wx-config. If you can get FindwxWidgets to do the job for you, that would be a much cleaner approach.
Getting something to build at configure time so you can use it later on in your CMakeLists.txt file is a bit more tricky. ExternalProject_Add() downloads and builds things at build time, but you need wxWidgets to be built earlier at configure time. I wrote an article recently for how to do at least the downloading part at configure time and you should be able to adapt it to do the whole build at configure time instead. The article uses Google Test as its example and can be found here:
https://crascit.com/2015/07/25/cmake-gtest/
It would be trivial to make it put the wxWidgets build wherever you like, not just in the CMAKE_BINARY_DIR area. That would allow you to have different wxWidgets builds for each build configuration and to be able to wipe out your application's build tree independently of the wxWidgets builds.
Hope that points you in the right direction.
The solution I use checks for wxWidgets installation in the system using find_package, if it's not found, then the script downloads wxWidgets from github and links the program against downloaded library. The lib is installed in the build directory, so only the first build is slow - subsequent builds do not even check wxWidgets sources timestamps, so the process is as fast as building using preinstalled wxWidgets library.
Here's how my script does it:
It quietly checks for wxWidgets installation using find_package(wxWidgets QUIET),
If it's found, the script adds a dummy library wxWidgets_external,
If it's not, then it creates an ExternalProject named wxWidgets_external which downloads, builds and installs the library in the build dir, setting wxWidgets_ROOT_DIR to point to the wxWidgets installation dir,
Then we add another ExternalProject pointing to a folder with the main program's source files and CMakeLists.txt build script. This external projects depends on wxWidgets_external which is either a dummy library in case wxWidgets is preinstalled in the system, or an external project set up to download the library from github,
In the aforementioned CMakeLists.txt we again call find_package, this time with REQUIRED parameter and use the library the standard way (https://docs.wxwidgets.org/trunk/overview_cmake.html). Because we set up the dependencies and variables correctly, this call will use either preinstalled wxWidgets (if it's available) or the one downloaded from github.
There are more quirks to it, but that's the gist of it. The full sample code (tested on Linux, Windows and Mac) is available on github (https://github.com/lszl84/wx_cmake_template).
Also see full blog post which explains this in more detail: https://justdevtutorials.medium.com/wxwidgets-cmake-multiplatform-superbuild-4ea86c4e6eda
Question: Can CMake generate build scripts that do not, in any way, use CMake? If not, how hard is it to gut a CMake generated automake script to not make any checks against CMake?
I am a big fan of CMake to the point where I am championing the idea that we transition to it in my current work environment. One thing that could ease this transition from our current build system to CMake would be if I could demonstrate that CMake can generate automake files that do not require cmake themselves.
Clearly, I would never want to do this for day to day use, but having the ability to easily create a branch of our code that can be built from source without requiring cmake would go a long way in helping me make my case.
The ability to do this depends on your OS, I'm presuming Unix/Makefile or Windows/MSVC. If you're using MSVC, the cmake dependency should be eliminated by declaring the CMAKE_SUPPRESS_REGENERATION option at the start of your cmake script.
SET(CMAKE_SUPPRESS_REGENERATION TRUE)
On Unix-based systems, however, the Makefiles are tied explicitly to the cmake build files (CMakeFiles, etc). I suspect that this dependency could be bypassed by the strategic commenting out of Makefile directives although, I cannot say what they might be.
No, CMake cannot do this. It doesn't really make sense, either, since without any CMake-support at build-time, there would be no way to check or update the makefiles/project-files themselves when the CMakeLists.txt files have changed.
If you are moving from Visual Studio to CMake, you may want to take a look at vcproj2cmake.
CMake generated files depend on cmake for various commands such as create / remove / etc... not just to regenerate the makefiles on a change so removing cmake is not going to work.
As someone who has taken a large complex piece of software and recently pulled out its existing build system, installing a new build system in its place. I can tell you that it's not easy, but I would definitely not want shell scripts as part of my build process, if they can be avoided. More and more systems will find themselves with CMake on them anyway, as more big name software packages like LLVM and KDE start using it—This is an area where it really accels, large projects.
One of the nice things about CMake is it builds things quicker. Resorting to have to fork shell instances to interpret a script really slows down the build process.
What about the 'atomic solution' ?
EX- auto-generate a "QT moc" file from CMakeLists.txt, then build project that depends on the .cpp file being generated
# inside project level CMakeLists.txt
# run shell script to create the "moc_GUICreator.cpp" auto-generated source file
if(UNIX)
execute_process(COMMAND "sh" ${CMAKE_CURRENT_SOURCE_DIR}/scripts/generate_moc.sh WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scripts )
endif(UNIX)
Where the .sh file contains:
# generate_moc.sh
echo "generating moc file: moc ../include/GUICreator.h -o ../src/moc_GUICreator.cpp "
moc ../include/GUICreator.h -o ../src/moc_GUICreator.cpp
Equivalent windows batch file, "moc_creator_win.bat":
moc "GUICreator.h" -o "moc_GUICreator.cpp"
Haven't tried this last bit in windows, but it or something very close should work, just after the if(UNIX) block in CMakeLists.txt:
if(WIN32)
execute_process(COMMAND "cmd" ${CMAKE_CURRENT_SOURCE_DIR}/scripts/moc_creator_win.bat WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scripts )
endif(WIN32)
So, basically if you're clever you can do whatever you want from a script and use CMake variables as args to it, I'm not sure if you can ask for more...
the point is to avoid 'non-portable build types' unless you really need to hack it into a specialized compiler, or don't feel like using QT Designer to place the widgets ;-)