short -- Is it possible to build a external binary/library out of a project with CMake, when the binary/library only has a makefile given?
So you have your own project, a bunch of CMakeLists.txt in your src-tree and this external library with its source-files. Your sources depend on this library and some binaries/libraries want to link against it. How would one compile this external library if it has only a makefile or Visual Studio project file and no given CMakeLists.txt?
Is there a chance to call configure/make out of CMake? Or run an batch-compile with VS under Windows? Or anything else?
Thanks for your help with this one...
It sounds like you want CMake's external project. I have worked with it quite extensively when developing the Titan build system, and it provides a way of managing multiple out of source builds. You can include ExternalProject, and then something like the following would build the project:
ExternalProject_Add(Qt
DOWNLOAD_DIR ${CMAKE_CURRENT_BINARY_DIR}
URL ${qt_file}
UPDATE_COMMAND ""
SOURCE_DIR ${qt_source}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ${qt_configure}
BUILD_COMMAND ${qt_build}
INSTALL_COMMAND "${qt_install}"
)
There is an article about external projects in the October 2009 issue of the source too. Using external project you can call any make commands available on the host system, we build Qt using their supplied configure command on Windows, Mac and Linux.
Related
I have a program that depends on an external library (SDL for example). I want CMake to take care of that dependency for me, so I was looking into FetchContent. As far as I understand, this module simply downloads the source code so that information on the external library is available at configure time. For example:
include(FetchContent)
FetchContent_Declare(sdl
GIT_REPOSITORY <...>
)
FetchContent_GetProperties(sdl)
# sdl_POPULATED, sdl_SOURCE_DIR and sdl_BINARY_DIR are ready now
if(NOT sdl_POPULATED)
FetchContent_Populate(sdl)
endif()
At some point, however, I want to build that source code and link it to my main executable. How to do it the "modern CMake way"?
The recommended way to build external libraries from source as part of your build depends on what the external lib provides build-wise.
External lib builds with cmake
If the external lib builds with cmake then you could add the lib to your build via a add_subdirectory(${libname_SOURCE_DIR}) call. That way cmake will build the external lib as a subfolder ("subproject"). The CMakeLists.txt file of the external lib will have some add_library(ext_lib_name ...) statements in it. In order to then use the external lib in your targets (an application or library that depends on the external lib) you can simply call target_link_libraries(your_application <PRIVATE|PUBLIC|INTERFACE> ext_lib_name) https://cmake.org/cmake/help/latest/command/target_link_libraries.html
I had a quick look at this github repo https://github.com/rantoniello/sdl - (let me know if you are referring to another library) and it actually looks like it is building with cmake and that it allows clients to statically or dynamically link against it: https://github.com/rantoniello/sdl/blob/master/CMakeLists.txt#L1688-L1740
So, ideally your applicaiton should be able to do
add_executable(myapp ...)
target_link_libraries(myapp PRIVATE SDL2-static) // Statically link againt SDL2
Due to their CMakeLists.txt file the SDL2-static comes with properties (include directories, linker flags/commands) that will automatically propergate to myapp.
External lib does not build with cmake
If a external lib doesn't build with cmake then one can try to use add_custom_target https://cmake.org/cmake/help/latest/command/add_custom_target.html to build the library. Something along the lines of:
add_custom_target(myExternalTarget COMMAND <invoke the repo's build system>)
You'd then need to set the target properties that are important for clients yourself via the the proper cmake functions set_target_properties, target_include_directories ... A great writeup how to get started with these kinds of things: https://pabloariasal.github.io/2018/02/19/its-time-to-do-cmake-right/
I'm trying to use this external project in a CMake project of mine; the external project is also a CMake project.
Now, the external project produces static libraries and a bunch of headers; but - I don't want those to be installed anywhere, I just need them to build stuff as part of my main project (and the files are not necessary after build and/or install of the main project).
How do I get CMake to "fetch, build, but not install" an external project? I was thinking I might hackishly do this by forcing an empty install command, but I'm sure there's a better, more elegant way.
As mentioned by #Tsyvarev, to skip the install of an external project, you can simply pass the argument INSTALL_COMMAND "", this is definitively the recommended approach.
For example:
ExternalProject_Add(Foo
GIT_REPOSITORY "git://github.com/Foo/Foo"
GIT_TAG "123456"
SOURCE_DIR ${CMAKE_BINARY_DIR}/Foo
BINARY_DIR ${CMAKE_BINARY_DIR}/Foo-build
CMAKE_CACHE_ARGS
-DFOO_ENABLE_BAR:BOOL=1
INSTALL_COMMAND ""
)
Then you would configured the dependent external project with -DFoo_DIR:PATH=${CMAKE_BINARY_DIR}/Foo so that find_package(Foo REQUIRED) works as excepted. For this to work, the assumption is that the external project provide a working config file for the build tree.
To learn more about config files, see Correct way to use third-party libraries in cmake project
I'm pretty new to cmake, but I am stuck with an odd behaviour that I'd like to understand and overcome.
I'm creating an executable that uses the Paho MQTT C library. The content of this executable is irrelevant, let's say it just connects to a broker for now, nothing fancy.
To compile my program, I use cmake with a CMakeLists.txt file, which does a few things:
use an ExternalProject block to pull the MQTT C lib from Github
declare the executables and the linked libs
Here it is :
project(myproject)
include(ExternalProject)
########## CMAKE VERSION AND FLAGS ##########
cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/bin)
########## MQTT EXTERNAL LIB ##########
set(MQTT_BUILD_DIR ${CMAKE_CURRENT_BINARY_DIR}/mqtt)
ExternalProject_Add(
project_mqtt
GIT_REPOSITORY https://github.com/eclipse/paho.mqtt.c
GIT_TAG v1.2.0
PREFIX ${MQTT_BUILD_DIR}
CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"
)
add_library(mqtt STATIC IMPORTED)
set(MQTT_LIB ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/lib/libpaho-mqtt3a.1.2.0.dylib)
set_property(TARGET mqtt PROPERTY IMPORTED_LOCATION ${MQTT_LIB})
add_dependencies(mqtt project_mqtt)
include_directories(${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/include)
link_directories(${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/lib)
########## EXECUTABLE ##########
add_executable(myproject ../src/main.cpp)
target_link_libraries(myproject mqtt)
Note that in the file I especially compile for Mac and I hardcoded the .dylib suffix but this is not a problem here
My folder structure is as follows :
myproject/
-- CMakeLists.txt
-- bin/
-- build/
-- src/
- main.cpp
My compilation workflow is very standard : cd build && cmake ../ && make.
My sources are in /src, I build everything in /build and my executable is written in /bin. The *.dylib are explicitely built locally in the project folder (in bin/lib) since I don't want to install them in the system (I want my project to be self-contained in a way).
It works well (I mean, it compiles).
But when I run the program, I get a dyld: Library not loaded: libpaho-mqtt3a.1.dylib error.
.. which is quite "normal", because my lib has been installed in bin/lib and my executable searches for them in its own folder (which is bin/). So for instance if I run : cd bin/lib && ../myproject it works perfectly well.
So, now here are my questions :
[Philosophically,] Is it a good way of doing this ? I want to use this external lib but I don't want to include the Paho lib sources in my code obviously, so I though ExternalProject was a good way to go. It compiles the lib at compile time before compiling my project and I find the process quite handy. I looked for a "package manager" for C++ but there does not seem to be a strong consensus (I'm looking at something like *composer* or *npm* for web apps), so I did not go this route.
Why, oh why, does my executable look for the library file in its own directory ./ ? Can I change that ? Is it some cmake configuration that I missed ? I guess that I could just use a cmake custom command to copy the library file into the /bin folder, but this looks like a hack to me. Isn't it ?
To be honest, I'd like to have no "path dependencies" in my executable, it's to say that the executable, when compiled on a specific system, could be put anywhere on this system and work flawlessly afterwards. I understand that means that the compiled Paho MQTT lib should be statically in my executable, but I wonder how to do that (or even what that really means, in fact). Is it possible ?
I may miss a whole part of how the C/C++ compile and external lib workflow works but I think I'm close to something quite robust, it's just this lib path thing at runtime that I would like to properly understand and fix.
I'm really open to suggestions on this
Thanks a lot !
I want to ask if it is possible to work with c++ source code of the opencv library, without using the compiled libraries like .lib and .dll.
I mean is it possible to work only with header files and .cpp files from the library?
I want to create a basic c++ class with a function that accepts an image and perform a series of opencv related operations, but I need the freedom to compile the code for a number of platforms as a native library (and I think that using .lib or .dll will forbid the compilation for several platforms). I guess that could be possible by using only the c++ source code. But using visual studio and adding Additional Include Directories, I do get errors LNK2019, that is probably "A function or variable is declared but not defined" although I do include the 'right' directories with .hpp's and .cpp's but probably I miss something.
This is a typical case for native code. What you need to do is download the source code (I believe you have already done that) and build it separately for platform(s) you need it for.
In your project include the result libraries (either static lib or dynamic dll files) as you would pre-compiled ones. This way library files are kept separate and built only once, not on every build of your project.
In Visual Studio you can set the include path separately for every configuration so you can set your project up to use library built for current architecture.
Adding to what #slawekwin wrote in his comment above, building and
linking dependencies on your target platforms can also be automated as
part of your build system.
If you built OpenCV before, you should have come across CMake. With
CMake it's possible to configure external projects (EP), which will
download source code of third party project and then configure, build,
and install it. This is an example for OpenCV copied from a github
repository:
ExternalProject_Add(opencv
GIT_REPOSITORY ${git_protocol}://code.opencv.org/opencv.git
GIT_TAG 2.3.1
SOURCE_DIR opencv
BINARY_DIR opencv-build
UPDATE_COMMAND ""
PATCH_COMMAND ""
CMAKE_GENERATOR ${gen}
CMAKE_ARGS
${ep_common_args}
-DBUILD_DOCS:BOOL=OFF
-DBUILD_EXAMPLES:BOOL=OFF
-DBUILD_NEW_PYTHON_SUPPORT:BOOL=OFF
-DBUILD_PACKAGE:BOOL=OFF
-DBUILD_SHARED_LIBS:BOOL=ON
-DBUILD_TESTS:BOOL=OFF
# -DCMAKE_BUILD_TYPE:STRING=Release
-DWITH_FFMPEG:BOOL=OFF
-DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR}/INSTALL
)
set( OPENCV_ROOT_DIR ${CMAKE_BINARY_DIR}/INSTALL )
set( OPENCV_DIR ${CMAKE_BINARY_DIR}/INSTALL )
It could also be interesting to take a look at a cross-platform package
manager for C++ called hunter, which is based on CMake EP.
I have the following situation:
I'm working on an application that depends on a number of third party libs, among them wxwidgets
I build the application for multiple target configurations (x86, arm, Linux, Windows) using Linux as my build host system
Due to the above mentioned multiple target configurations, I have chosen to build those third-party libs from source, using CMake's ExternalProject_Add function.
The third-party libs are built 'on-demand' at a location separate from my application's CMAKE_BINARY_DIR so that I can wipe the build tree for my application without having to rebuild the third-party libs (takes a looooong time).
The location of the third-party libs is different depending on what target configuration I build them for (obviously).
I'm quite new to CMake and the problem I currently face is this:
The source files in my application can't find the wx include files and I need to set the correct linker flags to be able to link my application against wxwidgets.
This seems to be handled by a utility 'wx-config' that provides exactly that info as output when run with either the --cppflags or --libs flag. I can not however, figure out how to catch that output and append it to the include dirs and linked libraries I setup from my CMakeLists.txt files.
So basically what I want is.
Build wxwidgets (if it doesn't exist) for the current target configuration
Run wx-config --cppflags and --libs to find out the correct include dirs and linker flags for the current target configuration
Use the info from step 2 when building targets that are my own application
So far I've tried something like this:
# Set a target-configuration-specific location
set(wxwidgetsTop ${MYPROJECT_EXTERNAL_DIR}/wxwidgets/wxwidgets_${MYPROJECT_CURRENT_TARGET_CFG})
# Build the project
ExternalProject_Add( wxWidgetsExternal
PREFIX ${wxwidgetsTop}
URL ${MYPROJECT_EXTERNAL_DIR}/tarballs/wxWidgets-3.0.2.tar.bz2
SOURCE_DIR ${wxwidgetsTop}/src/wxwidgets
CONFIGURE_COMMAND ${configure_cmdline}
BUILD_COMMAND make -j${MYPROJECT_NCPU}
INSTALL_COMMAND make install
)
# Create a wxwidgets target to be used as a dependency from other code
add_library(wxWidgets IMPORTED STATIC GLOBAL)
add_dependencies(wxWidgets wxWidgetsExternal)
# (non-working) attempt to get the correct include dirs and linker
# flags for wxwidgets
add_custom_command(TARGET wxWidgetsExternal
POST_BUILD
COMMAND ${INSTALL_DIR}/bin/wx-config ARGS --cppflags
COMMENT "Running wx-config"
)
but the above does not provide a way to actually use the result from the custom command to append the cppflags and linker options when building the targets that make up my application.
What is a good way to achieve what I want?
I see three different ways of doing this:
Method 1: use find_package
Use wxWidgets as a standalone requirement for your project, and expect the devs to install it before building your project. In your CMakeLists.txt you will need to call find_package(wxWidgets), like this:
find_package(wxWidgets COMPONENTS net gl core base)
if(wxWidgets_FOUND)
include(${wxWidgets_USE_FILE})
# and for each of your dependent executable/library targets:
target_link_libraries(<YourTarget> ${wxWidgets_LIBRARIES})
endif()
This has the advantage of not rebuilding the lib if you rebuild your project, however it requires some work for your user (they need to handle the installation of wxWidgets by hand) and for you (you need to setup include paths / compile definitions / ... by hand).
Method 2: embed wxWidgets
The second option is to bundle wxWidgets in your repo (svn external or git submodule) and usually (re)write the CMakeLists.txt of this lib to be target-oriented. Then, in your top-most CMakeLists.txt, you can do the following:
# for example, if you just need core and net:
target_link_librairies(my_app PUBLIC wxWidgetsCore wxWidgetsNet)
# No need to manually setup include dirs, etc...
To make a CMakeLists.txt target-oriented, you define include directories and other compilation properties for a target, not a directory. Example:
# When defining wxWidgetsCore, for example
add_library(wxWidgetsCore ...)
target_include_directories(wxWidgetsCore PUBLIC someDir)
target_compile_definitions(wxWidgetsCore PUBLIC -pedantic)
target_link_libraries(wxWidgetsCore PUBLIC someLib)
The drawback of this approach is that rebuilding your project will trigger a rebuild of wxWidgets. However, it is possible to trick this by not using "rebuild" but "clean just my app, then build". Here is some insight on how to achieve this.
Method 3: some sort of hybrid
The big drawback of method 2 leads to the third approach: don't put wxWidgets in your project, but create a CMakeLists.txt that will "import" the lib. The idea: you ask your user for the directory where wxWidgets is installed, then this script will setup everything for your project. First, put the CMakeLists.txt here:
/your-project-root
/thirdparty
/wxWidgets
CMakeLists.txt
/dir-where-wxwidgets-is-installed
...
Now, you define an imported target:
# When defining wxWidgetsCore, for example
set(WX_INCLUDE_DIR ${USER_SPECIFIED_WX_ROOT}/include)
add_library(wxWidgetsCore IMPORTED GLOBAL)
set_property(TARGET wxWidgetsCore APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES ${WX_INCLUDE_DIR})
See INTERFACE_INCLUDE_DIRECTORIES and INTERFACE_LINK_LIBRARIES. You need your user to have build wxWidgets somewhere in his system, but from your point of view you just do target_link_libraries(your_app PUBLIC wxWidgets...), as in method 2. The advantage is that this approach is interchangeable with method 2 transparently, and you don't put the whole dependency in your project.
Setting cppflags and linker flags has to be done at CMake time, but you are trying to run wx-config at build time and you are not capturing its output anyway, so your add_custom_command() isn't doing anything useful other than printing things to the build tool's output.
Ideally, you would use the FindwxWidgets module CMake already provides. It requires wxWidgets to already be built (but see further below). Have a look at the CMake documentation for it and see if that at least sounds like what you are trying to achieve manually by using wx-config. If you can get FindwxWidgets to do the job for you, that would be a much cleaner approach.
Getting something to build at configure time so you can use it later on in your CMakeLists.txt file is a bit more tricky. ExternalProject_Add() downloads and builds things at build time, but you need wxWidgets to be built earlier at configure time. I wrote an article recently for how to do at least the downloading part at configure time and you should be able to adapt it to do the whole build at configure time instead. The article uses Google Test as its example and can be found here:
https://crascit.com/2015/07/25/cmake-gtest/
It would be trivial to make it put the wxWidgets build wherever you like, not just in the CMAKE_BINARY_DIR area. That would allow you to have different wxWidgets builds for each build configuration and to be able to wipe out your application's build tree independently of the wxWidgets builds.
Hope that points you in the right direction.
The solution I use checks for wxWidgets installation in the system using find_package, if it's not found, then the script downloads wxWidgets from github and links the program against downloaded library. The lib is installed in the build directory, so only the first build is slow - subsequent builds do not even check wxWidgets sources timestamps, so the process is as fast as building using preinstalled wxWidgets library.
Here's how my script does it:
It quietly checks for wxWidgets installation using find_package(wxWidgets QUIET),
If it's found, the script adds a dummy library wxWidgets_external,
If it's not, then it creates an ExternalProject named wxWidgets_external which downloads, builds and installs the library in the build dir, setting wxWidgets_ROOT_DIR to point to the wxWidgets installation dir,
Then we add another ExternalProject pointing to a folder with the main program's source files and CMakeLists.txt build script. This external projects depends on wxWidgets_external which is either a dummy library in case wxWidgets is preinstalled in the system, or an external project set up to download the library from github,
In the aforementioned CMakeLists.txt we again call find_package, this time with REQUIRED parameter and use the library the standard way (https://docs.wxwidgets.org/trunk/overview_cmake.html). Because we set up the dependencies and variables correctly, this call will use either preinstalled wxWidgets (if it's available) or the one downloaded from github.
There are more quirks to it, but that's the gist of it. The full sample code (tested on Linux, Windows and Mac) is available on github (https://github.com/lszl84/wx_cmake_template).
Also see full blog post which explains this in more detail: https://justdevtutorials.medium.com/wxwidgets-cmake-multiplatform-superbuild-4ea86c4e6eda