How to add libraries installed with ExternalProject_Add to target includes - c++

I'm trying to include external libraries to a cmake project through ExternalProject_Add. To try out this feature I've created a minimal working example that involves adding pugixml to the project with ExternalProject_Add. However, I'm having problems finding a way to add the library's header files from the external project's local installation (i.e., pugixml's headers) in the project's include path.
The project tree of the minimal working example is organized as follows:
.
├── build
├── CMakeLists.txt
└── src
├── CMakeLists.txt
└── main.cpp
In this project tree, build refers to the build directory and the path where cmake is called to generate the build.
The contents of ./CMakeLists.txt are as follows:
cmake_minimum_required(VERSION 3.0)
include(ExternalProject)
ExternalProject_Add(pugixml
GIT_REPOSITORY https://github.com/zeux/pugixml.git
INSTALL_DIR ${PROJECT_BINARY_DIR}/extern_install/pugixml
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
)
add_subdirectory(src)
In the above example I've added pugixml as an external project that is to be installed within the project's binary dir, whose file will be used by an executable stored in ./src. Thus, the contents of ./src/CMakeLists.txt are:
project(foo)
add_executable(foo main.cpp)
target_link_libraries(foo ${pugixml_LIBRARIES})
include_directories(${pugixml_INCLUDE_DIR}) # this part doesn't work
This is precisely the part I'm having problems. I've assumed that once the external project was added and installed that ExternalProject_Add would have defined some convenience libraries to help refer to library files and include directories. However, that doesn't work. Does anyone know what's the right way of using ExternalProject_Add to include external libraries?

Unfortunately this will not work at all. The build of the external project is done at build time not during CMake configure/generate. The hint is The ExternalProject_Add function creates a custom target to drive download, update/patch, configure, build, install and test steps of an external project: (from CMake)
Hence You will have to define all the variables yourself. See also: CMake - linking to library downloaded from ExternalProject_add() (I'd actually mark this as a duplicate of the linked question as the problem is the same)
BTW: I do also dislike the way this was done. This way one cannot simply use find_package etc.
The intended way to use this is a "super-build project": Define 1 CMakeLists where you only have ExternalProject_Adds for all dependencies AND your project. Then it will work with find_package.

Related

cmake doesn't recognize dependency on static lz4 [duplicate]

I have a program that depends on an external library (SDL for example). I want CMake to take care of that dependency for me, so I was looking into FetchContent. As far as I understand, this module simply downloads the source code so that information on the external library is available at configure time. For example:
include(FetchContent)
FetchContent_Declare(sdl
GIT_REPOSITORY <...>
)
FetchContent_GetProperties(sdl)
# sdl_POPULATED, sdl_SOURCE_DIR and sdl_BINARY_DIR are ready now
if(NOT sdl_POPULATED)
FetchContent_Populate(sdl)
endif()
At some point, however, I want to build that source code and link it to my main executable. How to do it the "modern CMake way"?
The recommended way to build external libraries from source as part of your build depends on what the external lib provides build-wise.
External lib builds with cmake
If the external lib builds with cmake then you could add the lib to your build via a add_subdirectory(${libname_SOURCE_DIR}) call. That way cmake will build the external lib as a subfolder ("subproject"). The CMakeLists.txt file of the external lib will have some add_library(ext_lib_name ...) statements in it. In order to then use the external lib in your targets (an application or library that depends on the external lib) you can simply call target_link_libraries(your_application <PRIVATE|PUBLIC|INTERFACE> ext_lib_name) https://cmake.org/cmake/help/latest/command/target_link_libraries.html
I had a quick look at this github repo https://github.com/rantoniello/sdl - (let me know if you are referring to another library) and it actually looks like it is building with cmake and that it allows clients to statically or dynamically link against it: https://github.com/rantoniello/sdl/blob/master/CMakeLists.txt#L1688-L1740
So, ideally your applicaiton should be able to do
add_executable(myapp ...)
target_link_libraries(myapp PRIVATE SDL2-static) // Statically link againt SDL2
Due to their CMakeLists.txt file the SDL2-static comes with properties (include directories, linker flags/commands) that will automatically propergate to myapp.
External lib does not build with cmake
If a external lib doesn't build with cmake then one can try to use add_custom_target https://cmake.org/cmake/help/latest/command/add_custom_target.html to build the library. Something along the lines of:
add_custom_target(myExternalTarget COMMAND <invoke the repo's build system>)
You'd then need to set the target properties that are important for clients yourself via the the proper cmake functions set_target_properties, target_include_directories ... A great writeup how to get started with these kinds of things: https://pabloariasal.github.io/2018/02/19/its-time-to-do-cmake-right/

Moving external libraries "inside" your project

assuming I currently use
target_link_libraries(...)
in my CMakeLists.txt, which requires the libraries to be installed on the computer I am compiling on, is there a way to move these external libraries inside my project (e.g. into a libs/ folder)?
Yes, but there is multiple ways.
My most preferred is to actually install those libraries in the libs/ directory so the structure look like this:
- libs/
-lib1/
- include
- lib
-lib2/
- include
- lib
It's done by simply installing the library inside the directory of your project:
# inside lib1/build
cmake .. -DCMAKE_INSTALL_PREFIX=~/my/project/libs/lib1
make install
Then, in your project you can set the prefix path to the libs/ directory:
list(APPEND CMAKE_PREFIX_PATH "${CMAKE_SOURCE_PATH}/libs/")
# ...
find_package(lib1) # find the library in libs/lib1
The other ways would be to use add_subdirectory and have the source of the project inside libs. You'll be able to use the target they define so you can link to them.

How to get a VS solution with all the projects and solutions for single projects with CMake

TL;DR I basically want a modular solution of this question. Not only one solution with everything in it, but also a solution for each executable alone.
In my question here I wanted to know how I should split my CMakeLists.txt among different folders. Some folders contain code that will be a static library and some code will be executables or dynamic libraries that build on those static libraries.
Folder structure looks like this:
/path/to/base/
CMakeLists.txt
app1/
<src files for app1>
CMakeLists.txt
app2/
<src files for app2>
CMakeLists.txt
lib/
<src files for lib>
CMakeLists.txt
If I take the approach of having a CMakeLists.txt in the parent folder to all my libraries and executables and from the include all the directories I have the problem that when I generate VS projects/solutions that my projects for each target contains all the projects - not only the ones necessary for my intended target.
Contents of CMakeLists.txt in the base folder:
cmake_minimum_required(VERSION 3.1)
add_subdirectory(lib)
add_subdirectory(app1)
add_subdirectory(app2)
and the CMakeLists.txt in the app1 folder (app2 is equivalent)
cmake_minimum_required(VERSION 3.1)
project(app1)
add_executable(app1 <src of app1>)
target_link_libraries(app1 lib)
and CMakeLists.txt of lib:
add_library(lib <src of lib>)
If I run cmake on the CMakeLists.txt from base the solution contains all the projects. And even if I open just one of the projects; it also contains everything. VS project of app1 also builds app2 - which I don't want.
If I only run cmake on the CMakeLists.txt of app1 I get a solution which doesn't contain app2 but it also doesn't contain lib, because that's only mentioned for linking inside the cmake file and not as a target (it's in the base file)
CMake will create a solution for each call to project, which will then include all the targets of the current directory and its subdirectories (regardless whether they were defined before or after the actual call to project).
Here's what a well-behaved CMake script should do: Each CMakeLists that you want to be able to act as a root of its own sub-tree within the larger project should start with a cmake_minimum_required call, followed by the project call. Writing project in a CMakeLists basically means: This is a fully self-contained component that can be built independently, even if you delete all the files from its parent directories. As such, it makes sense to have a separate solution for each such project.
With this in mind we can see why this doesn't work in your case: Your apps are not fully self-contained, they depend on lib. The solution to this is to write the build scripts for the apps as if lib was provided as a third-party component, with find_package calls and everything.
In the compound build, you already have a target for lib, so the script invoked by the find_package call should be made to short-circuit to using that target and only actually go scavenging the system if it cannot find that existing target.
Also read up on how CMake's packaging system works, which provides some automation to handle such cases pretty elegantly.

Avoid path problems at runtime when using cmake's ExternalProject_Add

I'm pretty new to cmake, but I am stuck with an odd behaviour that I'd like to understand and overcome.
I'm creating an executable that uses the Paho MQTT C library. The content of this executable is irrelevant, let's say it just connects to a broker for now, nothing fancy.
To compile my program, I use cmake with a CMakeLists.txt file, which does a few things:
use an ExternalProject block to pull the MQTT C lib from Github
declare the executables and the linked libs
Here it is :
project(myproject)
include(ExternalProject)
########## CMAKE VERSION AND FLAGS ##########
cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/bin)
########## MQTT EXTERNAL LIB ##########
set(MQTT_BUILD_DIR ${CMAKE_CURRENT_BINARY_DIR}/mqtt)
ExternalProject_Add(
project_mqtt
GIT_REPOSITORY https://github.com/eclipse/paho.mqtt.c
GIT_TAG v1.2.0
PREFIX ${MQTT_BUILD_DIR}
CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"
)
add_library(mqtt STATIC IMPORTED)
set(MQTT_LIB ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/lib/libpaho-mqtt3a.1.2.0.dylib)
set_property(TARGET mqtt PROPERTY IMPORTED_LOCATION ${MQTT_LIB})
add_dependencies(mqtt project_mqtt)
include_directories(${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/include)
link_directories(${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/lib)
########## EXECUTABLE ##########
add_executable(myproject ../src/main.cpp)
target_link_libraries(myproject mqtt)
Note that in the file I especially compile for Mac and I hardcoded the .dylib suffix but this is not a problem here
My folder structure is as follows :
myproject/
-- CMakeLists.txt
-- bin/
-- build/
-- src/
- main.cpp
My compilation workflow is very standard : cd build && cmake ../ && make.
My sources are in /src, I build everything in /build and my executable is written in /bin. The *.dylib are explicitely built locally in the project folder (in bin/lib) since I don't want to install them in the system (I want my project to be self-contained in a way).
It works well (I mean, it compiles).
But when I run the program, I get a dyld: Library not loaded: libpaho-mqtt3a.1.dylib error.
.. which is quite "normal", because my lib has been installed in bin/lib and my executable searches for them in its own folder (which is bin/). So for instance if I run : cd bin/lib && ../myproject it works perfectly well.
So, now here are my questions :
[Philosophically,] Is it a good way of doing this ? I want to use this external lib but I don't want to include the Paho lib sources in my code obviously, so I though ExternalProject was a good way to go. It compiles the lib at compile time before compiling my project and I find the process quite handy. I looked for a "package manager" for C++ but there does not seem to be a strong consensus (I'm looking at something like *composer* or *npm* for web apps), so I did not go this route.
Why, oh why, does my executable look for the library file in its own directory ./ ? Can I change that ? Is it some cmake configuration that I missed ? I guess that I could just use a cmake custom command to copy the library file into the /bin folder, but this looks like a hack to me. Isn't it ?
To be honest, I'd like to have no "path dependencies" in my executable, it's to say that the executable, when compiled on a specific system, could be put anywhere on this system and work flawlessly afterwards. I understand that means that the compiled Paho MQTT lib should be statically in my executable, but I wonder how to do that (or even what that really means, in fact). Is it possible ?
I may miss a whole part of how the C/C++ compile and external lib workflow works but I think I'm close to something quite robust, it's just this lib path thing at runtime that I would like to properly understand and fix.
I'm really open to suggestions on this
Thanks a lot !

How to set up a CMake project correctly?

While it's easy to find surface level information on how to use CMake, information how to actually use CMake properly seems to be extremely hard to find. How should in a decently sized CMake project (one executable, one or more static libraries used by that executable, one or more external dependencies used by the static libraries) the folders and CMakeList.txt files be ordered? What CMakeList.txt files should have what commands?
A good way to learn how to use CMake effectively is by looking at other projects. LLVM and its subprojects are a good example.
Generally, good coding practices convert to good CMake practices; you want modularity, a clear style and flexibility.
An example of this might be to have rules for building your executable inside the src directory, then use that target in the root project folder. Something like this:
-my_proj
|
----CMakeLists.txt //contains directives for top-level dependencies and linking, includes subfolders
----src
|
----CMakeLists.txt //contains definition of your main executable target
----internal_lib
|
----CMakeLists.txt //contains definition of your internal static libraries
my_proj/CMakeLists.txt
add_subdirectory(src)
find_package (Threads REQUIRED) #find pthreads package
target_link_libraries (my_exe my_lib ${CMAKE_THREAD_LIBS_INIT}) #link against pthreads and my_lib
my_proj/src/CMakeLists.txt
add_subdirectory(internal_lib)
add_executable(my_exe my_source1.cpp mysource2.cpp)
my_proj/src/internal_lib/CMakeLists.txt
add_library(my_lib my_lib_source1.cpp my_lib_source2.cpp)
I hope this tutorial is exactly what you need to starting with a CMake configuration for a simple project including one executable and several libraries - take a look! I find the CMake by Example anyway the best possibility to learn CMake the easy way:
Using CMake with executables
add_executable(myapp main.c)
Using CMake with static libraries
add_library(test STATIC test.c)
Using CMake with dynamic libraries
add_library(test SHARED test.c)
Linking libraries to executables with CMake
add_subdirectory(libtest_project)
add_executable(myapp main.c)
target_link_libraries(myapp test)