How to avoid numerous builds of shared dependency in cmake - c++

Question

my team has a utility library, lets call it Utils which is built with CMake.
it is ExternalProject_Added in another library's CMake, let's call it A.
our executable, App, ExternalProject_Adds A, and Utils.
out problem is that utils is built twice, once when A is built and again when App is built when it could have been built only once.
Here are examplary CMake files:
Utils:
cmake_minimum_required(VERSION 3.16.3)
project(Utils)
include_directories(${PROJECT_SOURCE_DIR}/Inc)
file(GLOB SOURCE_FILES ${PROJECT_SOURCE_DIR}/Src/*.cpp)
add_library(${PROJECT_NAME} STATIC ${SOURCE_FILES})
install(TARGETS ${PROJECT_NAME} DESTINATION ${CMAKE_INSTALL_PREFIX})
A:
cmake_minimum_required(VERSION 3.16.3)
project(A)
include_directories(${PROJECT_SOURCE_DIR}/Inc)
file(GLOB SOURCE_FILES ${PROJECT_SOURCE_DIR}/Src/*.cpp)
add_library(${PROJECT_NAME} STATIC ${SOURCE_FILES})
install(TARGETS ${PROJECT_NAME} DESTINATION ${CMAKE_INSTALL_PREFIX})
include(ExternalProject)
target_link_libraries(${PROJECT_NAME} ${PROJECT_SOURCE_DIR}/Lib/libUtils.a)
ExternalProject_Add(Utils
GIT_REPOSITORY ${SOME_GIT_URL}/_git/Utils
GIT_TAG master
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${PROJECT_SOURCE_DIR}/Lib
)
ExternalProject_Get_Property(Utils SOURCE_DIR)
target_include_directories(${PROJECT_NAME} PRIVATE ${SOURCE_DIR}/Inc/)
add_dependencies(${PROJECT_NAME} Utils)
App:
cmake_minimum_required(VERSION 3.16.3)
project(App)
include_directories(${PROJECT_SOURCE_DIR}/Inc)
file(GLOB SOURCE_FILES ${PROJECT_SOURCE_DIR}/Src/*.cpp)
add_executable(${PROJECT_NAME} ${SOURCE_FILES})
install(TARGETS ${PROJECT_NAME} DESTINATION ${CMAKE_INSTALL_PREFIX})
include(ExternalProject)
target_link_libraries(${PROJECT_NAME} ${PROJECT_SOURCE_DIR}/Lib/libUtils.a)
ExternalProject_Add(Utils
GIT_REPOSITORY ${SOME_GIT_URL}/_git/Utils
GIT_TAG master
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${PROJECT_SOURCE_DIR}/Lib
)
ExternalProject_Get_Property(Utils SOURCE_DIR)
target_include_directories(${PROJECT_NAME} PRIVATE ${SOURCE_DIR}/Inc/)
add_dependencies(${PROJECT_NAME} Utils)
target_link_libraries(${PROJECT_NAME} ${PROJECT_SOURCE_DIR}/Lib/libA.a)
ExternalProject_Add(A
GIT_REPOSITORY ${SOME_GIT_URL}/_git/A
GIT_TAG master
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${PROJECT_SOURCE_DIR}/Lib
)
ExternalProject_Get_Property(A SOURCE_DIR)
target_include_directories(${PROJECT_NAME} PRIVATE ${SOURCE_DIR}/Inc/)
add_dependencies(${PROJECT_NAME} A)
how can we achieve the wanted result?

Answer 1

ExternalProject_Add fetches, builds and installs a separate project. It supports invoking many different build systems: Makefiles, Automake, ninja, CMake, and pretty much anything else too if you supply it the appropriate build commands. This means, however, that ExternalProject_Add has no real way of "communicating" with the build system it's invoking. It simply invokes it, waits for it to run, and then provides the installed files (binaries, headers, etc) to the main project being built.
In your case, this means that when "A" is built, a separate instance of CMake is launched to build it. Once this separate instance has finished running, the "A" target is considered to be built, and "libA.a" can be used. The targets defined within the build of "A" are not visible to the build of "App" - they're separate CMake builds and simply don't share their targets. Therefore, the version of "Utils" that "A" has built is not immediately visible to "App".
There is a way to make this work, though: Simply make "A" install the "Utils" library that it builds. In the end, the build and installation of "A" should place both libA.a and libUtils.a in the target install directory (i.e. ${CMAKE_INSTALL_PREFIX}/lib). It must also place the necessary headers in that install directory (i.e. in ${CMAKE_INSTALL_PREFIX}/include to keep with conventions).
In general, you also shouldn't put the install directories of subprojects into the project's source tree. Instead, put them at ${CMAKE_CURRENT_BINARY_DIR}/some_subdirectory. Once the library is built, you can copy it to the install directory (with the install command) if you want the library to be visible to users of the project.
Here's a rough (incomplete) outline of what the build script of "A" will have to do:
cmake_minimum_required(VERSION 3.16.3)
project(A)
include_directories(${PROJECT_SOURCE_DIR}/Inc)
file(GLOB SOURCE_FILES "${PROJECT_SOURCE_DIR}/Src/*.cpp")
add_library(${PROJECT_NAME} STATIC ${SOURCE_FILES})
install(TARGETS ${PROJECT_NAME} DESTINATION ${CMAKE_INSTALL_PREFIX}/Lib)
# TODO: Also install headers of A to ${CMAKE_INSTALL_PREFIX}/Inc
include(ExternalProject)
ExternalProject_Add(Utils
GIT_REPOSITORY ${SOME_GIT_URL}/_git/Utils
GIT_TAG master
INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}/Utils_install
)
ExternalProject_Get_Property(Utils INSTALL_DIR)
target_include_directories(${PROJECT_NAME} PRIVATE ${INSTALL_DIR}/Inc/)
target_link_libraries(${PROJECT_NAME} ${INSTALL_DIR}/Lib/libUtils.a)
add_dependencies(${PROJECT_NAME} Utils)
# TODO: Install libraries from ${INSTALL_DIR}/Lib to ${CMAKE_INSTALL_PREFIX}/Lib
# TODO: Install headers from ${INSTALL_DIR}/Inc to ${CMAKE_INSTALL_PREFIX}/Inc
"Utils" will similarly have to install both its binary and its headers. (The example script assumes that "Utils" places its binary in ${CMAKE_INSTALL_PREFIX}/Lib and its headers in ${CMAKE_INSTALL_PREFIX}/Include, just like "A".)
Once you've changed the build scripts of "Utils" and "A" like that, "App" only has to build "A" and can use the copy of "Utils" that "A" provides.