I have a C++ project that requires two external libraries (boost and htslib). I link the libraries to my projects target octopus as follows:
find_package (Boost 1.65 REQUIRED COMPONENTS ${REQUIRED_BOOST_LIBRARIES} REQUIRED)
if (Boost_FOUND)
target_include_directories (octopus PRIVATE ${Boost_INCLUDE_DIR})
target_link_libraries (octopus ${Boost_LIBRARIES})
endif (Boost_FOUND)
find_package (HTSlib 1.4 REQUIRED)
if (HTSlib_FOUND)
target_include_directories (octopus PRIVATE ${HTSlib_INCLUDE_DIRS})
target_link_libraries (octopus ${HTSlib_LIBRARIES})
endif (HTSlib_FOUND)
Both boost and htslib are usually installed into /usr/local, and therefore have header files in /usr/local/include. However, users can specify alternative library locations by specifying CMake variables BOOST_ROOT and HTSLIB_ROOT.
The problem is that if only one of the libraries is given an alternative location, then the header files in the include directory of the other linked library (e.g. in /usr/local/include) get included for both libraries, and if incomparable versions of the library are installed then compilation can fail. For example, if I set BOOST_ROOT to ~/.linuxbrew then Boost_INCLUDE_DIR is correctly set to ~/.linuxbrew/include, but HTSlib_INCLUDE_DIRS is /usr/local/include, which contains /usr/local/include/boost, and for reasons I don't quite understand, these are the headers used for building, even though they are incompatible with the libraries in ~/.linuxbrew/lib/boost.
How can I ensure that the include directory for a linked library are used only for that library?
Which header file is considered is a matter of include directories order - the compiler will typically consider the first file that matches the given name in any of the include folders. While you cannot tell the compiler to look for one file in one specific directory and for the other in another specific directory, what you can determine to some degree is the order of the include directories. It sounds like in your case, the boost include directory should be checked before the other, so try the BEFORE keyword in the target_include_directories call, like this:
target_include_directories (octopus BEFORE PRIVATE ${Boost_INCLUDE_DIR})
Related
What is the file structure of CMake library?
For example, here is my library located in library folder:
library
\mylib1.so
\mylib2.so
The headers are in other dir out of library.
Can I use find_library(mylib1 PATHS library) to find my library and use target_include_directories() or include_directories() to include my headers? For my testing, it's failed. So what's the right structure?
Do I need put the header files (.h or .hpp) into library/include folder and put .so in lib folder or put them all into the library folder?
I didn't find any explain in CMake documents. I also find other commands in CMake documents but they are lack of something in details that I don't understand. So I'd like to known how do I find the documents like this.
find_library is used to find a file, but it doesn't create identify provide any information about include directories or other information required by the linking target.
To be able to use your lib after installing it's best to make use of find_package.
You could place files like into a install directory MyLibrary like this:
MyLibrary/lib/MyLib-0.1.1/MyLibraryConfig.cmake
MyLibrary/lib/MyLib-0.1.1/MyLibraryConfigVersion.cmake
MyLibrary/lib/MyLib-0.1.1/libmylibrary.so
MyLibrary/include/MyLib-0.1.1/include/...
MyLibraryConfigVersion.cmake
This file is used to check, if the version of the library is compatible with the build configuration of the project using find_package. Additionaly it provides cmake with information about the version of the library:
set(PACKAGE_VERSION 0.1.1)
if(NOT UNIX # must be the same target platform
OR NOT CMAKE_SIZEOF_VOID_P EQUAL 8 # assuming the installed lib is 64 bit
OR PACKAGE_VERSION VERSION_LESS PACKAGE_FIND_VERSION)
set(PACKAGE_VERSION_COMPATIBLE FALSE) # inform find_package this is not a suitable version of the lib
else()
set(PACKAGE_VERSION_COMPATIBLE TRUE)
if(PACKAGE_FIND_VERSION STREQUAL PACKAGE_VERSION)
set(PACKAGE_VERSION_EXACT TRUE)
endif()
endif()
MyLibraryConfig.cmake
This file is used to actually create the imported lib to use by the project calling find_package; It's only included, if the version file considers the configuration suitable.
if (NOT TARGET MyLibrary) # avoid issues with using find_package(MyLibrary) multiple times
add_library(MyLibrary IMPORTED)
# provide the necessary info about the target
set_target_properties(MyLibrary PROPERTIES
IMPORTED_LOCATION "${CMAKE_CURRENT_LIST_DIR}/libmylibrary.so" # specify absolute library file location using the directory containing this cmake file
)
target_include_directories(MyLibrary INTERFACE "${CMAKE_CURRENT_LIST_DIR}/../../include/MyLib-0.1.1/include")
endif()
This allows you to "tell" cmake about the location of the install dir and use find_package like this:
list(APPEND CMAKE_PREFIX_PATH /path/to/MyLibrary) # this could be passed via -D option during configuration instead
find_package(MyLibrary REQUIRED)
Note that there are alternative paths where to put the cmake configuration files, see the documentation of find_package. Furthermore there is cmake functionality that can take over generating some, if not all of the cmake configuration files for you automatically on installation, see
install()
The CMakePackageConfigHelpers cmake module
I've started playing a little bit with C++ and to make it happen I decided to write a simple game engine.
For this purpose, I'm using CLion as my IDE and it works all good but adding libraries is just a nightmare. First I've installed all required libraries like glew, glfw or glm using brew, all went fine. Then I spent almost 2 hours to get it to work on my project.
My biggest mystery is the reason why it works, I've worked with build systems in java, python or golang and everything was always clear to me. However, I have no idea why it works the way it works and I'd love to know!
Here is my CMakeLists file.
cmake_minimum_required(VERSION 3.10)
project(untitled2)
find_package(GLEW REQUIRED)
find_package(GLFW3 REQUIRED)
set(CMAKE_CXX_STANDARD 17)
add_executable(untitled2 main.cpp)
target_link_libraries(untitled2 ${GLEW_LIBRARIES})
target_link_libraries(untitled2 glfw)
Now I have a few questions:
1. Why am I able to use GLM library without including it in the CMakeLists?
2. Why do I need to include glfw and glew but not glm?
3. Why do I need to use ${GLEW_LIBRARIES} and not some name like glew? (I tried different names, but nothing worked.)
btw. I'm using macOS.
The first thing to remember is that C++ doesn't (yet) have a real module system like newer languages. It just has a list of directories that it searches for header files, a list of directories that it searches for libraries, and a list of libraries to search for symbols when linking. The target_link_libraries directive just adds compiler flags that add to those three lists.
Now, on to this specific scenario. Most of the magic happens in the find_package directive. That directive really just ends up running cmake scripts. Sometimes those are packaged with cmake, sometimes they're installed along with the package you're finding. In the end, those scripts can do basically whatever they want. They all have the same objective, to give you a way to add the appropriate compiler flags to use the package, but there are a couple common ways they do that.
The older way is to set variables that you can use to tell the compiler what directories to search for headers and libraries and what libraries to link to. That's the approach GLEW seems to have taken. It sets the variables GLEW_LIBRARIES and GLEW_INCLUDE_DIRS and you then have to use link_libraries and include_directories to tell the compiler what to do. This was the only approach available in older versions of cmake (pre 2.8 IIRC), so while it's not as nice to use, it is still how many libraries' find_package scripts work.
The newer way is to create imported targets. Those targets have appropriate properties set so that any targets that link to the imported target inherit the appropriate include directories and library flags. This is the approach GLFW took. It creates an imported target named glfw that has the INTERFACE_INCLUDE_DIRECTORIES and INTERFACE_LINK_LIBRARIES properties set. When you pass that to target_link_libraries, your untitled2 target will inherit those include directories and libraries.
Finally, GLM is a header-only library. There are no library files to link to, so as long as the appropriate directory is added to the compilers header search path you'll be able to include and use GLM.
Since you used homebrew to install your libraries, all of their headers are likely under the same base directory; most likely "/usr/local/include". All of their library files are similarly likely under the same directory; probably something "/usr/local/lib". That means that your compiler will be able to find any of their headers and libraries is you tell it to search "/usr/local/include" for headers and "/usr/local/lib" for libraries.
So, to finally answer the question: Thing's work because the glfw target told cmake that it should set the compiler flags to add "/usr/local/include" to its list of include directories. Since that's the same directory it needs to search for GLM and GLEW, the compiler is able to find the headers for all of your libraries. The compiler is also able to find the library files it need to link to because cmake told it to look for them explicitly via the list GLEW_LIBRARIES and the inherited properties from the glfw target. GLM doesn't have any library files to link to, so there's nothing to tell it about.
You really shouldn't rely on everything being in the same place though. You should be able to tell the compiler about everything like this (note that I haven't actually tested this):
cmake_minimum_required(VERSION 3.10)
project(untitled2)
set(CMAKE_CXX_STANDARD 17)
add_executable(untitled2 main.cpp)
# This will fill the variables GLEW_INCLUDE_DIRES and GLEW_LIBRARIES
# that you can use to add the appropriate compiler flags
find_package(GLEW REQUIRED)
# This will create an imported target named glfw that you can link to
# to inherit the appropriate include directories and libraries
find_package(GLFW3 REQUIRED)
# This also creates an imported target named glm that you can "link to"
# to inherit the appropriate include directories
find_package(glm REQUIRED)
# GLEW uses an old-style find_package script, so you have to
# explicitly tell cmake about GLEW's include directories
target_include_directories(untitled2 PUBLIC ${GLEW_INCLUDE_DIRS})
# And the library files to link to
target_link_libraries(untitled ${GLEW_LIBRARIES})
# cmake will automatically add the appropriate include directories
# and library files that the imported glfw target tells it about
target_link_libraries(untitled2 glfw)
# You use the target_link_libraries directive with the glm imported target
# even though you're not actually linking to any libraries. It's just how
# you tell cmake you want your untitled2 target to inherit the appropriate
# include directories from the imported glm target
target_link_libraries(untitled2 glm)
Given a project with "app" and "lib" sibling directories, where "app" builds an executable depending on the (static) library built by "lib". What I'd like is a situation where the normal build process builds only the library, but if I build "app" it builds both "lib" and "app".
What I'm currently doing now is that in app, I include lib with add_subdirectory, but for whatever reason this is pulling in all of lib's indirect dependencies into the link line through a mechanism I'm not aware of. What I'd like is to have my app just build libmylib.a and libmylib.pc, then app could just calculate its own link line from libmylib.pc (or specify it manually), but I'm not sure how that's done.
Here's a minimum working example I've got set up now:
lib/CMakeLists.txt
cmake_minimum_required(VERSION 3.8.0)
project(mylib CXX)
find_package(PkgConfig REQUIRED)
pkg_check_modules("mylib" "libssl")
find_package(Boost REQUIRED)
set(LIBDIR "${PROJECT_SOURCE_DIR}")
set(HEADERS "${LIBDIR}/map_printer.hpp")
set(SOURCES "${LIBDIR}/map_printer.cpp")
add_library("mylib" "${SOURCES}")
target_include_directories("mylib" PUBLIC "${LIBDIR}"
"${Boost_INCLUDE_DIR}"
"${mylib_INCLUDE_DIRS}")
target_link_libraries("mylib" "${Boost_LIBRARIES}" "${mylib_LIBRARIES}")
install(TARGETS "mylib" ARCHIVE DESTINATION "lib")
install(FILES ${HEADERS} DESTINATION "include")
app/CMakeLists.txt
cmake_minimum_required(VERSION 3.8.0)
project(mylib CXX)
set(APPDIR "${PROJECT_SOURCE_DIR}")
set(LIBDIR "${APPDIR}/../lib")
set(SOURCES "${APPDIR}/main.cpp")
add_subdirectory("${LIBDIR}" "build")
list(APPEND LIBS "mylib")
add_executable("myapp" "${SOURCES}")
target_include_directories("myapp" PUBLIC "${LIBDIR}")
target_link_libraries("myapp" "${LIBS}")
install(TARGETS "myapp" DESTINATION "bin")
To get a working example, here are some source files that pull in libssl in the lib (but this function is not used in the app) - I put them in gists because they are only included for completeness and I didn't want to clutter up the question text:
lib/map_printer.cpp
lib/map_printer.hpp
app/main.cpp
The problem is that when I cmake app and then do make VERBOSE=1, the linker command generated is:
/usr/lib/hardening-wrapper/bin/c++ CMakeFiles/myapp.dir/main.cpp.o -o myapp build/libmylib.a -lssl
But I have not specified -lssl anywhere in app. Normally this would be fine, but in my real app, -lssl and several other unnecessary symbols are being included as .so files because of indirect dependencies. When I remove them from the linker command manually, the task builds and runs just fine. Ideally I'd be pulling in the built .a with its .pc file (not generated in this example) and if excess dependencies are necessarily being pulled in, I could tweak the link line manually, but with this method, the linker flags (and possibly other things) are leaking out of the lib scope in some way I don't understand.
Linking is all about resolving symbols so that the final target (standalone executable or shared object) has everything it needs to be launched. Things are simple when you depend only on a single library or upon a collection of libraries that in turn depend on nothing else. This is less likely to be the case for any program of moderate or larger size. The underlying issue is how to deal with transitive dependencies, e.g. the dependencies of the things directly used by your program.
CMake understands all of this and is designed to make it simple for you to consume libraries without understanding the entire dependency graph. If you look at the documentation for target_link_libraries, you'll see the PRIVATE, PUBLIC, and INTERFACE keywords described. This allows you to describe the private requirements of the library (compile definitions, compile arguments, dependent libraries, etc.) that it needs when you build the library. The public part allows you to specify things that both the library and its dependents (consumers) need. The interface part lets you specify things that the dependents need, but not the library itself. The commands target_compile_definitions and target_include_directories operate similarly.
The upshot of all of this is that with a properly declared dependency in CMake, the client of that dependency just adds it to its list of dependencies in its own target_link_libraries command and it naturally picks up all compile definitions, include directories and transitory link dependencies necessary for successful compilation and linking.
A CppCon 2017 presentation Modern CMake for modular design goes over this in more detail.
In CMake most third party libraries don't require me to specify their include directory at all...they seem to take care of that for me behind the scenes. However, some third party libraries seem to put that work on my plate.
Are the differences I'm finding across third party libraries indicative of me doing something wrong? Or is CMake just a less structured environment where some third party libraries are going to hold your hand more than others? What is going on here? Best practices?
I'll give a couple examples. OpenCV makes things super easy, no need for me to mention their include directory:
set(OpenCV_DIR ${THIRD_PARTY_DIR}/OpenCV)
find_package(OpenCV REQUIRED)
...
add_executable(${PROJECT_NAME} ${HEADER_FILES} ${SOURCE_FILES})
target_link_libraries(${PROJECT_NAME} ${OpenCV_LIBS})
However, Google's Protocol Buffer requires me to use the include_directories command in order to use their header files:
set(CMAKE_PREFIX_PATH ${THIRD_PARTY_PROTOBUF_DIR})
find_package(Protobuf ${THIRD_PARTY_DIR}/protobuf-2.6.1)
...
include_directories(${PROTOBUF_INCLUDE_DIRS})
add_library(${PROJECT_NAME} SHARED ${HEADER_FILES} ${SOURCE_FILES})
target_link_libraries(${PROJECT_NAME} ${PROTOBUF_LIBRARIES})
The target_include_directories() command was introduced with CMake version 2.8.11 in May 2013. That gave developers of libraries the possibility of self-propagating include paths.
But changing everything to target_include_directories() does break the backwards compatibility with previous versions of CMake (see target_include_directories prior to 2.8.12?).
So it's mainly a legacy issue I assume will vanish over time.
Most of time you can see what the library is using by the cmake_minimum_required() line at the top of the library's root CMakeLists.txt. And in case of e.g. the Protocol Buffers library, they do have updated to their CMakeLists.txt to cmake_minimum_required(VERSION 2.8.12) and do now make use of target_include_directories() (see commit from July 2015).
I'm using CMake to build a project which among other things link to boost. I use CMake 2.8.7, I have
set(Boost_NO_SYSTEM_PATHS true)
and I use
find_package(Boost COMPONENTS system filesystem regex REQUIRED)
I then link using
target_link_libraries(projectname ${Boost_LIBRARIES})
I use the environment variable BOOST_ROOT to specify the location of Boost, and my question is as follows:
When I set
BOOST_ROOT=/opt/Boost_1_47
CMake passes the full path to the libraries to the linker, whereas if I set
BOOST_ROOT=/usr
it links using
-lboost_filesystem-mt
etc. CMakeLists.txt is the same in both cases, the only thing I change is the environment varible BOOST_ROOT. Why doesn't CMake pass the full path in both cases?
The linker is able to find the libraries which are in standard paths like /lib, /lib64, /usr/lib, /usr/lib64 etc. So in that case CMake does not feel the need to tell the linker where is the library is located. But in case of /opt/boost_1_47, as its not a standard path so linker doesn't know where is the library located.
Just try setting the LINK_DIRECTORIES in CMake to Path/To/Boost/Libraries you will notice a different behavior.