Avoiding many "../include" in CMake - c++

I just started getting into CMake with C++ and was wondering how other programmers avoid having to do "../include" in all their CMakeFiles.txt.
One example is here: https://github.com/clab/cnn/blob/master/examples/CMakeLists.txt
They create an executable for each example without having to call INCLUDE_DIRECTORIES(...).
I tried adding the headers when calling ADD_LIBRARY(...), but that didn't seem to work.
Example:
tl/src/CMakeLists.txt:
SET(SRCS "x1.cpp" "x2.cpp")
SET(HDRS "../include/tl/x1.h" "../include/tl/x2.h")
INCLUDE_DIRECTORIES("../include")
ADD_LIBRARY(test_lib ${SRCS} ${HDRS})
tl/CMakeLists.txt:
PROJECT(TEST_LIB VERSION 0.1)
ADD_SUBDIRECTORY("src")
tl/examples/CMakeLists.txt:
INCLUDE_DIRECTORIES("../include")
ADD_EXECUTABLE(e1 e1.cpp)
TARGET_LINK_LIBRARIES(e1 test_lib)
Edit: I believe that INCLUDE_DIRECTORIES(...) is only necessary one per each directory throughout the tree.

Just add the INCLUDE_DIRECTORIES command at the top level. No need to explicitly add included files then.
tl/CMakeLists.txt:
PROJECT(TEST_LIB VERSION 0.1)
INCLUDE_DIRECTORIES("include")
ADD_SUBDIRECTORY("src")
ADD_SUBDIRECTORY("examples")
tl/src/CMakeLists.txt:
ADD_LIBRARY(test_lib "x1.cpp" "x2.cpp")
tl/examples/CMakeLists.txt:
ADD_EXECUTABLE(e1 e1.cpp)
TARGET_LINK_LIBRARIES(e1 test_lib)

How about this? tl/CMakeLists.txt
SET(SRCS "x1.cpp" "x2.cpp")
SET(HDRS "../include/tl/x1.h" "../include/tl/x2.h")
INCLUDE_DIRECTORIES("${CMAKE_PROJECT_DIR}/include")
ADD_LIBRARY(test_lib ${SRCS} ${HDRS})
You do not need to add headers to ADD_LIBRARY or ADD_EXECUTABLE if you want to compile your programs and libs. Only source files "cpp,c,cxx,..." are required because through the include macro #include you tell the compiler where it finds the header files.
With INCLUDE_DIRECTORIES(...) you only add a search path to the compiler where to look for headers. If you have your headers in the same directory as your source you don't need another search path. Also subpaths with include macro like this #include "../../include" is possible. So it really depends on the structure of your source files. Also remember, compiler settings setups know where some of the system headers are found. That is why you also do not need to define them.
And last but not least there are cmake scripts and pkg search files where adding paths to specific libraries is done automatically.

This is how I did it in a project of mine:
file(GLOB_RECURSE SRC
engine/*.cpp
platfoorm/*.cpp
)
file(GLOB_RECURSE INCLUDES
engine/*.h
platform/*.h
)
This generate an internal "hardcoded" list of files that the generated makefil will use, so if you add a new source file you will need to re-run cmake.
Of course you can change and add paths, the one in the example are the one I used in my project.

Related

CMake add library with subdirectories

TL;DR
Using CMake, how can I include subdirectories into a library such that they can be included without referencing the directories they reside?
End TL;DR
In attempt to be brief and speak in higher level ideas of what and how, I have removed everything that I consider to be unnecessary details. I will make edits if need be. As such, this is a brief synopsis of my project structure.
ParentDir
--src
----source.cpp
----source.h
----entities_dir
------entity.cpp
------entity.h
------CMakeLists.txt
----CMakeLists.txt
--CMakeLists.txt
--main.cpp
as it currently stands, I have a library defined by the CMakeLists in the src directory. As such, I can include src files in main by #include as apposed to #include "src/file.h" I would like to be able to do the same for my headers that exist within the subdirectories of src.
CMakeLists.txt
cmake_minimum_required(VERSION 3.6)
project(Project)
add_executable(Project ${SOURCE_FILES} main.cpp)
include_directories(src)
add_subdirectory(src)
target_link_libraries(Project Library) # Engine Libraries
src/CMakeLists.txt
file(GLOB SOURCE_FILES *.cpp)
file(GLOB HEADER_FILES *.h)
add_library(Library STATIC ${SOURCE_FILES} ${HEADER_FILES})
main.cpp
#include <source.h> // this works
#include <entity.h> // this does not work but I want it to
#include <entities/entity.h> // this works but I don't want this
int main() {}
I am not sure how to do this exactly. I have tried to GLOB_RECURSE, add_subdirectory(entities), etc. I have also tried creating a library called Entities inside the src/entities/CMakeLists.txt and linking that with link_libraries. None of these have been successful. What is the proper way to accomplish this, because I think I am probably approaching this completely wrong.
You need that path in your compilers header search path, which is achieved with include_directories() call. You can amend your existing include_directories(src) call to be:
include_directories(src src/entities)
Also, this SO post is related and worth reading: Recursive CMake search for header and source files. There is an excerpt there from the CMake website itself recommending against the usage of file(GLOB ...), which lends to recommending against recursive solutions in general. As a heavy user of CMake, I agree with the arguments made against it.
You just need to add:
include_directories(${CMAKE_CURRENT_LIST_DIR})
In each CMakeLists.txt in your your hierarchy.
CMake in itself doesn't compile your project, it only calls your toolchain and passes parameters to it, and your toolchain doesn't 'know' that it is being called by CMake, or the structure of your project. As such, you need to tell the toolchain where to locate include files.
While it is commonplace to have a CMakeLists.txt in every subdirectory, this is by no means a requirement. The CMakeLists.txt in your src directory could contain all instructions necessary to generate a build. Generally, CMakeLists.txt are put at each level to make the structure easier to manage, eg. each directory only needs to know what it needs to do (presumably, with the files in that directory).

How to make cmake find a shared library in a subfolder

I'm trying to learn how to make a shared library. And the following seems to work (please comment if you have some feedback to this method, I basically have no idea what I'm doing).
In my library project, I've put the header files into a folder named "include", and the source files into "src".
My library's CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(mycustomlib)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Include header files
include_directories(include)
# Create shared library
add_library(${PROJECT_NAME} SHARED ${SOURCES})
# Install library
install(TARGETS ${PROJECT_NAME} DESTINATION lib)
# Install library headers
file(GLOB HEADERS include/*.h)
install(FILES ${HEADERS} DESTINATION include)
My application's CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(myprogram)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Create executable
add_executable(${PROJECT_NAME} ${SOURCES})
# Find and link library
find_library(MYCUSTOMLIB mycustomlib)
target_link_libraries(${PROJECT_NAME} ${MYCUSTOMLIB})
And this is working. The problem is that I want to put both the headers and the library into subfolders (specifically: /usr/local/include/mycustomlib/ for the headers, and /usr/local/lib/mycustomlib/ for the library).
So this is my attempt:
My library's new CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(mycustomlib)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Include header files
include_directories(include)
# Create shared library
add_library(${PROJECT_NAME} SHARED ${SOURCES})
# Install library
install(TARGETS ${PROJECT_NAME} DESTINATION lib/${PROJECT_NAME})
# Install library headers
file(GLOB HEADERS include/*.h)
install(FILES ${HEADERS} DESTINATION include/${PROJECT_NAME})
My application's new CMakeLists.txt:
cmake_minimum_required(VERSION 2.4.0)
project(myprogram)
# Find source files
file(GLOB SOURCES src/*.cpp)
# Create executable
add_executable(${PROJECT_NAME} ${SOURCES})
# Find and link library
find_library(MYCUSTOMLIB mycustomlib/mycustomlib)
target_link_libraries(${PROJECT_NAME} ${MYCUSTOMLIB})
And this is not working. Now, I'm forced to specify the .so file of the library like this:
find_library(MYCUSTOMLIB mycustomlib/libmycustomlib.so)
How come?
I'll deal with your actual problem first and offer additional comments after that. Technically speaking, you are asking CMake to find a library named mycustomlib/mycustomlib, but what you really want to say is you want find mycustomlib and it can be found in a subdirectory called mycustomlib. A couple of alternative ways to call find_library() to achieve this for your second case would be:
find_library(MYCUSTOMLIB mycustomlib PATH_SUFFIXES mycustomlib)
find_library(MYCUSTOMLIB mycustomlib PATHS /usr/local/lib/mycustomlib)
The latter is making more assumptions than it should about where you have the library installed, so I'd favour the first option. The first option assumes CMake would already find libraries in /usr/local/lib, which it seems it is from your question. You can influence where CMake looks for libraries by modifying CMAKE_PREFIX_PATH and CMAKE_LIBRARY_PATH. I'd expect either of the above options to make your second case work.
Now to other observations. You've requested a very old minimum CMake version in the first line of each of your CMakeLists.txt files. You probably want to consider at the very least making this 2.8 (personally, I'd suggest more like 3.2 or later, but it depends on what your project needs to support).
You have used file globbing to obtain your list of sources and headers. This is not robust and should generally be avoided (see a discussion of this here). You will see plenty of example code use method this for simplicity, but it is not recommended for real world projects (the CMake documentation even says not to use it). Explicitly list out your source and header files individually if you want robust builds.
If you are happy to require CMake 2.8.11 or later (and you should be these days), rather than calling include_directories() which makes everything pick up the header search path you specified, you should prefer to attach the search path requirement to the target that needs it. You do this with target_include_directories(). The equivalent of your code above would be:
target_include_directories(${PROJECT_NAME} PUBLIC include)
This gives much better control of your inter-target dependencies as your project grows in size and complexity. For a more in-depth discussion of this topic, see this article and perhaps also this one (disclosure: I wrote both articles).
Are your library and program totally separate source code repositories? Can they be built in the same project? You can build multiple targets in one CMakeLists.txt file. The project name doesn't have to have any relationship to the names of any of the targets (you often see the PROJECT_NAME variable re-used for the target name in simple examples, which is unfortunate since it suggests a relationship between the two, but for all but simple projects this won't be the case). If they are in the same repository, building them together would be a much simpler build since you wouldn't have to install the library for the executable to find it and link to it.
If they must be built in separate projects, then something like the following for the application's project should get you close:
cmake_minimum_required(VERSION 2.8.11)
project(myprogram)
# List your program's sources here explicitly
add_executable(myprogram src/foo.cpp src/bar.cpp)
# Find and link library
find_library(MYCUSTOMLIB mycustomlib PATH_SUFFIXES mycustomlib)
target_link_libraries(myprogram PUBLIC ${MYCUSTOMLIB})
# Find library's headers and add it as a search path.
# Provide the name of one header file you know should
# be present in mycustomlib's include dir.
find_path(MCL_HEADER_PATH mycustomlib.h PATH_SUFFIXES mycustomlib)
target_include_directories(myprogram PUBLIC ${MCL_HEADER_PATH})
For extra points, you could try to confirm that the header path is in the same area as the library by checking the common path prefix, or you could just derive
the MCL_HEADER_PATH from the MYCUSTOMLIB path by assuming a directory structure. Both approaches have advantages and drawbacks. If you want to explore the latter, the get_filename_component() command will be your friend.
Hopefully that points you in the right direction.

SWIG and CMake: make use of information provided by `target_include_directories()`

With CMake 2.8+ you can avoid setting include directories redundantly by using target_include_directories().
E.g. by writing
add_libary(mylib SHARED ${SOURCES})
target_include_directories(mylib PUBLIC ./include)
.. you just have to link against mylib to add the needed include folder to your target.
But how can I make use of this information when I have to use CMake modules which don't make use of this capability yet? (in my case SWIG)
When I configure a SWIG project I currently have to hard code a lot of information:
set(SWIG_MODULE_${PYTHON_MODULE_NAME}_EXTRA_DEPS
"../long/relative/path/1/include/some/header1.h"
"../long/relative/path/1/include/some/header2.h"
"../long/relative/path/2/include/some/header1.h"
"../long/relative/path/2/include/some/header2.h")
I also have to use the old fashioned include_directories() to make the swig generator know what it needs to know:
include_directories(
"../long/relative/path/1/include
"../long/relative/path/2/include)
Otherwise the %include statements inside .i files won't work any more.
Of course I could set variables containing the paths but then I would provide the information I wanted to get rid of..
Is there a way to either extract the directory information from a target or (better of course) make the SWIG CMake module correctly use it?
My current solution:
With some (very beautiful) CMake magic you can automate listing all header files from the interface part of a library and set the include directories:
function(swig_add_library_dependencies swig_module library_names)
foreach(library_name ${library_names})
get_property(LIBRARY_INCLUDES
TARGET ${library_name}
PROPERTY INTERFACE_INCLUDE_DIRECTORIES)
foreach(INCLUDE_PATH ${LIBRARY_INCLUDES})
include_directories(${INCLUDE_PATH})
file(GLOB_RECURSE header_files "${INCLUDE_PATH}/*.h")
list(APPEND SWIG_MODULE_${swig_module}_EXTRA_DEPS ${header_files})
# export variable to parent scope
set(SWIG_MODULE_${swig_module}_EXTRA_DEPS
${SWIG_MODULE_${swig_module}_EXTRA_DEPS} PARENT_SCOPE)
endforeach()
endforeach()
endfunction()
to be used like this:
swig_add_library_dependencies(<swig_module_name> "library1;library2")
or discretely like this:
swig_add_library_dependencies(<swig_module_name> library1)
swig_add_library_dependencies(<swig_module_name> library2)
Disadvantages:
uses GLOB_RECURSE
only works if target_include_directories had been used correctly
creates dependencies to all header files found in include directories
Have a look at the documentation for get_property:
https://cmake.org/cmake/help/v3.0/command/get_property.html?highlight=get_property
you would do something like this:
get_property(MY_INCLUDES TARGET my_target PROPERTY INTERFACE_INCLUDE_DIRECTORIES)
to get the interface include directories from the target my_target and store them in the variable MY_INCLUDES

Recursive CMake search for header and source files

I am new to CMake and would like to ask if somebody can help in the following problem.
I have C++ source and header files in their respective folders and now, I want to make a CMake text file that recursively searches for them.
Currently, I am doing it in this way:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
PROJECT(CarDetectorDAISY)
file(GLOB_RECURSE SRCS *.cpp)
file(GLOB_RECURSE HDRS *.h)
ADD_EXECUTABLE(stereo_framework ${SRCS} ${HDRS})
TARGET_LINK_LIBRARIES(stereo_framework)
This creates my CarDetectorDAISY.sln solution file and when I try to build it, it shows
an error that header files are not found (No such file or directory).
It would be really grateful if someone can please help me. Thanks.
You're probably missing one or more include_directories calls. Adding headers to the list of files in the add_executable call doesn't actually add then to the compiler's search path - it's a convenience feature whereby they are only added to the project's folder structure in IDEs.
So, in your root, say you have /my_lib/foo.h, and you want to include that in a source file as
#include "my_lib/foo.h"
Then in your CMakeLists.txt, you need to do:
include_directories(${CMAKE_SOURCE_DIR})
If, instead you just want to do
#include "foo.h"
then in the CMakeLists.txt, do
include_directories(${CMAKE_SOURCE_DIR}/my_lib)
I should mention that file(GLOB...) is not the recommended way to gather your list of sources - you should really just add each file explicitly in the CMakeLists.txt. By doing this, if you add or remove a source file later, the CMakeLists.txt is modified, and CMake automatically reruns the next time you try and build. From the docs for file:
We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.
My answer is mostly a hack, but I find it useful if you don't want to add all dir manually.
This macro will recursively scan all sub-directories (and their sub-directories, etc ...). If a directory contains a header (.h) file, it will append its path to the return_list. This list can then be used with target_include_directories.
MACRO(HEADER_DIRECTORIES return_list)
FILE(GLOB_RECURSE new_list *.h)
SET(dir_list "")
FOREACH(file_path ${new_list})
GET_FILENAME_COMPONENT(dir_path ${file_path} PATH)
SET(dir_list ${dir_list} ${dir_path})
ENDFOREACH()
LIST(REMOVE_DUPLICATES dir_list)
SET(${return_list} ${dir_list})
ENDMACRO()
Usage:
HEADER_DIRECTORIES(header_dir_list)
list(LENGTH header_dir_list header_dir_list_count)
message(STATUS "[INFO] Found ${header_dir_list_count} header directories.")
target_include_directories(
my_program
PUBLIC
${header_dir_list} # Recursive
)
Macro credit: Christoph
Tested with Cmake 3.10
Just to further clarify one point in Fraser's answer:
Headers should not be passed to ADD_EXECUTABLE.
The reason is that the intended compilation command on Linux for example is just:
gcc main.c mylib.c
and not:
gcc main.c mylib.c mylib.h
The C pre-processor then parses mylib.c, and sees a #include "mylib.h", and uses it's search path for those files.
By using include_directories instead, we modify the cpp preprocessor search path instead, which is the correct approach. In GCC, this translates to adding the -I flag to the command line:
gcc -Inew/path/to/search/for/headers main.c mylib.c

Handling header files dependencies with cmake

I am using CMake on a small C++ project and so far it works great... with one twist :x
When I change a header file, it typically requires recompiling a number of sources files (those which include it, directly or indirectly), however it seems that cmake only detects some of the source files to be recompiled, leading to a corrupted state. I can work around this by wiping out the project and rebuilding from scratch, but this circumvents the goal of using a make utility: only recompiling what is needed.
Therefore, I suppose I am doing something wrong.
My project is very simply organized:
a top directory where all resources sit, the main CMakeLists.txt sits there
a "include" directory where all public headers lies (in various subdirectories)
a "src" directory where all the subdirectories for sources files are, the src CMakeLists.txt sits there
a CMakeLists.txt per subdirectory in the "src" directory
The main directory has:
cmake_minimum_required(VERSION 2.8)
project(FOO)
set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)
# Compiler Options
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -std=c++0x -Wall -Wextra -Werror")
include_directories($(FOO_SOURCE_DIR)/include)
add_subdirectory(src)
The "src" directory:
add_subdirectory(sub1)
add_subdirectory(sub2)
add_subdirectory(sub3)
add_subdirectory(sub4)
add_executable(foo main.cpp)
target_link_libraries(foo sub1 sub2 sub3 sub4)
Where sub4 depends on sub3 which depends on sub2 which depends on sub1
And an example of a subdirectory (sub3):
set(SUB3_SRCS
File1.cpp
File2.cpp
File3.cpp
File4.cpp
File5.cpp
File6.cpp
)
add_library(sub3 ${SUB3_SRCS})
target_link_libraries(sub3 sub1 sub2)
I'd be glad if anyone could point my mistake to me, searching here or on CMake didn't yield anything so I guess it's very easy or should work out of the box...
(for reference, I am using cmake version 2.8.2 on MSYS)
EDIT:
Thanks to Bill's suggestion I have checked the depend.make file generated by CMake, and it is indeed lacking (severely). Here is an example:
src/sub3/CMakeFiles/sub3.dir/File1.cpp.obj: ../src/sub3/File1.cpp
Yep, that's all, none of the includes were referenced at all :x
You should look at the depend.make files in your binary tree. It will be in CMakeFiles/target.dir/depend.make. Try to find one of those that is missing a .h file that you think it should have. Then create a bug report for cmake or email the cmake mailing list.
I just hit the same issue. After changing paths in include_directories() from absolute to relative it added appropriate dependencies.
Looks like CMake tries to guess which headers are system and which are project related. I suspect that directories that starts with / passed as -isystem /some/path and thus are not presented in generated dependencies.
If you can't replace ${FOO_SOURCE_DIR} with relative path you can try to calculate relative path using appropriate CMake functions. I.e.:
file(RELATIVE_PATH FOO_SOURCE_REL_DIR
${CMAKE_CURRENT_SOURCE_DIR}
${FOO_SOURCE_DIR}/.)
include_directories(${FOO_SOURCE_REL_DIR}/include)
Did you run cmake before or after adding the includes to your cpp files?
I ran into this problem and re-running cmake fixed it. I'd added the include post-cmake.
Apparently cmake removes system include paths from the dependency trees (thank you #ony for this hint). This probably makes sense most of the time, but sometimes cmake doesn't know what the compiler thinks is a system path or not. We are using a custom gcc build that ignores /usr/include, but cmake thinks it doesn't ignore it. To force cmake to make /usr/include a dependency that is NOT optimized away, try this trick: prepend /. to the path.
I am trying to make all of the library dependencies use the cmake dependency feature, including certain "3rd" party libraries that are not always installed by default on Linux or even available. For example, Z-lib compression.
The following interface target worked fine if the Z lib includes were not in /usr/include, but would break if they were.
find_package(ZLIB REQUIRED)
message(status "found zlib ${ZLIB_LIBRARIES}")
message(status "found zlib includes ${ZLIB_INCLUDE_DIRS}")
target_link_libraries(zlib_target INTERFACE ${ZLIB_LIBRARIES})
target_include_directories(zlib_target INTERFACE ${ZLIB_INCLUDE_DIRS})
I changed the last line to
target_include_directories(zlib_target INTERFACE /.${ZLIB_INCLUDE_DIRS})
and it worked. Now targets that depended on zlib_target would automatically get -I/./usr/include during compilation.
Make sure that the include statement for the missing header is placed before the first program instruction. In my case cmake depend.make was not including the header file, because it was following the first program instruction.