Keeping file hierarchy across subdirectories in CMake - c++

Till date I still do not really understand what the 'best practice' is for doing this for a CMake project with many subdirectories.
Say I have a project hierarchy as such and each subdirectory has source files in it...
--CMake Project Source dir
|-- SubD1
|-- SubSubD1
|-- SubD2
What I would usually do is to do add_subdirectory(SubD1) and respectively for D2 in the CMakeLists.txt of the root directory and recursively for the subdirectory in the CMakeLists.txt of the SubD1 directory, while declaring variables in each subdirectory and making them visible in the root directory with PARENT_SCOPE.
That means if a file Source2.cpp exists in `SubSubD1', I'd simply do
set(SUBSUBD1_SOURCES Source2.cpp PARENT_SCOPE)
and expect to be able to use SUBSUBD1_SOURCE in my SubD1 directory.
Subsequently, say Source.cpp exists in SubD1, I would do
set(SUBD1_SOURCES ${SUBSUBD1_SOURCES} Source.cpp PARENT_SCOPE)
so that all sources would be visible in root dir.
The problem is of course that the file paths aren't kept when the variables arrive at the root directory. What I'm currently doing is for all source files that I set, I include a ${CMAKE_CURRENT_LIST_DIR}, making it
set(SUBSUBD1_SOURCES ${CMAKE_CURRENT_LIST_DIR}/Source2.cpp PARENT_SCOPE)
and
set(SUBD1_SOURCES ${SUBSUBD1_SOURCES} ${CMAKE_CURRENT_LIST_DIR}/Source.cpp PARENT_SCOPE)
In this case, I could then say, do add_executable(myProg SUBSUBD1_SOURCES) in the root directory of my CMake project.
Are there any better ways of doing this then having to always include a CMake variable in front of all source files?

There is a fourth way if you're using newer versions of CMake.
Take a look at target_sources() command of CMake.
It seems like you are declaring your target in your CMakeLists.txt
add_executable(my_target "subd1/CMakeLists.txt" "subd2/CMakeLists.txt")
add_subdirectory(subd1)
add_subdirectory(subd2)
Instead of propagating your Source files up to the root you can depend on the target you have defined in the root CMakeLists.txt. That means subd1/CMakeLists.txt may look like:
target_sources(my_target PRIVATE "subd1/Source.cpp" "subd1/Source2.cpp")
[EDIT]
As stated in the comments you must give the relative path of the source-files to target_sources(). I use target_sources() because I do not want the explicit source file listing to pollute the targets CMakeLists.txt. Another use case is that target_sources() can be invoked with the PUBLIC or INTERFACE keyword to propagate source files to depending targets. Well I never used target_sources() that way.
[/EDIT]
If you're using IDEs like Visual Studio that support folders you make want to also declare a source_group() in the CMakeLists.txt that contains your target. So the root CMakeLists.txt may look like:
add_executable(my_target "subd1/CMakeLists.txt" "subd2/CMakeLists.txt")
add_subdirectory(subd1)
add_subdirectory(subd2)
...
source_group(subd1 REGULAR_EXPRESSION "subd1/*")
source_group(subd2 REGULAR_EXPRESSION "subd2/*")
I'm using this approach because it leads to much cleaner CMakeLists.txt files, its lesser work and I think the introduction of not needed variables only raises the complexity of your CMakeLists.txt files.
CMakeLists.txt as target sources
I currently use the CMakeLists.txt of the sub folders as source files of the target because otherwise CMake will complain that the add_executable command has no source files given.

There are 3 ways I have used before. I normally prefer the 1st way, but have already used all 3 depending on the use case:
1. You directly name the sources in your root CMakeLists.txt file
set(
SUBD1_SOURCES
"SubD1/SubSubD1/Source2.cpp"
"SubD1/Source.cpp"
)
set(
SUBD2_SOURCES
"SubD2/Source3.cpp"
)
add_executable(myProg ${SUBD1_SOURCES} ${SUBD2_SOURCES})
2. You use OBJECT intermediate libraries to collect/group your sources
SubD1/SubSubD1/CMakeLists.txt:
add_library(SubSubD1Objs OBJECT Source2.cpp)
SubD1/CMakeLists.txt:
add_subdirectory(SubSubD1)
add_library(SubD1Objs OBJECT Source.cpp)
CMakeLists.txt:
add_executable(myProg $<TARGET_OBJECTS:SubSubD1Objs> $<TARGET_OBJECTS:SubD1Objs>)
3. You write your own function() to collect the data (and do the prefixing)
CMakeLists.txt:
function(my_collect_sources)
foreach(_source IN ITEMS ${ARGN})
if (IS_ABSOLUTE "${_source}")
set(source_abs "${_source}")
else()
get_filename_component(_source_abs "${_source}" ABSOLUTE)
endif()
set_property(GLOBAL APPEND PROPERTY GlobalSourceList "${_source_abs}")
endforeach()
endfunction(my_collect_sources)
add_subdirectory(SubD1)
#add_subdirectory(SubD2)
get_property(MY_SOURCES GLOBAL PROPERTY GlobalSourceList)
add_executable(myProg ${MY_SOURCES})
SubD1/CMakeLists.txt:
add_subdirectory(SubSubD1)
my_collect_sources(Source.cpp)
SubD1/SubSubD1/CMakeLists.txt:
my_collect_sources(Source2.cpp)

In your case there's no need to use add_subdirectory since you have only one target which is created in the root CMakeLists.txt. You can simply write this:
add_executable(myProg
SubD1/Source.cpp
SubD1/SubSubD1/Source2.cpp)
Use add_subdirectory for subdirectories creating their own targets so there's no information to pass upwards.

Related

CMake and external dependency

I'd like to add an external dependency to my project. The one I'm trying to add is the Leptonica library as a submodule.
My project has the following directory structure:
|root
CMakeLists.txt
|-bin
|-build
|-buildsystem
|-executable
|-leptonica
|--CMakeLists.txt
|--cmake
|---Configure.cmake
|-production
In my root CMakeLists.txt file I added ADD_SUBDIRECTORY(${ROOT_DIR}/leptonica)
Unfortunately, CMake is not searching for Configure.cmake in the proper directory:
CMake Error at leptonica/CMakeLists.txt:107 (include):
include could not find load file:
Configure
CMake Error: File
<root>/cmake/templates/LeptonicaConfig-version.cmake.in does not exist.
CMake Error at leptonica/CMakeLists.txt:113 (configure_file):
configure_file Problem configuring file
When I build the project by myself, everything goes fine. In my opinion, the problem is with CMAKE_SOURCE_DIR. When using add_subdirectory it has the value of ROOT CMake instead ROOT/leptonica, so it's searching the wrong paths - as you can see in Leptonica CMake, it's used to determinate paths of its files.
What should be the proper way to fix this - should I set CMAKE_SOURCE_DIR to ROOT/leptonica just before calling add_subdirectory and set it back when it's finished, or does some other, more elegant solutions exist?
Not every CMake project is suitable for inclusion via add_subdirectory.
Among those are projects which uses CMAKE_SOURCE_DIR or CMAKE_BINARY_DIR variables.
However, inclusion via ExternalProject_Add (optionally wrapped with execute_process) always works.
Modifying variable CMAKE_SOURCE_DIR (and CMAKE_BINARY_DIR too) is a bad idea: this variable should be changed only by CMake itself. Otherwise you may get weird errors.
Instead, you may replace (automatically, with some script) all references to the variable with another variable, which is not used in the project. This new variable you may safely set before stepping into the subproject.
${CMAKE_SOURCE_DIR} and ${CMAKE_BINARY_DIR} are set relative to the top-level CMakeLists.txt. If you need something relative to your current CMakeLists.txt (leptonica), use ${CMAKE_CURRENT_SOURCE_DIR} and ${CMAKE_CURRENT_BINARY_DIR}.
If you're having trouble finding a cmake file like LeptonicaConfig-version.cmake.in, try appending the appropriate directory to ${CMAKE_MODULE_DIR}.
list(APPEND ${CMAKE_MODULE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/cmake/templates)
I prefer to use ${CMAKE_CURRENT_SOURCE_DIR} over ${CMAKE_SOURCE_DIR} any day because using the latter will break your build if you try to integrate it into a super-build later. If I need to pass my current top-level directory to subdirectories, then I do the following and use that later down the chain.
set( LEPTONICA_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR} )

CMake add library with subdirectories

TL;DR
Using CMake, how can I include subdirectories into a library such that they can be included without referencing the directories they reside?
End TL;DR
In attempt to be brief and speak in higher level ideas of what and how, I have removed everything that I consider to be unnecessary details. I will make edits if need be. As such, this is a brief synopsis of my project structure.
ParentDir
--src
----source.cpp
----source.h
----entities_dir
------entity.cpp
------entity.h
------CMakeLists.txt
----CMakeLists.txt
--CMakeLists.txt
--main.cpp
as it currently stands, I have a library defined by the CMakeLists in the src directory. As such, I can include src files in main by #include as apposed to #include "src/file.h" I would like to be able to do the same for my headers that exist within the subdirectories of src.
CMakeLists.txt
cmake_minimum_required(VERSION 3.6)
project(Project)
add_executable(Project ${SOURCE_FILES} main.cpp)
include_directories(src)
add_subdirectory(src)
target_link_libraries(Project Library) # Engine Libraries
src/CMakeLists.txt
file(GLOB SOURCE_FILES *.cpp)
file(GLOB HEADER_FILES *.h)
add_library(Library STATIC ${SOURCE_FILES} ${HEADER_FILES})
main.cpp
#include <source.h> // this works
#include <entity.h> // this does not work but I want it to
#include <entities/entity.h> // this works but I don't want this
int main() {}
I am not sure how to do this exactly. I have tried to GLOB_RECURSE, add_subdirectory(entities), etc. I have also tried creating a library called Entities inside the src/entities/CMakeLists.txt and linking that with link_libraries. None of these have been successful. What is the proper way to accomplish this, because I think I am probably approaching this completely wrong.
You need that path in your compilers header search path, which is achieved with include_directories() call. You can amend your existing include_directories(src) call to be:
include_directories(src src/entities)
Also, this SO post is related and worth reading: Recursive CMake search for header and source files. There is an excerpt there from the CMake website itself recommending against the usage of file(GLOB ...), which lends to recommending against recursive solutions in general. As a heavy user of CMake, I agree with the arguments made against it.
You just need to add:
include_directories(${CMAKE_CURRENT_LIST_DIR})
In each CMakeLists.txt in your your hierarchy.
CMake in itself doesn't compile your project, it only calls your toolchain and passes parameters to it, and your toolchain doesn't 'know' that it is being called by CMake, or the structure of your project. As such, you need to tell the toolchain where to locate include files.
While it is commonplace to have a CMakeLists.txt in every subdirectory, this is by no means a requirement. The CMakeLists.txt in your src directory could contain all instructions necessary to generate a build. Generally, CMakeLists.txt are put at each level to make the structure easier to manage, eg. each directory only needs to know what it needs to do (presumably, with the files in that directory).

CMake doesn't include header directory of submodule A within submodule B

I have a CMake project that looks like this:
project/
CMakeLists.txt
subprojectA/
CMakeLists.txt
include/
headerA.hpp
src/
libraryA.cpp
subprojectB/
CMakeLists.txt
src/
mainB.cpp
The "library" subproject, A, is compiled as a static library, becoming libsubprojectA.a. The "main" project, B, is compiled as a binary and depends on the library. mainB.cpp includes a reference to headerA.hpp.
Here is subprojectA/CMakeLists.txt:
project(SubProjectA)
include_directories(include)
add_library(subprojectA STATIC src/libraryA.cpp)
set(${PROJECT_NAME}_INCLUDE_DIRS
${PROJECT_SOURCE_DIR}/include
CACHE INTERNAL "${PROJECT_NAME}: Include Directories" FORCE)
And here is subprojectB/CMakeLists.txt:
project(SubProjectB)
include_directories(${SubProjectA_INCLUDE_DIRS})
add_executable(mainBinary src/mainB.cpp)
target_link_libraries(mainBinary subprojectA)
The main Project CMakeLists.txt looks like:
project(Project)
add_subdirectory(subprojectB)
add_subdirectory(subprojectA)
Note that subprojectB, the main project, is listed before subprojectA.
Here's the problem. When I first run "cmake" on this project, ${SubProjectA_INCLUDE_DIRS} is not set within SubProjectB.
What I think is happening is that the CMakeLists for SubProjectB loads first, when ${SubProjectA_INCLUDE_DIRS} has not yet been set. It sets its own include path to an empty string as a result. However, even though libsubprojectA.a gets built successfully before mainBinary, the include path was already set empty beforehand. As a result, I get this error when trying to make mainBinary:
subprojectB/src/mainB.cpp:1:23: fatal error: headerA.hpp: No such file or directory
#include "headerA.hpp"
^
It's a workaround to put subprojectA before subprojectB in the main Project CMakeLists in the declarative world of CMake. What I really want is to know the proper way to indicate to CMake that the include_directories(${SubProjectA_INCLUDE_DIRS}) line depends on the definitions that exist inside SubProjectA's CMakeLists. Is there a better way to do this?
If you want to express that include directory subprojectA/include is an interface of the library subprojectA, attach this property to the target with target_include_directories command:
subprojectA/CMakeLists.txt:
project(SubProjectA)
add_library(subprojectA STATIC src/libraryA.cpp)
# PUBLIC adds both:
# 1) include directories for compile library and
# 2) include directories for library's interface
target_include_directories(subprojectA PUBLIC include)
So any executable(or other library) which linked with subprojectA will have this include directory automatically:
subprojectB/CMakeLists.txt:
project(SubProjectB)
add_executable(mainBinary src/mainB.cpp)
target_link_libraries(mainBinary subprojectA)
Of course, for use last command properly you need to process directory with library before one with executable:
CMakeLists.txt:
project(Project)
add_subdirectory(subprojectA)
add_subdirectory(subprojectB)

CMake with gmock

I just want to make sure that my understanding about CMakeLists.txt is correct. My dummy project structure:
|-+ dummy
|-+ CMakeLists.txt
|-+ src
|-- CMakeLists.txt
|-- Converter.cpp
|-- Converter.hpp
|-- main.cpp
|-+ tests
|-- CMakeLists.txt
|-- Converter_ut.cpp
|-+ thirdparty
|-+ gmock-1.7.0
My goal is to create build process with CMake. This is my first attempt so I assume that there are some mistakes. It works but I am not sure if I understand everything correctly and I would be thankful if you could share with some comments / suggestions.
dummy/CMakeLists.txt
cmake_minimum_required (VERSION 2.8.11)
project (SUB)
add_subdirectory (src)
add_subdirectory (tests)
cmake_minimum_required is pretty self-explanatory,
project (SUB) sets project variables like ${SUB_SOURCE_DIR} and ${SUB_BINARY_DIR},
add_subdirectory, tells CMake to go and process CMakeLists.txt in the following directories
src/CMakeLists.txt
add_library (Sub
main.cpp
Converter.cpp)
target_include_directories (Sub PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
# Executable
add_executable (converter
Converter.cpp)
target_link_libraries (converter Sub)
add_library, creates library called "Sub" from two source files,
target_include_directories, tells the compiler where are the header files for "Sub" library (is that "PUBLIC" really needed here?),
add_executable, creates "converter" executable from Converter.cpp (why main.cpp is not needed here?),
target_link_libraries, links "Sub" library with "converter" executable
tests/CMakeLists.txt
# GMOCK
set (GMOCK_DIR "../thirdparty/gmock-1.7.0")
add_subdirectory(${GMOCK_DIR} ${CMAKE_BINARY_DIR}/gmock)
include_directories(SYSTEM ${GMOCK_DIR}/include ${GMOCK_DIR}/gtest/include)
# Executable
add_executable (tests
Converter_ut.cpp)
target_link_libraries (tests gmock_main Sub)
set (GMOCK_DIR ...), sets local variable "GMOCK_DIR" with my gmock folder location,
add_subdirectory, tells CMake to jump into gmock location and run their CMakeLists.txt, what is the second argument? {CMAKE_BINARY_DIR}/gmock?
add_executable, creates second executable file
target_link_libraries, links gmock_main library with second executable, "Sub" library is needed here because Converter_ut.cpp
needs to include "Converter.hpp" from src directory
Thank you in advance. I have read plenty of sites / tutorials already but I am still not sure about that.
One more thing - I cannot really imagine project with plenty of source files. Isn't there a better way to add source files to add_library and add_executable functions than listing it manually? Something like "take all *.cpp files from current directory"?
Thanks.
Cmake is not properly a programming language supporting a full paradigm, so use it, but if possible never start creating "a framework with it" (it would be cumbersome without proper syntactic sugar), it is intended to make small scripts not to write thousand lines of code (and despite few good frameworks exists, I tend to not use them: "If I cannot code it in few lines, then it's not job for CMAKE").
The important parts are (not that it is slightly different, I copy-pasted the improved version i still have to commit):
cmake_minimum_required( VERSION 2.8)
project( Infectorpp2)
# find additional cmake scripts (I'm driving away from this)
set( CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
# create list of files from a glob
file( GLOB INFECTOR_SOURCE_FILES
"include/Infectorpp/priv/*.cpp")
# create list of files from a glob
file( GLOB INFECTOR_HEADER_FILES
"include/Infectorpp/priv/*.hpp"
"include/Infectorpp/*.hpp")
# or just "add_executable" the dollar "${}" just expand the list
add_library( libInfectorpp2 STATIC
${INFECTOR_SOURCE_FILES}
${INFECTOR_HEADER_FILES})
If you are not using 3rd party libraries, then you do not need to add target_include_directories because for your own application relative paths suffice.
For the testing part you are mostly ok to me, but I would do:
enable_testing()
## details omitted...
# create list of files from a glob
file( GLOB GMOCK_TESTS_SOURCE_FILES
"locationToTestFiles/*.cpp")
# Executable
add_executable (tests
${GMOCK_TESTS_SOURCE_FILES})
target_link_libraries (tests gmock_main Sub)
add_test(tests tests)
Also note that CMAKE is the only reason why I find useful having different extensions for C++ files, because of GLOBS, if you want to exclude some file you have to change its extension (to cc, c++, cxx or what your prefer).
You can do mostly anything following the same pattern, note that with GLOB you have to re-configure CMake to detect newly added files, however still better than adding them manually to build script (and anyway that will not cause a whole recompilation, CMake keep track of data and will avoid to re-compile old files)
Some people find useful adding files manually to CMake scripts because "I can keep old file there". Don't do that, move old files into an "old" folder, or just let your subversion system keep memory of them for you.
You will catch most errors earlier, and you will have a "ready to ship" project (you won't accidentally left wrong files that users will attempt to compile)
Another important point:
Do out of source builds, from your script I guess you are still not doing that.

How can I build a C++ project with multiple interdependent subdirectories?

I have a C++ project where I've used directories as more of an organizational element -- the way one might use packages in Java or directories in PHP. Directories are not intended to be self-sufficient elements, but rather just a way of organizing the whole of the project and keeping me from being overwhelmed by sources. How can I construct my CMakeLists.txt files to deal with this? Making the directories libraries doesn't seem to fit here, since they are all interdependent and not intended to be used that way.
As a related issue, most of the examples I've seen of multiple subdirectories in CMake (and there aren't very many of those) have ignored or glossed over the issue of setting include_directories, which is something I've been having trouble with. Short of combing my source files to determine which file depends on which and in what directory, is there anyway to just set all directories under /src/ as potential include directories and let CMake work out which ones are actually dependent?
Here's an example structure:
--src
--top1
--mid1
--bot1
--src1.cpp
--hdr1.h
--bot2
--src2.cpp
--hdr2.h
--mid2
--bot3
--src3.cpp
--src4.cpp
--hdr3.h
--top2
--mid3
--src5.cpp
--hdr4.h
So on and so forth. How can I structure my CMakeLists.txt files to handle this sort of structure?
Since the directory structure in your project is just there to keep your files organized, one approach is to have a CMakeLists.txt that automatically finds all sources files in the src directory and also adds all directories as include directories that have a header file in them. The following CMake file may serve as a starting point:
cmake_minimum_required(VERSION 3.12)
project (Foo)
file (GLOB_RECURSE Foo_SOURCES CONFIGURE_DEPENDS "src/*.cpp")
file (GLOB_RECURSE Foo_HEADERS CONFIGURE_DEPENDS "src/*.h")
set (Foo_INCLUDE_DIRS "")
foreach (_headerFile ${Foo_HEADERS})
get_filename_component(_dir ${_headerFile} PATH)
list (APPEND Foo_INCLUDE_DIRS ${_dir})
endforeach()
list (REMOVE_DUPLICATES Foo_INCLUDE_DIRS)
add_executable(FooExe ${Foo_SOURCES})
target_include_directories(FooExe PRIVATE ${Foo_INCLUDE_DIRS})
The two file(GLOB_RECURSE ... commands determine the set of source and header files. The foreach loop computes the set of include directories from the list of all header files. The CONFIGURE_DEPENDS flags tells CMake to re-run the glob command at build time.
One drawback with computing the set of source files is that CMake will not automatically detect when new files are added to your source tree. You manually have to re-create your build files then.
Though #sakra gave a good answer to this question, I believe it is more proper to approach it more in depth.
We want to separate our code into modules and libraries for many reasons. Like code encapsulation, re usability, easier debugging etc. This idea would propagate in compiling process too.
In other word, we want to divide the compilation process into little compilation steps, each belong to one module. So every module must have its own compilation procedure. This is why we use one CMakeLists.txt file per directory. Hence every directory would have its own compilation commands and there would be one master CMakeLists.txt file in the root directory of your project.
Here is an example. Consider the following structure of a project:
src/
|
- main.cpp
|
_sum/
|
- sum.h
|
- sum.cpp
We would have one CmakeLists.txt Per directory. First directory is the root directory of the project which src/ folder is in it. here is content for that file:
cmake_minimum_required(VERSION 3.4)
project(multi_file)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_FLAGS "-Wall")
add_subdirectory(src)
Next CMakeLists.txt would located in src/ directory:
add_subdirectory("sum")
add_executable(out main.cpp)
target_link_libraries(out sum)
And the last one will be in the sum/ directory:
add_library(sum SHARED sum.cpp)
I hope this helps. I created a github repository in case you feel you need to see the code or you need further explanation.
I'm not an expert on CMake but since there are no other answers I'll take a look at the documentaton and give it a go. Organizing source and include files in different directories is pretty much the norm.
It looks like CMake allows you to give a list of include directories:
http://www.cmake.org/cmake/help/cmake-2-8-docs.html#command:include_directories
So something like:
include_directories("src/top1/mid1/bot1" "src/top1/mid1/bot2/" ... )
These are passed to the compiler so it can find the header files and will be passed for each of the source files. So any of your source files should be able to include any of the header files (which I think is what you're asking for).
Similar to that you should be able to list all your source files in the add_executable command:
add_executable(name "src/top1/mid1/bot1/src1.cpp" "src/top1/id1/bot2/src2.cpp" ...)
So this would be a naive way of getting everything to build. Each source file will be compiled and will look for headers in all those directories and then the object files will get linked together. Consider if there is any way of simplifying this such that you don't need so many include folders, maybe there are only a few common header files that need to be referenced by all source files. If things get more complex you can buiild sub-hierarchies into libraries etc. Also consider seperating source files and headers (e.g. in src and include).