Related
About a year ago I asked about header dependencies in CMake.
I realized recently that the issue seemed to be that CMake considered those header files to be external to the project. At least, when generating a Code::Blocks project the header files do not appear within the project (the source files do). It therefore seems to me that CMake consider those headers to be external to the project, and does not track them in the depends.
A quick search in the CMake tutorial only pointed to include_directories which does not seem to do what I wish...
What is the proper way to signal to CMake that a particular directory contains headers to be included, and that those headers should be tracked by the generated Makefile?
Two things must be done.
First add the directory to be included:
target_include_directories(test PRIVATE ${YOUR_DIRECTORY})
In case you are stuck with a very old CMake version (2.8.10 or older) without support for target_include_directories, you can also use the legacy include_directories instead:
include_directories(${YOUR_DIRECTORY})
Then you also must add the header files to the list of your source files for the current target, for instance:
set(SOURCES file.cpp file2.cpp ${YOUR_DIRECTORY}/file1.h ${YOUR_DIRECTORY}/file2.h)
add_executable(test ${SOURCES})
This way, the header files will appear as dependencies in the Makefile, and also for example in the generated Visual Studio project, if you generate one.
How to use those header files for several targets:
set(HEADER_FILES ${YOUR_DIRECTORY}/file1.h ${YOUR_DIRECTORY}/file2.h)
add_library(mylib libsrc.cpp ${HEADER_FILES})
target_include_directories(mylib PRIVATE ${YOUR_DIRECTORY})
add_executable(myexec execfile.cpp ${HEADER_FILES})
target_include_directories(myexec PRIVATE ${YOUR_DIRECTORY})
First, you use include_directories() to tell CMake to add the directory as -I to the compilation command line. Second, you list the headers in your add_executable() or add_library() call.
As an example, if your project's sources are in src, and you need headers from include, you could do it like this:
include_directories(include)
add_executable(MyExec
src/main.c
src/other_source.c
include/header1.h
include/header2.h
)
Structure of project
.
├── CMakeLists.txt
├── external //We simulate that code is provided by an "external" library outside of src
│ ├── CMakeLists.txt
│ ├── conversion.cpp
│ ├── conversion.hpp
│ └── README.md
├── src
│ ├── CMakeLists.txt
│ ├── evolution //propagates the system in a time step
│ │ ├── CMakeLists.txt
│ │ ├── evolution.cpp
│ │ └── evolution.hpp
│ ├── initial //produces the initial state
│ │ ├── CMakeLists.txt
│ │ ├── initial.cpp
│ │ └── initial.hpp
│ ├── io //contains a function to print a row
│ │ ├── CMakeLists.txt
│ │ ├── io.cpp
│ │ └── io.hpp
│ ├── main.cpp //the main function
│ └── parser //parses the command-line input
│ ├── CMakeLists.txt
│ ├── parser.cpp
│ └── parser.hpp
└── tests //contains two unit tests using the Catch2 library
├── catch.hpp
├── CMakeLists.txt
└── test.cpp
How to do it
1. The top-level CMakeLists.txt is very similar to Recipe 1, Code reuse with functions and macros
cmake_minimum_required(VERSION 3.5 FATAL_ERROR)
project(recipe-07 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(GNUInstallDirs)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY
${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY
${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY
${CMAKE_BINARY_DIR}/${CMAKE_INSTALL_BINDIR})
# defines targets and sources
add_subdirectory(src)
# contains an "external" library we will link to
add_subdirectory(external)
# enable testing and define tests
enable_testing()
add_subdirectory(tests)
2.Targets and sources are defined in src/CMakeLists.txt (except the conversion target)
add_executable(automata main.cpp)
add_subdirectory(evolution)
add_subdirectory(initial)
add_subdirectory(io)
add_subdirectory(parser)
target_link_libraries(automata
PRIVATE
conversion
evolution
initial
io
parser
)
3.The conversion library is defined in external/CMakeLists.txt
add_library(conversion "")
target_sources(conversion
PRIVATE
${CMAKE_CURRENT_LIST_DIR}/conversion.cpp
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/conversion.hpp
)
target_include_directories(conversion
PUBLIC
${CMAKE_CURRENT_LIST_DIR}
)
4.The src/CMakeLists.txt file adds further subdirectories, which in turn contain CMakeLists.txt files. They are all similar in structure; src/evolution/CMakeLists.txt contains the following:
add_library(evolution "")
target_sources(evolution
PRIVATE
${CMAKE_CURRENT_LIST_DIR}/evolution.cpp
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/evolution.hpp
)
target_include_directories(evolution
PUBLIC
${CMAKE_CURRENT_LIST_DIR}
)
5.The unit tests are registered in tests/CMakeLists.txt
add_executable(cpp_test test.cpp)
target_link_libraries(cpp_test evolution)
add_test(
NAME
test_evolution
COMMAND
$<TARGET_FILE:cpp_test>
)
How to run it
$ mkdir -p build
$ cd build
$ cmake ..
$ cmake --build .
Refer to: https://github.com/sun1211/cmake_with_add_subdirectory
Add include_directories("/your/path/here").
This will be similar to calling gcc with -I/your/path/here/ option.
Make sure you put double quotes around the path. Other people didn't mention that and it made me stuck for 2 days. So this answer is for people who are very new to CMake and very confused.
CMake is more like a script language if comparing it with other ways to create Makefile (e.g. make or qmake). It is not very cool like Python, but still.
There are no such thing like a "proper way" if looking in various opensource projects how people include directories. But there are two ways to do it.
Crude include_directories will append a directory to the current project and all other descendant projects which you will append via a series of add_subdirectory commands. Sometimes people say that such approach is legacy.
A more elegant way is with target_include_directories. It allows to append a directory for a specific project/target without (maybe) unnecessary inheritance or clashing of various include directories. Also allow to perform even a subtle configuration and append one of the following markers for this command.
PRIVATE - use only for this specified build target
PUBLIC - use it for specified target and for targets which links with this project
INTERFACE -- use it only for targets which links with the current project
PS:
Both commands allow to mark a directory as SYSTEM to give a hint that it is not your business that specified directories will contain warnings.
A similar answer is with other pairs of commands target_compile_definitions/add_definitions, target_compile_options/CMAKE_C_FLAGS
I had the same problem.
My project directory was like this:
--project
---Classes
----Application
-----.h and .c files
----OtherFolders
--main.cpp
And what I used to include the files in all those folders:
file(GLOB source_files CONFIGURE_DEPENDS
"*.h"
"*.cpp"
"Classes/*/*.cpp"
"Classes/*/*.h"
)
add_executable(Server ${source_files})
And it totally worked.
You have two options.
The Old:
include_directories(${PATH_TO_DIRECTORY})
and the new
target_include_directories(executable-name PRIVATE ${PATH_TO_DIRECTORY})
To use target_include_directories, You need to have your executable defined - add_executable(executable-name sourcefiles).
So your code should appear like
add_executable(executable-name sourcefiles)
target_include_directories(executable-name PRIVATE ${PATH_TO_DIRECTORY})
You can read more here https://cmake.org/cmake/help/latest/command/target_include_directories.html
This worked for me:
set(SOURCE main.cpp)
add_executable(${PROJECT_NAME} ${SOURCE})
# target_include_directories must be added AFTER add_executable
target_include_directories(${PROJECT_NAME} PUBLIC ${INTERNAL_INCLUDES})
Don't forget to include ${CMAKE_CURRENT_LIST_DIR}.
That's what was causing problems for me.
Example should be like this:
target_include_directories(projectname
PUBLIC "${CMAKE_CURRENT_LIST_DIR}/include"
)
PUBLIC for dependencies which you want to be included by a parent project.
PRIVATE for ones that you don't.
Note to site curators: This answer is very long. In case you are wondering, no it is not from a blog post. I wrote this specifically tailored to answer this question. If you think the length of the answer and its content warrant closing the question as needing focus, then I have no qualms with that. I personally am not a fan of the question anyway, but wanted to give a good answer because it has gotten so much attention over the years and thought the existing answers were lacking in certain ways.
In all the answers to this questions, there is a whole lot of "how" (to get what you want), and precious little "why" (digging into the problem that motivated the question and what the asker may have misunderstood about the ways in which different types of tools like IDEs and build tools do / do not interact and share information with each other, and what information CMake passes / needs to pass to those tools).
This question is vexxing, as it is motivated by a specific behaviour of a specific IDE- Code::Blocks) and CMake, but then poses a question unrelated to that IDE and instead about Makefiles and CMake, assuming that they have done something wrong with CMake which led to a problem with Makefiles, which led to a problem with their IDE.
TL;DR CMake and Makefiles have their own way of tracking header dependencies given include directories and source files. How CMake configures the Code::Blocks IDE is a completely separate story.
What is an "external" header in CMake?
I realized recently that the issue seemed to be that CMake considered those header files to be external to the project. [...]
It therefore seems to me that CMake consider those headers to be external to the project, and does not track them in the depends
As far as I know, there is no official or useful definition of "external header" when it comes to CMake. I have not seen that phrase used in documentation. Also note that the word "project" is a quite overloaded term. Each buildsystem generated by CMake consists of one top-level project, possibly including other external or subdirectory projects. Each project can contain multiple targets (libraries, executables, etc.). What CMake refers to as a target sometimes translates to what IDEs call projects (Ix. Visual Studio, and possibly Code::Blocks). If you had to given such a phrase a meaning, here's what would make sense to me:
In the case that the question is referring to some IDEs' sense of the word "project", which CMake calls "targets", header files are external to a project would be those that aren't intended to be accessed through any of the include directories of a target (Ex. Include directories that come from targets linked to the target in question).
In the case that the question is referring to CMake's sense of the word "project": Targets are either part of a project (defined/created by a call to the project() command, and built by the generated buildsystem), or IMPORTED, (not built by the generated buildsystem and expected to already exist, or built by some custom step added to the generated buildsystem, such as via ExternalProject_Add). Include directories of IMPORTED targets would be those headers which are external to the CMake project in question, and include directories of non-IMPORTED targets would be those that are "part of" the project.
Does CMake track header dependencies? (It depends!)
[...] CMake consider those headers to be external to the project, and does not track them in the depends
I'm not super familiar with the history of CMake, or with header dependency tracking in build tooling, but here is what I've gathered from the searching I have done on the topic.
CMake itself doesn't have much to do with any information related to header/include dependencies of implmentation files / translation units. The only way in which that information is important to CMake is if CMake needs to be the one to tell the generated buildsystem what those dependencies are. It's the generated buildsystem which wants to track changes in header file dependencies to avoid any unnecessary recompilation. For the Unix Makefiles generator in particular, before CMake 3.20, CMake would do the job of scanning header/include dependencies to tell the Makefiles buildsystem about those dependencies. Since v3.20, where supported by the compiler, CMake delegates that resposibility to the compiler by default. See the option which can be used to revert that behaviour here.
The exact details of how header/include dependency scanning differs for each supported CMake generator. For example, you can find some high-level description about the Ninja capabilities/approach on their manual. Since this question is only about Makefiles, I won't attempt to go into detail about other generators.
Notice how to get the header/include dependency information for the buildsystem, you only need to give CMake a list of a target's include directories, and a list of the implementation source files to compile? You don't need to give it a list of header files because that information can be scanned for (either by CMake or by a compiler).
Do IDEs get information about target headers by scanning?
Each IDE can display information in whatever way it wants. Problems like you are having with the IDE not showing headers usually only happen for IDE display formats of the project layout other than the filesystem layout (project headers files are usually in the same project directory as implementation files). For example, such non-filesystem layout views are available in Visual Studio and Code::Blocks.
Each IDE can get header information in whatever way it chooses. As far as I am aware (but I may be wrong for Visual Studio), both Visual Studio and Code::Blocks expect the list of project headers to be explicitly listed in the IDE project configuration files. There are other possible approaches (Ex. header dependency scanning), but it seems that many IDEs choose the explicit list approach. My guess would be because it is simple implementation-wise.
Why would scanning be burdensome for an IDE to find header files associated with a target?(Note: this is somewhat speculation, since I am not a maintainer of any such tools and have only used a couple of them) An IDE could implement the file scanning (which itself is a complicated task), but to know which headers are "in" the target, they'd either need to get information from the buildsystem about how the translation units of the target will get compiled, and that's assuming that all "not-in-target" header include paths are specified with a "system"-like flag, which doesn't have to be the case. Or, it could try to get that information from the meta-buildsystem, which here is CMake. Or it could try to do what CMake now does and try to invoke the selected compiler to scan dependencies. But in either case, they'd have to make some difficult decision about which buildsystems, meta buildsystems, and/or compilers to support, and then do the difficult work of extracting that information from whatever formats those tools store that information in, possibly without any guarantees that those formats will be the same in future tool versions (supporting a change in the format in a newer tool version could be similar to having to supporting a completely separate tool). The IDE could do all that work, or it could just ask you to give it a list of the headers belonging to each target. As you can see, there are cons to the diversity in tooling that the C/C++ ecosystem has. There are pros too, but that's outside the scope of this question.
On the bright side, CMake actually does have a mechanism to try to take some of that work off your shoulders. For such IDEs that have non-filesystem-views, it does implement a simple-heuristic to try to find header files that are associated with source files...
How does header file discovery work for the Code::Block IDE generator for CMake?
At least, when generating a Code::Blocks project the header files do not appear within the project (the source files do).
Here's something interesting: The CodeBlocks editor has the concept of source files and header files that are part of a project, and since CMake doesn't expect/require its users to tell it about each and every header file in the project (it only needs to know about what include directories should be associated with targets), it tries to use a certain heuristic to discover header files that are associated to implementation files. That heuristic is very basic: take the path of each source file in a project, and try changing the extenstion to be like one that is usually given to header files, and see if any such file exists. See the cmExtraCodeBlocksGenerator::CreateNewProjectFile member function in :/Source/cmExtraCodeBlocksGenerator.cxx.
In "Pitchfork Layout" terminology, it would be said that the heuristic assumes that the project uses "merged-header" placement instead of "split-header" placement, where there are separate src/ and include/ directories. So if you don't use merged-header layout, or otherwise have any target headers that don't meet that heuristic, such as utility header files, you'll need to explicitly tell CMake about those files (Ex. using target_sources) for it to pass that knowledge on to the IDE config it generates.
Further readings:
Here's the CMake documentation on its Code::Blocks generator (not much info related to the topic at hand, but good to link anyway).
Here's Code::Blocks' documentation on its "Project View". Here's the .cpb xml schema documentation (see in particular, the Unit element).
If you want to read the CMake code which does the associated header detection, you can find it in the cmExtraCodeBlocksGenerator::CreateNewProjectFile function in the Source/cmExtraCodeBlocksGenerator.cxx file.
Closing Words
I'm certain there are many people who know these tools better than I do. If you are one of those people and notice that I have made a mistake, please graciously correct me in the comments or in chat, or just to edit this post.
Note that while installation of build artifacts is an important part of many projects' lifecycles and is therefore incorporated into the designs of most C/C++ buildsystems, since the question didn't explicitly ask about the configuring the installation part, I have chosen to leave it out of this answer, since it in itself is not a trivial topic to cover (just see how long the related chapters in the "Mastering CMake" book are: The chapter on installation, and the chapter on importing and exporting).
In newer CMake versions we can limit our include-paths to target, like:
target_include_directories(MyApp PRIVATE "${CMAKE_CURRENT_LIST_DIR}/myFolder")
I mean, if the CMakeLists.txt has multiple targets, else, the include-paths are NOT shared with other CMakeLists.txt scripts, and it's enough to do something like:
include_directories("${CMAKE_CURRENT_LIST_DIR}/myFolder")
However, maybe we can simulate what target_include_directories(...) does for CMake 2.8.10 or older versions, like:
set_property(
TARGET MyApp
APPEND PROPERTY
INCLUDE_DIRECTORIES "${CMAKE_CURRENT_LIST_DIR}/myFolder"
)
All done, but seems if you want source-files to be re-compiled once any header-file they use is changed, all such header-files need to be added to each target as well, like:
set(SOURCES src/main.cpp)
set(HEADERS
${CMAKE_CURRENT_LIST_DIR}/myFolder/myHeaderFile.h
${CMAKE_CURRENT_LIST_DIR}/myFolder/myOtherHeader.h
)
add_executable(MyApp ${SOURCES} ${HEADERS})
Where with "seems" I mean that, CMake could detect such header-files automatically if it wanted, because it parses project's C/C++ files anyway.
I am using CLion also my project structure is the following :
--main.cpp
--Class.cpp
--Class.h
--CMakeLists.txt
The CMakeLists.txt before the change:
add_executable(ProjectName main.cpp)
The CMakeLists.txt after the change:
add_executable(ProjectName main.cpp Class.cpp Class.h)
By doing that the program compiled successfully.
I made a Non-Qt project, Plain C++ Application in QT creator. My current folder structure is like this:
.
├── client
│ ├── client.cpp
│ └── client.h
├── CMakeLists.txt
├── CMakeLists.txt.user
├── main.cpp
└── server
├── server.cpp
└── server.h
My CMakeLists.txt file looks like this:
cmake_minimum_required(VERSION 3.5)
project(eshraagh-project LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
add_executable(eshraagh-project main.cpp server/server.h server/server.cpp client/client.h client/client.cpp)
In my main file, I included server.h and client.h and made instances of both of them, and it didn't give any errors.
The problem arises when I try to use an external library (this library). I have no idea how to use it.
I tried to copy contents of its src folder (cpnet-export.h, cpnet-network.h and cpnet-network.c) into my project and add them to CMakeLists, just like I did with server and client files, but it doesn't work. I copied the server part of the repository and pasted it in my main, but it gives me errors such as undefined reference to 'cpnet_bind' and more (A little note: The repository uses the functions with net prefix, but QT recommender told me to write cpnet prefix instead. I don't know why this is happening).
Linking a library of an executable in CMake is achieved by using the target_link_libraries function. In your case,
target_link_libraries(eshraagh-project PRIVATE cpnet)
Here, cpnet could be the name of a target that you CMake project knows (otherwise, it is turned into some kind of platform specific -lcpnet linker flag). In order to make CMake know about the cpnet library, you can
add the external library sources to your project (as a git submodule, or by tracking them yourself), for this example let's say they the sources are located at lib/cpnet
tell CMake to include the configuration instructions from that folder. This works if the external project uses CMake, too, and doesn't mess with global variables and settings (this seems to be the case here).
So in your toplevel CMakeLists.txt:
add_subdirectory(lib/cpnet)
Note that this must be done before the target_link_libraries call.
I'm working on a C++ project organized into libraries as follows:
├── Lib_1
│ ├── ...
│ └── CMakeLists.txt
├── Lib_2
│ ├── ...
│ └── CMakeLists.txt
│ ...
├── Lib_N
│ ├── ...
│ └── CMakeLists.txt
├── Main.cpp
└── CMakeLists.txt
With main executable outside of the folder structure. The main CMakeLists has the following contents:
cmake_minimum_required(VERSION 3.10)
project(MyConsoleApp VERSION 1.0)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
add_subdirectory(Lib_1)
add_subdirectory(Lib_2)
...
add_subdirectory(Lib_N)
add_executable(${PROJECT_NAME} Main.cpp)
target_link_libraries(${PROJECT_NAME}
Lib_1
Lib_2
...
Lib_N
)
and CMakeLists in sub-folders:
set(Lib_k_Src # k = 1,2,...,N
src1.h
src1.cpp
...
)
add_library(Lib_k ${Lib_k_Src})
I'd like to attach a BOOST (or any other) unit test suite to each Library component, and make sure it runs every time a component is built. Or, alternatively, generate executables with test suites that can be run separately from the main exec.
So far, all of my attempts failed at integrating both Boost and CppUnit with the main executable resulting in linker error (usually LNK1104) when attaching a third party unit test library. I've created Windows environment variables for boost include and lib dirs, and tried some available examples with CMake, but these won't even configure in the CMakeGUI. The only luck I've had was with CppUnit in a separate solution without wrappers generated by CMake with a CppTestRunner at runtime through Main.cpp.
Any idea on how to approach this?
I've spent days trying to solve this, and even thought about implementing my own assert macros for testing so that they can be called from main at runtime.
My setup with Boost can be found here. Currently, I've generated a test library Symplekt_GeometryBase_Tests to Symplekt_GeometryBase as a prototype.
Thanks for any helpful insight.
You're missing a number of things. Be sure to re-read Boost's extensive documentation on the different usage variants in which the unit tests library can be consumed during a build.
Use find_package(Boost REQUIRED) to find Boost during CMake configure. Depending on whether you use the header only version or the library version, you will in the latter case need to add unit_test_framework as the required component for the find call. You probably want to do this in your top-level CMakeLists.txt. If this fails to find Boost automatically, try setting the Boost_ROOT environment variable to the installation directory for Boost on your machine, or check out the numerous other answers here on StackOverflow for finding Boost with CMake. (Hint: If this keeps failing for no apparent reason, you probably haven't built/installed Boost correctly).
Have your test executable target pull in Boost as a dependency by calling target_link_libraries(mytest PUBLIC Boost::boost). Again, if you're not using the header-only setup, you will also want to link to Boost::unit_test_framework in the same manner. Get rid of all the ${BOOST_WHATEVER} variables you're currently using, you won't need any of that.
You will want to call enable_testing. This should ideally be done once in the root CMakeLists before including any tests.
Use add_test to register the test targets with CMake's test mechanism. It seems you're already doing this.
Your unit tests will now be registered with CMake's test runner and can be executed through building the respective CMake meta-targets (like RUN_TESTS) or via ctest.
You can have the tests execute automatically during the build by adding a custom build step that invokes the test runner.
I have a package called MYLIBS consisting of two libraries, lib1 and lib2, which I want to export through the configuration file for the package. The project structure is as follows:
├── Lib1
│ ├── CMakeLists.txt
│ ├── lib1-class.cpp
│ └── lib1-class.h
├── lib2
│ └── CMakeLists.txt
│ ├── lib2-class.cpp
│ ├── lib2-class.h
├── cmake
│ └── LIBSConfig.cmake.in
├── CMakeLists.txt
In lib2 I have:
add_library(lib2
STATIC
${SOURCE_FILES}
)
target_include_directories(lib2 PRIVATE /path/to/lib1)
target_link_libraries(lib2 PUBLIC lib1)
add_dependencies(lib2 lib1)
install(
TARGETS
lib2
DESTINATION
lib/MYLIBS/lib2
EXPORT
lib2Exports
)
install(
EXPORT
lib2Exports
DESTINATION
lib/MYLIBS/lib2
)
The same as lib1 except that lib1 does not have the add_dependencies() and target_include/link() as it does not have one.
In my configuration file template, I have:
#PACKAGE_INIT#
## PROJECT_LIBRARIES is filled-in during the package build. in this case : lib1,lib2
set(#PROJECT_NAME#_LIBRARIES #PROJECT_LIBRARIES#)
## The public variables to be used by the client project:
#PROJECT_NAME_INCLUDE_DIRS is all the include paths
#PROJECT_NAME_LIBRARIES is the name of all the libraries
unset(#PROJECT_NAME#_INCLUDE_DIRS)
foreach(INCLUDE_DIR ${INCLUDE_DIRS})
set_and_check(#PROJECT_NAME#_INCLUDE_DIR ${INCLUDE_DIR})
list(APPEND #PROJECT_NAME#_INCLUDE_DIRS ${#PROJECT_NAME#_INCLUDE_DIR})
endforeach()
## PACKAGE_PACKAGE_DIRNAME_include is filled-in during the package build
foreach(lib ${#PROJECT_NAME#_LIBRARIES})
list(APPEND INCLUDE_DIRS #PACKAGE_PACKAGE_DIRNAME_include#/${lib})
endforeach(lib)
# Looks up the information about the exported targets in this package
foreach(lib ${#PROJECT_NAME#_LIBRARIES})
if(NOT TARGET ${lib})
include(#PACKAGE_PACKAGE_DIRNAME_lib#/${lib}/${lib}Exports.cmake)
endif()
endforeach(lib)
So I go through the export files for libraries one by one and include them. The problem is that I have to do that in the right order, i.e. lib1 first and then lib2, otherwise I get an error when reading the configuration file by FindPackage().
I am not really sure how the transitive dependencies would work tbh. Since these libraries are include()ed from the same export file, is there a way of telling CMake about the dependencies in the configuration file or in the export file of lib2 considering that we know where the export files for the dependencies are going to be on the system?
I can see target_link_libraries() has a PUBLIC option. How am I supposed to use that? Would it be of any help?
To begin with, you can remove the add_dependencies line. See target_link_libraries and add_dependencies.
Second, you have
target_include_directories(lib2 PRIVATE /path/to/lib1)
But that should not be needed. Instead, remove it, and add this to lib1:
target_include_directories(lib1 PUBLIC /path/to/lib1)
Those are just clean-ups though.
You didn't post the error, and there is lots of other important information missing in your post, so I do some guessing.
I guess the error is something along the lines of
The following imported targets are referenced, but are missing: lib2
You export lib1 and lib2 in two separate 'export sets' - lib1Exports and lib2Exports. Putting them in one 'export set' would solve the problem and be the easiest way forward, at least in the two-target example.
I guess you know that, and you are not doing it because the scale of your build system is bigger than two targets. However, that leads directly to your problem - it means you must manage the order dependencies between 'export sets'.
This is independent of dependencies between targets. An 'export set' is a different 'unit' with an independent dependency graph. CMake doesn't help you to manage it. You have to manage the dependencies between 'export sets'. The problem is that you are not currently managing or expressing those dependencies. See below for your options regarding expressing those dependencies.
target_link_libraries(PUBLIC) does not help you. Read about it in Transitive Usage Requirements.
If you think of an analogy to preprocessor files, you might see your options. Think of lib2_private.h which does not #include lib1_private.h. A alllibs.h will need to include those two in the correct order. Because the _private headers are private, and because clients will always include alllibs.h instead, that will work. In this approach, you manage the total dependency tree in one place.
An alternative approach would be to create lib2_internal.h which contains
#include "lib1_private.h"
#include "lib2_private.h"
and lib1_internal.h which contains
#include "lib1_private.h"
In this approach, you manage dependencies close to their dependers, so you would have multiple places which specify subsets of the total dependency tree. The alllibs.h could use
#include "lib1_internal.h"
#include "lib2_internal.h"
or
#include "lib2_internal.h"
#include "lib1_internal.h"
and the order would not matter.
Your configuration file with the loop is alllibs.h - it is the only file clients include. Can you manage the order entirely there? Yes, if you can manage the order within the #PROJECT_NAME#_LIBRARIES variable. By the way, you should call that #PROJECT_NAME#_EXPORT_SETS probably. If you don't see why, have another look at what I said above about it being a different 'unit'.
You didn't give much information, but I guess you are populating that with multiple
list(APPEND MYPROJ_EXPORT_SETS fooExports)
calls, perhaps in some macro. So the order is not easily maintainable, as it would be as a single set() call.
So, your options to express 'export set' dependencies are:
Manage them in the configuration file - replace the loop with a hardcoded ordered list
Add more variables to express dependencies of export sets wherever you populate the MYPROJ_EXPORT_SETS variable, and replace the loop in your configuration file with something more complex that takes those dependencies into account.
Same as (2), but generate intermediate files and don't care about the include order within the configuration file.
(1) probably makes most sense, but you might also have to step back and think harder about the abstractions/wrappers you're creating which led you here.
I am new to programming in general so I decided that I would start by making a simple vector class in C++. However I would like to get in to good habits from the start rather than trying to modify my workflow later on.
I currently have only two files vector3.hpp and vector3.cpp. This project will slowly start to grow (making it much more of a general linear algebra library) as I become more familiar with everything, so I would like to adopt a "standard" project layout to make life easier later on. So after looking around I have found two ways to go about organizing hpp and cpp files, the first being:
project
└── src
├── vector3.hpp
└── vector3.cpp
and the second being:
project
├── inc
│ └── project
│ └── vector3.hpp
└── src
└── vector3.cpp
Which would you recommend and why?
Secondly I would like to use the Google C++ Testing Framework for unit testing my code as it seems fairly easy to use. Do you suggest bundling this with my code, for example in a inc/gtest or contrib/gtest folder? If bundled, do you suggest using the fuse_gtest_files.py script to reduce the number or files, or leaving it as is? If not bundled how is this dependency handled?
When it comes to writing tests, how are these generally organized? I was thinking to have one cpp file for each class (test_vector3.cpp for example) but all compiled in to one binary so that they can all be run together easily?
Since the gtest library is generally build using cmake and make, I was thinking that it would make sense for my project to also be built like this? If I decided to use the following project layout:
├── CMakeLists.txt
├── contrib
│ └── gtest
│ ├── gtest-all.cc
│ └── gtest.h
├── docs
│ └── Doxyfile
├── inc
│ └── project
│ └── vector3.cpp
├── src
│ └── vector3.cpp
└── test
└── test_vector3.cpp
How would the CMakeLists.txt have to look so that it can either build just the library or the library and the tests? Also I have seen quite a few projects that have a build and a bin directory. Does the build happen in the build directory and then the binaries moved out in to the bin directory? Would the binaries for the tests and the library live in the same place? Or would it make more sense to structure it as follows:
test
├── bin
├── build
└── src
└── test_vector3.cpp
I would also like to use doxygen to document my code. Is it possible to get this to automatically run with cmake and make?
Sorry for so many questions, but I have not found a book on C++ that satisfactorily answers these type of questions.
C++ build systems are a bit of a black art and the older the project
the more weird stuff you can find so it is not surprising that a lot
of questions come up. I'll try to walk through the questions one by one and mention some general things regarding building C++ libraries.
Separating headers and cpp files in directories. This is only
essential if you are building a component that is supposed to be used
as a library as opposed to an actual application. Your headers are the
basis for users to interact with what you offer and must be
installed. This means they have to be in a subdirectory (no-one wants
lots of headers ending up in top-level /usr/include/) and your
headers must be able to include themselves with such a setup.
└── prj
├── include
│ └── prj
│ ├── header2.h
│ └── header.h
└── src
└── x.cpp
works well, because include paths work out and you can use easy
globbing for install targets.
Bundling dependencies: I think this largely depends on the ability of
the build system to locate and configure dependencies and how
dependent your code on a single version is. It also depends on how
able your users are and how easy is the dependency to install on their
platform. CMake comes with a find_package script for Google
Test. This makes things a lot easier. I would go with bundling only
when necessary and avoid it otherwise.
How to build: Avoid in-source builds. CMake makes out of source-builds
easy and it makes life a lot easier.
I suppose you also want to use CTest to run tests for your system (it
also comes with build-in support for GTest). An important decision for
directory layout and test organization will be: Do you end up with
subprojects? If so, you need some more work when setting up CMakeLists
and should split your subprojects into subdirectories, each with its
own include and src files. Maybe even their own doxygen runs and
outputs (combining multiple doxygen projects is possible, but not easy
or pretty).
You will end up with something like this:
└── prj
├── CMakeLists.txt <-- (1)
├── include
│ └── prj
│ ├── header2.hpp
│ └── header.hpp
├── src
│ ├── CMakeLists.txt <-- (2)
│ └── x.cpp
└── test
├── CMakeLists.txt <-- (3)
├── data
│ └── testdata.yyy
└── testcase.cpp
where
(1) configures dependencies, platform specifics and output paths
(2) configures the library you are going to build
(3) configures the test executables and test-cases
In case you have sub-components I would suggest adding another hierarchy and use the tree above for each sub-project. Then things get tricky, because you need to decide if sub-components search and configure their dependencies or if you do that in the top-level. This should be decided on a case-by-case basis.
Doxygen: After you managed to go through the configuration dance of
doxygen, it is trivial to use CMake add_custom_command to add a
doc target.
This is how my projects end up and I have seen some very similar projects, but of course this is no cure all.
Addendum At some point you will want to generate a config.hpp
file that contains a version define and maybe a define to some version
control identifier (a Git hash or SVN revision number). CMake has
modules to automate finding that information and to generate
files. You can use CMake's configure_file to replace variables in a
template file with variables defined inside the CMakeLists.txt.
If you are building libraries you will also need an export define to
get the difference between compilers right, e.g. __declspec on MSVC
and visibility attributes on GCC/clang.
As a starter, there are some conventional names for directories that you cannot ignore, these are based on the long tradition with the Unix file system. These are:
trunk
├── bin : for all executables (applications)
├── lib : for all other binaries (static and shared libraries (.so or .dll))
├── include : for all header files
├── src : for source files
└── doc : for documentation
It is probably a good idea to stick to this basic layout, at least at the top-level.
About splitting the header files and source files (cpp), both schemes are fairly common. However, I tend to prefer keeping them together, it is just more practical on day-to-day tasks to have the files together. Also, when all the code is under one top-level folder, i.e., the trunk/src/ folder, you can notice that all the other folders (bin, lib, include, doc, and maybe some test folder) at the top level, in addition to the "build" directory for an out-of-source build, are all folders that contain nothing more than files that are generated in the build process. And thus, only the src folder needs to be backed up, or much better, kept under a version control system / server (like Git or SVN).
And when it comes to installing your header files on the destination system (if you want to eventually distribute your library), well, CMake has a command for installing files (implicitly creates a "install" target, to do "make install") which you can use to put all the headers into the /usr/include/ directory. I just use the following cmake macro for this purpose:
# custom macro to register some headers as target for installation:
# setup_headers("/path/to/header/something.h" "/relative/install/path")
macro(setup_headers HEADER_FILES HEADER_PATH)
foreach(CURRENT_HEADER_FILE ${HEADER_FILES})
install(FILES "${SRCROOT}${CURRENT_HEADER_FILE}" DESTINATION "${INCLUDEROOT}${HEADER_PATH}")
endforeach(CURRENT_HEADER_FILE)
endmacro(setup_headers)
Where SRCROOT is a cmake variable that I set to the src folder, and INCLUDEROOT is cmake variable that I configure to wherever to headers need to go. Of course, there are many other ways to do this, and I'm sure my way is not the best. The point is, there is no reason to split the headers and sources just because only the headers need to be installed on the target system, because it is very easy, especially with CMake (or CPack), to pick out and configure the headers to be installed without having to have them in a separate directory. And this is what I have seen in most libraries.
Quote: Secondly I would like to use the Google C++ Testing Framework for unit testing my code as it seems fairly easy to use. Do you suggest bundling this with my code, for example in a "inc/gtest" or "contrib/gtest" folder? If bundled, do you suggest using the fuse_gtest_files.py script to reduce the number or files, or leaving it as is? If not bundled how is this dependency handled?
Don't bundle dependencies with your library. This is generally a pretty horrible idea, and I always hate it when I'm stuck trying to build a library that did that. It should be your last resort, and beware of the pitfalls. Often, people bundle dependencies with their library either because they target a terrible development environment (e.g., Windows), or because they only support an old (deprecated) version of the library (dependency) in question. The main pitfall is that your bundled dependency might clash with already installed versions of the same library / application (e.g., you bundled gtest, but the person trying to build your library already has a newer (or older) version of gtest already installed, then the two might clash and give that person a very nasty headache). So, as I said, do it at your own risk, and I would say only as a last resort. Asking the people to install a few dependencies before being able to compile your library is a much lesser evil than trying to resolve clashes between your bundled dependencies and existing installations.
Quote: When it comes to writing tests, how are these generally organised? I was thinking to have one cpp file for each class (test_vector3.cpp for example) but all compiled in to one binary so that they can all be run together easily?
One cpp file per class (or small cohesive group of classes and functions) is more usual and practical in my opinion. However, definitely, don't compile them all into one binary just so that "they can all be run together". That's a really bad idea. Generally, when it comes to coding, you want to split things up as much as it is reasonable to do so. In the case of unit-tests, you don't want one binary to run all the tests, because that means that any little change that you make to anything in your library is likely to cause a near total recompilation of that unit-test program, and that's just minutes / hours lost waiting for recompilation. Just stick to a simple scheme: 1 unit = 1 unit-test program. Then, use either a script or a unit-test framework (such as gtest and/or CTest) to run all the test programs and report to failure/success rates.
Quote: Since the gtest library is generally build using cmake and make, I was thinking that it would make sense for my project to also be built like this? If I decided to use the following project layout:
I would rather suggest this layout:
trunk
├── bin
├── lib
│ └── project
│ └── libvector3.so
│ └── libvector3.a products of installation / building
├── docs
│ └── Doxyfile
├── include
│ └── project
│ └── vector3.hpp
│_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
│
├── src
│ └── CMakeLists.txt
│ └── Doxyfile.in
│ └── project part of version-control / source-distribution
│ └── CMakeLists.txt
│ └── vector3.hpp
│ └── vector3.cpp
│ └── test
│ └── test_vector3.cpp
│_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
│
├── build
└── test working directories for building / testing
└── test_vector3
A few things to notice here. First, the sub-directories of your src directory should mirror the sub-directories of your include directory, this is just to keep things intuitive (also, try to keep your sub-directory structure reasonably flat (shallow), because deep nesting of folders is often more of a hassle than anything else). Second, the "include" directory is just an installation directory, its contents are just whatever headers are picked out of the src directory.
Third, the CMake system is intended to be distributed over the source sub-directories, not as one CMakeLists.txt file at the top-level. This keeps things local, and that's good (in the spirit of splitting things up into independent pieces). If you add a new source, a new header, or a new test program, all you need is to edit one small and simple CMakeLists.txt file in the sub-directory in question, without affecting anything else. This also allows you to restructure the directories with ease (CMakeLists are local and contained in the sub-directories being moved). The top-level CMakeLists should contain most of the top-level configurations, such as setting up destination directories, custom commands (or macros), and finding packages installed on the system. The lower-level CMakeLists should contain only simple lists of headers, sources, and unit-test sources, and the cmake commands that register them to compilation targets.
Quote: How would the CMakeLists.txt have to look so that it can either build just the library or the library and the tests?
Basic answer is that CMake allows you to specifically exclude certain targets from "all" (which is what is built when you type "make"), and you can also create specific bundles of targets. I can't do a CMake tutorial here, but it is fairly straight forward to find out by yourself. In this specific case, however, the recommended solution is, of course, to use CTest, which is just an additional set of commands that you can use in the CMakeLists files to register a number of targets (programs) that are marked as unit-tests. So, CMake will put all the tests in a special category of builds, and that is exactly what you asked for, so, problem solved.
Quote: Also I have seen quite a few projects that have a build ad a bin directory. Does the build happen in the build directory and then the binaries moved out in to the bin directory? Would the binaries for the tests and the library live in the same place? Or would it make more sense to structure it as follows:
Having a build directory outside the source ("out-of-source" build) is really the only sane thing to do, it is the de facto standard these days. So, definitely, have a separate "build" directory, outside the source directory, just as the CMake people recommend, and as every programmer I have ever met does. As for the bin directory, well, that is a convention, and it is probably a good idea to stick to it, as I said in the beginning of this post.
Quote: I would also like to use doxygen to document my code. Is it possible to get this to automatically run with cmake and make?
Yes. It is more than possible, it is awesome. Depending on how fancy you want to get, there are several possibilities. CMake does have a module for Doxygen (i.e., find_package(Doxygen)) which allows you to register targets that will run Doxygen on some files. If you want to do more fancy things, like updating the version number in the Doxyfile, or automatically entering a date / author stamps for source files and so on, it is all possible with a bit of CMake kung-fu. Generally, doing this will involve that you keep a source Doxyfile (e.g., the "Doxyfile.in" that I put in the folder layout above) which has tokens to be found and replaced by CMake's parsing commands. In my top-level CMakeLists file, you will find one such piece of CMake kung-fu that does a few fancy things with cmake-doxygen together.
Structuring the project
I would generally favour the following:
├── CMakeLists.txt
|
├── docs/
│ └── Doxyfile
|
├── include/
│ └── project/
│ └── vector3.hpp
|
├── src/
└── project/
└── vector3.cpp
└── test/
└── test_vector3.cpp
This means that you have a very clearly defined set of API files for your library, and the structure means that clients of your library would do
#include "project/vector3.hpp"
rather than the less explicit
#include "vector3.hpp"
I like the structure of the /src tree to match that of the /include tree, but that's personal preference really. However, if your project expands to contain subdirectories within /include/project, it would generally help to match those inside the /src tree.
For the tests, I favour keeping them "close" to the files they test, and if you do end up with subdirectories within /src, it's a pretty easy paradigm for others to follow if they want to find a given file's test code.
Testing
Secondly I would like to use the Google C++ Testing Framework for unit testing my code as it seems fairly easy to use.
Gtest is indeed simple to use and is fairly comprehensive in terms of its capabilities. It can be used alongside gmock very easily to extend its capabilities, but my own experiences with gmock have been less favourable. I'm quite prepared to accept that this may well be down to my own shortcomings, but gmock tests tends to be more difficult to create, and much more fragile / difficult to maintain. A big nail in the gmock coffin is that it really doesn't play nice with smart pointers.
This is a very trivial and subjective answer to a huge question (which probably doesn't really belong on S.O.)
Do you suggest bundling this with my code, for example in a "inc/gtest" or "contrib/gtest" folder? If bundled, do you suggest using the fuse_gtest_files.py script to reduce the number or files, or leaving it as is? If not bundled how is this dependency handled?
I prefer using CMake's ExternalProject_Add module. This avoids you having to keep gtest source code in your repository, or installing it anywhere. It is downloaded and built in your build tree automatically.
See my answer dealing with the specifics here.
When it comes to writing tests, how are these generally organised? I was thinking to have one cpp file for each class (test_vector3.cpp for example) but all compiled in to one binary so that they can all be run together easily?
Good plan.
Building
I'm a fan of CMake, but as with your test-related questions, S.O. is probably not the best place to ask for opinions on such a subjective issue.
How would the CMakeLists.txt have to look so that it can either build just the library or the library and the tests?
add_library(ProjectLibrary <All library sources and headers>)
add_executable(ProjectTest <All test files>)
target_link_libraries(ProjectTest ProjectLibrary)
The library will appear as a target "ProjectLibrary", and the test suite as a target "ProjectTest". By specifying the library as a dependency of the test exe, building the test exe will automatically cause the library to be rebuilt if it is out of date.
Also I have seen quite a few projects that have a build ad a bin directory. Does the build happen in the build directory and then the binaries moved out in to the bin directory? Would the binaries for the tests and the library live in the same place?
CMake recommends "out-of-source" builds, i.e. you create your own build directory outside the project and run CMake from there. This avoids "polluting" your source tree with build files, and is highly desirable if you're using a vcs.
You can specify that the binaries are moved or copied to a different directory once built, or that they are created by default in another directory, but there's generally no need. CMake provides comprehensive ways to install your project if desired, or make it easy for other CMake projects to "find" the relevant files of your project.
With regards to CMake's own support for finding and executing gtest tests, this would largely be inappropriate if you build gtest as part of your project. The FindGtest module is really designed to be used in the case where gtest has been built separately outside of your project.
CMake provides its own test framework (CTest), and ideally, every gtest case would be added as a CTest case.
However, the GTEST_ADD_TESTS macro provided by FindGtest to allow easy addition of gtest cases as individual ctest cases is somewhat lacking in that it doesn't work for gtest's macros other than TEST and TEST_F. Value- or Type-parameterised tests using TEST_P, TYPED_TEST_P, etc. aren't handled at all.
The problem doesn't have an easy solution that I know of. The most robust way to get a list of gtest cases is to execute the test exe with the flag --gtest_list_tests. However, this can only be done once the exe is built, so CMake can't make use of this. Which leaves you with two choices; CMake must try to parse C++ code to deduce the names of the tests (non-trivial in the extreme if you want to take into account all gtest macros, commented-out tests, disabled tests), or test cases are added by hand to the CMakeLists.txt file.
I would also like to use doxygen to document my code. Is it possible to get this to automatically run with cmake and make?
Yes - although I have no experience on this front. CMake provides FindDoxygen for this purpose.
In addition to the other (excellent) answers, I am going to describe a structure I've been using for relatively large-scale projects.
I am not going to address the subquestion about Doxygen, since I would just repeat what is said in the other answers.
Rationale
For modularity and maintainability, the project is organized as a set of small units.
For clarity, let's name them UnitX, with X = A, B, C, ... (but they can have any general name).
The directory structure is then organized to reflect this choice, with the possibility to group units if necessary.
Solution
The basic directory layout is the following (content of units is detailed later on):
project
├── CMakeLists.txt
├── UnitA
├── UnitB
├── GroupA
│ └── CMakeLists.txt
│ └── GroupB
│ └── CMakeLists.txt
│ └── UnitC
│ └── UnitD
│ └── UnitE
project/CMakeLists.txt could contain the following:
cmake_minimum_required(VERSION 3.0.2)
project(project)
enable_testing() # This will be necessary for testing (details below)
add_subdirectory(UnitA)
add_subdirectory(UnitB)
add_subdirectory(GroupA)
and project/GroupA/CMakeLists.txt:
add_subdirectory(GroupB)
add_subdirectory(UnitE)
and project/GroupB/CMakeLists.txt:
add_subdirectory(UnitC)
add_subdirectory(UnitD)
Now to the structure of the different units (let's take, as an example, UnitD)
project/GroupA/GroupB/UnitD
├── README.md
├── CMakeLists.txt
├── lib
│ └── CMakeLists.txt
│ └── UnitD
│ └── ClassA.h
│ └── ClassA.cpp
│ └── ClassB.h
│ └── ClassB.cpp
├── test
│ └── CMakeLists.txt
│ └── ClassATest.cpp
│ └── ClassBTest.cpp
│ └── [main.cpp]
To the different components:
I like having source (.cpp) and headers (.h) in the same folder. This avoids a duplicate directory hierarchy, makes maintenance easier. For installation, it is no problem (especially with CMake) to just filter the header files.
The role of the directory UnitD is to later on allow including files with #include <UnitD/ClassA.h>. Also, when installing this unit, you can just copy the directory structure as is. Note that you can also organize your source files in subdirectories.
I like a README file to summarize what the unit is about and specify useful information about it.
CMakeLists.txt could simply contain:
add_subdirectory(lib)
add_subdirectory(test)
lib/CMakeLists.txt:
project(UnitD)
set(headers
UnitD/ClassA.h
UnitD/ClassB.h
)
set(sources
UnitD/ClassA.cpp
UnitD/ClassB.cpp
)
add_library(${TARGET_NAME} STATIC ${headers} ${sources})
# INSTALL_INTERFACE: folder to which you will install a directory UnitD containing the headers
target_include_directories(UnitD
PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
PUBLIC $<INSTALL_INTERFACE:include/SomeDir>
)
target_link_libraries(UnitD
PUBLIC UnitA
PRIVATE UnitC
)
Here, note that it is not necessary to tell CMake that we want the include directories for UnitA and UnitC, as this was already specified when configuring those units. Also, PUBLIC will tell all targets that depend on UnitD that they should automatically include the UnitA dependency, while UnitC won't be required then (PRIVATE).
test/CMakeLists.txt (see further below if you want to use GTest for it):
project(UnitDTests)
add_executable(UnitDTests
ClassATest.cpp
ClassBTest.cpp
[main.cpp]
)
target_link_libraries(UnitDTests
PUBLIC UnitD
)
add_test(
NAME UnitDTests
COMMAND UnitDTests
)
Using GoogleTest
For Google Test, the easiest is if its source is present in somewhere your source directory, but you don't have to actually add it there yourself.
I've been using this project to download it automatically, and I wrap its usage in a function to make sure that it is downloaded only once, even though we have several test targets.
This CMake function is the following:
function(import_gtest)
include (DownloadProject)
if (NOT TARGET gmock_main)
include(DownloadProject)
download_project(PROJ googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG release-1.8.0
UPDATE_DISCONNECTED 1
)
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE) # Prevent GoogleTest from overriding our compiler/linker options when building with Visual Studio
add_subdirectory(${googletest_SOURCE_DIR} ${googletest_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()
endfunction()
and then, when I want to use it inside one of my test targets, I will add the following lines to the CMakeLists.txt (this is for the example above, test/CMakeLists.txt):
import_gtest()
target_link_libraries(UnitDTests gtest_main gmock_main)