In my company, we're developing a middle sized project which we plan to use CMake as the build platform generator. In this scenario, me and my colleagues were discussing about which way the CMake should be used. Our discussions came to a turn which we must decide the methods to use. Our directory structure is similar to this:
<"our project"> \
modules \
module_1 \
tests \
test_example.cpp
mock
some_mock_class.hpp
some_class.hpp
some_class.cpp
...
module_2 \
...
module_3 \
...
utility \
...
1- First thing is first, my colleague thinks that folders like "src" and "includes" are reminder of C programming and has no place in a modern C++ program, so we don't need them. So we removed them from the structure, but being a Linux guy; I'm not sure if this is a good idea. Should we set a "include" directory for headers, so CMake can install them appropriately to the include dirs of the install target; or can CMake handle them appropriately?
2- Should we make a CMakeLists.txt to the root of the project which includes and defines all the targets, or should we make a CMakeLists.txt per module, and then use "add_subdirectory" directives to include them? My colleague thinks on CMakeLists.txt is the best, because this way the module implementors don't need to think about CMake at all, and one or two admins of deployment can maintain the file; but I think every module implementor is more aware of which libraries they use, and how to compile their modules - which he disagrees. What do you suggest in this case?
If you did use CMake for such a middle-sized project before (or know of a case) can you please recommend us what they did and, if possible, why?
Sincerely
The topic is huge, but in short my personal recommendation. For a middle project I assime a component model should be already applied. Then reasonable then is, to have component directories with their onwn CMakeLists.txt which are referenced by the top-level CMakeLists.txt via add_subdirectory(). Each component - a separate library (I like static ones).
For the component folders I find reasonable to hide all internal stuff (aka implementation and private headers, ...) under a private sub-directory to do not be exposed to the outside. Then, in the top component directory you have only headers which are to be used by the others. In the private directory you can mix sources and headers - this is only a matter of the taste for mid projects. And the private directory can also be decomposed if the component is large. But then you need to decide either to add all artifacts to the single CMakeLists.txt of the component, or to have sub-libraries. But in that case the users should link to them individually instead to link to the component's library only.
In the best case, the folder structure should follow the dependencies structure and form a tree-view build system, where the components have as less knowlege about internals of the other components as possible. In that case you will have a good configurability and flexibility in case of possible refactorings. In the other words, the design of the build system seems to me similar to the class design in C++ - same principles.
The real (target) build directory where you run cmake can be located anywhere, normally outside of the source directory. A good place for it could be a RAM disc if you enough memory. Then for the clean build you need just to remove it, that's it. But the source and the build itself have no dependency from its location.
Ah yes, one more hint. My recommendation would be to include headers by the path starting from the component directory like #include "SomeHeader.hpp" which is located as ComponentX/SomeHeader.hpp. Then the CMakelists.txt is used to do the ComponentX directory known to your component. This means, the paths to the headers are not hardcoded in the source files. This brings some limitation like unique file names, but makes changes to the components location much easier.
Hope this anyhow helps.
Related
What would the best way to replace global variables and instead move towards using target properties to control things like building tests in included libraries from the main applications CMakeLists.txt file?
Background
We have a c++ application, A, that uses, amongst others, inhouse libraries: lib B and lib C, that are included as git submodules in A:s repository. All respositories (a, B and C) share a common structure internal structure with all of them using Gtest as a unit test framework. This leads to all projects having a "unit_tests" target as well as a "gtest" target.
As a consequence, when including lib B and lib C in A:s CMakeLists.txt using add_subfolder() there is a conflict, as CMake requires target names to be unique and there is a total of 3 "unit_tests" targets in these 3 repos. There are other test targets as well, but they are so far unique. Renaming the test targets to b_unit_tests and a_unit_test would cure this, but doesnt feel right, and we would also need to rename the gtest targets to a_gtest, b_gtest...
Currently we have solved it by having global CMake variables B_BUILD_TESTS, C_BUILD_TESTS that are set to false in A:s CMakeLists.txt and control the inclusison of test in B and Cs CMakeLists.txt files using add_submodule(unit_tests). This has not been a real problem as we don't want to build and run unit tests for lib B and C when building app A. Doing so would have been mostly a waste of time.
Having watched Daniel Pfeifers ccpnow talk and read other blog posts on using CMake in a declarative way, I've started rewriting our CMake system with modern CMake practices in mind. So now the use of setting a global flag for every included library feels like an anti pattern I would like to avoid. Ideally I feel that tests would be best controlled by setting a property on an included targets. Something like below.
add_subdirectory(B)
set_target_properties(B PROPERTIES BUILD_TESTS false)
target_link_libraries(A PRIVATE B)
Ideally this would still include bs test targets, only make them not depend on b:s main target. But this seems very difficult to do without renaming all of bs targets with a b_ prefix. Technically b_unit_tests is still a different target than b, though from the context of A you would think of it as being part of target B.
Unfortunately I can't get the above to work as is seems like CMake already has a set of pre-defined properties for targets and adding new one doesn't seem fully supported. There is define_property and set_property, but from what I can see you cant use them to define properties on targets.
Suggestion how to rewrite the CMakeLists.txt in a more modern way with clear separation and configuration without resorting to global variables are greatly appreciated.
It seems that for modern CMake, libraries should export their paths and sources and in the export process you can add a namespace. Though I don't really get if this should also apply to libraries that are included as submodules and built together with the application? Is this the route I should go with libs A and
Why don't just rename test targets so that they wouldn't conflict with each other? You can also rule them with single BUILD_TESTS variable and set it to ON or OFF by default depending on how a library is built - as a standalone project, or as part of other one.
I would like to have the following structure A -> B -> C, where:
C is boilerplate code, wrappers for third-party libraries, very
basic code etc.
B is the common classes, functions and data
structures specific to the project's domain.
A is the project itself.
I would like to make it easy to reuse C or B(+C) in future in my other projects. In addition, I have the following requirements:
As all three projects are in-progress, I would like to have an ability to build C, C+B and C+B+A in one shot.
I would prefer the static linkage over dynamic, so that C and C+B would be static libraries, and C+B+A would be the executable
I would like to keep cmake lists and config files simple and clean. Examples which I found in the official wiki and over the internet are pretty big and monstrous.
It would be great if it won't require changing more than a couple of lines if I'd change the locations of A, B or C in the filesystem.
All these three components are using google-test, but I'm not sure if it is important for the project layout.
I am pretty new to cmake and I don't even understand is it better to write XXXConfig.cmake or FindXXX.cmake files. Also, I am not sure, how should I pass relative paths from subcomponent to the parent component using X_INCLUDE_DIRS.
First I have to admit that I agree with #Tsyvarev. Your CMake environment should fit to your processes/workflow and should take project sizes and team structure into account. Or generally speaking the environment CMake will be used in. And this tends to be - in a positive way - very alive.
So this part of your question is difficult to answer and I'll concentrate on the technical part:
CMake has to know the location of the dependencies - relative or absolute - by
having a monolithic source tree (the one you don't want anymore)
CMake share library with multiple executables
CMake: How to setup Source, Library and CMakeLists.txt dependencies?
a common directory location for includes/libraries/binaries
Custom Directory for CMake Library Output
cmake install not installing libraries on windows
getting the paths via config files/variable definitions
How can I get cmake to find my alternative boost installation?
How to add_custom_command() for the CMake build process itself?
using registration in or installation from a database provided on the host
Making cmake library accessible by other cmake packages automatically
cmake wont run build_command in ExternalProject_Add correctly
To keep your CMake files as simple as possible I would recommend to group your CMake code into separate dedicated files:
Prefer toolchain files over if(SomeCompiler) statements
Move common/repeating code parts as function() bodies into a shared CMake include file
Move complex non-target specific code parts into their own (CMake) script files
Example Code
Since you have specifically asked for the find_package() variant, taking Use CMake-enabled libraries in your CMake project and the things listed above:
MyCommonCode.cmake
cmake_policy(SET CMP0022 NEW)
function(my_export_target _target _include_dir)
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Config.cmake"
"
include(\"\$\{CMAKE_CURRENT_LIST_DIR\}/${_target}Targets.cmake\")
set_property(
TARGET ${_target}
APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES \"${_include_dir}\"
)
"
)
export(
TARGETS ${_target}
FILE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Targets.cmake"
EXPORT_LINK_INTERFACE_LIBRARIES
)
export(PACKAGE ${_target})
endfunction(my_export_target)
C/CMakeLists.txt
include(MyCommonCode.cmake)
...
my_export_target(C "${CMAKE_CURRENT_SOURCE_DIR}/include")
B/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(C REQUIRED)
...
target_link_libraries(B C)
my_export_target(B "${CMAKE_CURRENT_SOURCE_DIR}/include")
A/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(B REQUIRED)
...
target_link_libraries(A B)
This keeps all 3 build environments separate, only sharing the relatively static MyCommonCode.cmake file. So in this approach I have so far not covered your first point, but would recommend the use of a external script to chain/trigger your build steps for A/B/C.
In our workflow, we can have a module A that is composed of several header files, module A not producing any binary (side note: it will obviously be used by other modules, that include some of the headers from module A to produce binaries).
A good example would be a header-only library, for which CMake 3 introduces a good support thanks to the notion of INTERFACE library (see this SO answer, and CMake's documentation of the feature).
We can make an interface library target out of module A:
add_library(module_A INTERFACE)
That gives us all the nice features of CMakes targets (it is possible to use it as another target's dependency, to export it, to transitively forward requirements etc.)
But in this case, the headers in module A do not show up in our IDE (Xcode, yet we expect it to be the same with most/every other IDE).
This proves to be a major drawback in the workflow, since we need the files composing module A to be shown in the IDE for edition. Is it possible to achieve that ?
Several months down the line, I did not find a way to directly list the header files for an INTERFACE library.
Since the question still has some views, here is what I ended up doing (i.e. what appears like the lesser hack currently available).
Imagine module A is a header only library. In the CMakeLists.txt declaring its target:
# Define 'modA_headers' variable to list all the header files
set(modA_headers
utility.h
moreUtilities.h
...)
add_library(moduleA INTERFACE) # 'moduleA' is an INTERFACE pseudo target
#
# From here, the target 'moduleA' can be customised
#
target_include_directories(moduleA ...) # Transitively forwarded
install(TARGETS moduleA ...)
#
# HACK: have the files showing in the IDE, under the name 'moduleA_ide'
#
add_custom_target(moduleA_ide SOURCES ${modA_headers})
I do not accept this answer, since I expect further releases of CMake to offer a more semantically correct approach, which will then be accepted : )
You can use the new target_sources command in CMake 3.1.
add_library(moduleA INTERFACE)
target_include_directories(moduleA INTERFACE ...)
target_sources(moduleA INTERFACE
${CMAKE_CURRENT_SOURCE_DIR}/utility.h
${CMAKE_CURRENT_SOURCE_DIR}/moreUtilities.h
)
It is also transitive.
http://www.cmake.org/cmake/help/v3.1/command/target_sources.html#command:target_sources
The limitation of not being able to export targets which have INTERFACE_SOURCES has been lifted for CMake 3.3.
Let's say I have a Qt application where I have two classes with the same name in two different namespaces:
namespace namespace1
{
class SomeClass;
}
namespace namespace2
{
class SomeClass;
}
and I have a project directory structure according to it:
-->src/
-->namespace1/
-->someclass.cpp
-->namespace2/
-->someclass.cpp
When I compile the application with qmake, it puts all object (.o) files to one directory - so it creates someclass.o file first and then it rewrites it with the second someclass.o - which is a name collision so it is bad.
Why does qmake not take into account the directory structure of the source files and why does it not create something like namespace1_someclass.o and namespace2_someclass.o?
Yes, I can put my classes to one directory and name them namespace1_someclass.cpp and namespace2_someclass.cpp and there will be no name collisions, but this causes little inconvenience while looking at the source files in the project explorer in Qt Creator because when there are lot of source files in the project, it is much less readable than if there was the directory structure which I can expand or collapse.
One more extreme is to have the directory structure like this:
-->src/
-->namespace1/
-->namespace1_someclass.cpp
-->namespace2/
-->namespace2_someclass.cpp
which solves name collision but it redundantly duplicates the namespace names - and therefore again less readable.
Why does qmake not have at least an option to put the object files to the directory structure according to the source files? Do creators of Qt not see that this is an important feature?
And one more thing - you could recommend me to use cmake tool instead of qmake but I see the use of cmake much much much more difficult than qmake and qmake does its job excellent for me so far - except object files placement.
You can actually put object files alongside source files by using:
CONFIG += object_parallel_to_source
or
CONFIG += object_with_source
depending on your qmake version.
Source: https://wiki.qt.io/Undocumented_QMake#Config_features
Depending on what you are trying to build, you may be able to use the subdirs template in qmake to do this. You'll need to put a project file in each of your namespace directories, and in this you can specify different output directories for your object files.
-->src/main.pro
-->namespace1/n1.pro
-->someclass.cpp
-->namespace2/n2.pro
-->someclass.cpp
main.pro:
TEMPLATE = subdirs
SUBDIRS = namespace1 namespace2
n1.pro and n2.pro:
include("../common.pri")
OBJECTS_DIR = $${PWD}
TARGET = some_target
TEMPLATE = some_qmake_template
common.pri: configurations common to both projects.
Concerning your fears that CMake might be too complicated: I have been working on projects using both build systems. While I agree that qmake is probably easier to begin with, CMake definitely has its merits, too:
It makes out-of-source builds very easy. Just execute cmake <Path to source> in your build directory. This is great when your sources are on an NFS share, for example, and you want the object files to be placed on a local file system.
Its support for finding additional libraries is very powerful. Lots of FindXXX.cmake files are already shipped with your CMake distribution, making the inclusion of "heavy" libraries such as OpenCV as easy as FIND_PACKAGE(OpenCV REQUIRED).
It even has out-of-the-box support for Qt. In fact, I use it for a larger software project where Qt is used for the GUI part. We decided on CMake because we required platform independence and multiple libraries which we could not easily add via qmake.
All in all, use the build system you are comfortable with (as long as your build system does not inhibit your software development).
I have been struggling with this for quite a while, and my adventures with cmake have only resulted in hackish solutions that I am pretty sure are not correct.
I created a library that consists of several files, as follows:
-libfolder
-codepart1folder
-CMakeLists.txt
-codepart1.cpp
-codepart1.hpp
-codepart2folder
-codepart3folder
-lib.cpp
-lib.hpp
-CMakeLists.txt
I wrote a CMakeLists file to compile the library (after some experimentation), and I can generate a lib.a file. Now I would like to include this code as a library in other projects, and access it through the interface in lib.hpp. What is the best way to do this, in terms of directory structure, and what I need to put into CMakeLists.txt in my root project?
My current attempt has been to add -libfolder as a subfolder to my current project, and add the commands:
include_directories(${PROJECT_SOURCE_DIR}/libfolder)
link_directories(${PROJECT_BINARY_DIR}/libfolder)
add_subdirectory(libfolder)
target_link_libraries(project lib)
When I run make, the library compiles fine, but when project.cpp compiles, it complains that it cannot find codepart1.hpp (which is included in lib.hpp, included from project.cpp).
I suspect that this is the wrong way about doing this, but I cannot wade through the CMake documentation and find a good tutorial on setting up projects like this. Please help, CMake gurus!
The clean way to import one CMake project into another is via the find_package command. The package declaration is done by using the export command. An advantage of using find_package is that it eliminates the need to hard-code paths to the package's files.
Regarding the missing hpp file, you didn't include codepart1folder, so it's not on the include path.
Ok, so after consulting a coworker of mine who is a CMake guru, it seems CMake does not have support for what I am trying to do, leaving one with 3 options:
Add all of the dependencies to the parent projects CMakeLists.txt - not very clean, but it will get the thing to work. You'll have to do this for every project you add the code to, and go back and fix things if your library changes.
clean up your library headers. This is done through some compiler hackery. The idea is to forward-declare every class, and use only pointers or boost::shared_ptr, and then include the dependencies only in the cpp file. That way you can build the cpp file using all the findpackage stuff, and you get the bonus of being able to use the lib by only including the header and linking to the library.
Look into build systems. Having portable code and fast code compilation with complex dependencies is not a solved problem! From my investigations it turned out to be quite complicated. I ended up adopting my coworkers build system which he created himself in cmake, using things he picked up from Google.
Looking at your post you don't seem to add 'codepart1folder' to the includes anywhere. How are you including codepart1.hpp as:
#include <codepart1.hpp>
#include "codepart1folder/codepart1.hpp"
I don't think there is a standard accepted way to structure cmake projects. I've looked at a bunch of cmake repos and they tend to have differences. Personally I do the following:
-project
CMakeLists.txt
-build
-cmake
OptionalCmakeModule.cmake
-src
-Main
Main.cpp
Main.hpp
-DataStructs
SomeTree.hpp
SomeObject.hpp
-Debug
Debug.hpp
-UI
Window.hpp
Window.cpp
Basically that dumps all the source code into 1 directory, then you perform an out of source build with: 'mkdir build && cd build && cmake .. && make' in the projects root folder.
If you have separate libs as part of your project, then you might want a separate libs directory with another subfolder for your specific lib.
I have some of my repos on: https://github.com/dcbishop/ if you want to look at the CMakeLists.txt files.
The main problems with my project structure are that I use the FILE_GLOB which is apparently the 'wrong' way to do things (if you add files after running 'cmake ..' then they won't be picked up hen you do a 'make'). I haven't figured out what the 'right' way to do it is (from what I can see it involves keeping a separate list of files) I also only use 1 CMakeLists.txt file.
Some projects also choose to separate their cpp and hpp files into separate directories. So you would have an include and src folders (at least for the hpp files that are intended to be used externally). I think that would mainly be for projects that are mainly large libraries. Would also make installing header files much easier.
You are probably missing
include_directories(${PROJECT_SOURCE_DIR}/libfolder/codepart1folder)
In such a case you might want to set( CMAKE_INCLUDE_CURRENT_DIR on) to add all folders to the include directory path variable.
Check cmake's output on the command line whether the correct include folders are set or not. Additionally you can always use message() as "print debugging" for cmake variables.
In case of include directories however you need to read the directory property to see what is actually in the include directories.
get_property(inc_dirs DIRECTORY PROPERTY INCLUDE_DIRECTORIES)
message("inc_dirs = ${inc_dirs}")
I hope this helps you figuring out what is missing.
Edit
I just saw your comment about added codepart1folder in the libfolder. It is only available in the libfolder's include_directory path and not propagated to the root folder.
Since the include codepart1.hpp is present in the lib.hpp however you need to have it also available in the project path otherwise you will get missing declaration errors when you build your project.