At the early stage of our development, we have created micro-repositories, to be able to deeply modify repositories while minimizing interference with each other's work.
Now that we have a fairly stable version of our codebase, we want to merge some of the micro-repositories into a monorepo. That will ease the users' experience while building from source, as well as the deployment phase.
All our micro-repositories are built with CMake, and each of them obviously defines a project(). They also have 'internal' dependencies: some of the micro-repositories is built first and then used by the others with find_package().
RepoA
|-- CMakeLists.txt
| project(RepoA)
RepoB
|-- CMakeLists.txt
| project(RepoB)
| find_package(RepoA)
RepoC
|-- CMakeLists.txt
| project(RepoC)
| find_package(RepoA)
| find_package(RepoB)
While migrating them into a monorepo, the first option that came to my mind is moving each of them into a subdirectory, then adding add_subdirectory() to the root CMakeLists.txt file to build them. In the first instance, I would define a global project by using project() in the root CMakeLists.txt file, and I would also keep the project() definition in subdirectories.
Monorepo/
|-- CMakeLists.txt
| project(Monorepo)
| add_subdirectory(RepoA)
| add_subdirectory(RepoB)
| add_subdirectory(RepoC)
|-- RepoA/
|-- CMakeLists.txt
| project(RepoA)
|-- RepoB/
|-- CMakeLists.txt
| project(RepoB)
| find_package(RepoA)
|-- RepoC/
|-- CMakeLists.txt
| project(RepoC)
| find_package(RepoA)
| find_package(RepoB)
Now the question is: is that a good choice? Would you advise having a global project and multiple sub-projects, all of them defined through the use of the CMake project() command? Are there better strategies?
And also, would this strategy have drawbacks?
Many thanks for your kind help!
Related
I have a project that I'm trying to configure to make use of CMake, and to make my life easier I'm organizing it as a series of libraries that I'm treating as subprojects.
MyProject
|-- common-cmake
| |-- common-functions1.cmake
| |-- common-functions2.cmake
| `-- CMakeLists.txt
|-- lib1
| |-- include
| | `-- someClass.h
| |-- src
| | `-- someClass.cpp
| |-- test
| | |-- include
| | | `-- someClassTest.h
| | |-- src
| | | `-- someClassTest.cpp
| | `-- CMakeLists.txt
| `-- CMakeLists.txt
|-- lib2
| `-- <snip>
|-- lib3
| `-- <snip>
`-- CMakeLists.txt
There's a global CMakeLists.txt in MyProject that handles the cross-library specifics (such as version information and installation directories) and ensures that everything is included via
add_subdirectory(cmake-common)
add_subdirectory(lib1)
add_subdirectory(lib2)
add_subdirectory(lib3)
common-cmake contains common logic to avoid repetition in each library's or test's CMakeLists.txt (such as common shared library definition since all of the libraries have a common structure). Each library contains a CMakeLists.txt with the necessary information so that it itself can be compiled, and includes its test directory. Each test in turn has its own CMakeLists.txt to provide the necessary details for the compilation of the tests for the library it is contained within. Thus, both the library and its test CMakeLists.txt is dependent on details the global CMakeLists.txt provides and functions that are contained within the common-cmake.
After fighting with the specifics of this, I've got it working. I can kick-off a build from the global level and build/test/install all libraries as well as for an individual library via the appropriate make target.
Now that I've got this working at the global level, I'm trying to get this working in Eclipse. I've managed to get the global project imported and working by calling
cmake -G "Eclipse CDT4 - Unix Makefiles"
at the global level, however this means that everything is contained within a single Eclipse project and I find it extremely unwieldy to work with. Ideally I'd like to have each of the libraries as its own project within Eclipse. I can import the library as a project from within Eclipse directly, but when I do that Eclipse in turn starts throwing errors relating to failure to CMakeCache.txt not existing, or the common functions from common-cmake not being available (since it's not longer starting from the global CMakeLists.txt, those common files are never added).
The only way around this that I can think of would be to allow each library to be both part of the global project (as it is right now) as well as to be usable independently. I've been trying to figure out how this could be done, but unfortunately I haven't had any success.
add_subdirectory("..")
add_subdirectory("../common-cmake")
add_subdirectory doesn't allow me to add anything that's outside of the current directory's subdirectories (which makes a lot of sense), and the only way I've found is to include each file individually and have a rather ugly flag in place to prevent duplication, where the global and the project check each other's flag to prevent stepping on each other's toes.
# Global CMakeLists.txt
add_subdirectory(cmake-common)
if(NOT DEFINED ${PROJECT_BUILD})
set(FULL_BUILD true)
add_subdirectory(lib1)
add_subdirectory(lib2)
add_subdirectory(lib3)
endif()
# lib CMakeLists.txt
if(NOT DEFINED ${FULL_BUILD})
set(PROJECT_BUILD true)
include("../common-cmake/common-functions1.cmake")
include("../common-cmake/common-functions2.cmake")
endif()
Is there a way to accomplish this? Is what I'm trying to do antithetical to CMake and thus not possible? Is there a better way to achieve my end goal?
I've started a new c++ project , and I am confused with all the CMake capabilities. I have tried to understand better by looking at examples and CMake tutorials
I should create a new project composed of:
Library: It contains some common classes that will be used by the following module(s) (e.g., vector, matrix, image, etc..)
Module (possibly more than 1 in the future): It contains some module-specific classes (e.g., classifier, estimator, etc.) and a main.
My proposed folder structure is as below:
|-- Root Project
|-- CMakeLists.txt
|
|-- Library
| |-- CMakeLists.txt
| |-- include
| | |-- CMakeLists.txt (?)
| | `-- Lib_Class.h
| `-- src
| |-- CMakeLists.txt (?)
| `-- Lib_Class.h
|
|-- Application 1
| |-- CMakeLists.txt
| |-- include
| | |-- CMakeLists.txt (?)
| | `-- Method.h
| `-- src
| |-- CMakeLists.txt (?)
| |-- Method.cpp
| `-- main.cpp
|
|-- Application 2
| |-- CMakeLists.txt
| |
`
The problem arises when I have to actually add the code to the different CMakeLists.txt files. According to my reasoning, I would have:
Root/CMakeLists.txt: For creating the project and adding the subdirectories of the Library and the Module(s).
Library/CMakeLists.txt: This creates the library with the header (from include folder) and source (from src folder) files.
Module/CMakeLists.txt: This creates an executable from the src/main.cpp file using the Library and the module-specific classes with header files in include folder and source files in src folder.
I have 2 questions:
First, I also found solutions in other replies with CMakeLists.txt files in the Library/src and Module/src folders. But I really don't understand how to use them and what to write inside them, because I would have used only the CMakeLists.txt file in the parent folder.
Second, in case I want to link an external library (e.g., OpenCV or dlib) should I link it in the modules and library, individually, or should I link it in the root CMakeLists.txt file (provided that the library is used everywhere)?
I really need some assistance to try to understand CMAKE. Can someone explain or please direct me to a suitable tutorial on this subject.
Matthieu, thank you very much for your help. According to the explanation you provided me, I came out with the following CMakeLists.txt files:
Root/CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
project(Project_Name)
add_subdirectory(Library)
add_subdirectory(Application)
Library/CMakeLists.txt
project(Library)
set(LIB_HEADERS
include/Lib_Class.h
)
set(LIB_SOURCES
src/Lib_Class.cpp
)
add_library(Library_Name SHARED ${LIB_SOURCES} ${LIB_HEADERS})
Application/CMakeLists.txt
project(Application)
set(APP_HEADERS
include/Method.h
)
set(APP_SOURCES
src/Method.cpp
src/main.cpp
)
add_executable(Application_Name ${APP_SOURCES} ${APP_HEADERS})
target_link_libraries(Application_Name Library_Name)
Now everything seems to work grat! Thank you again and sorry again for being confusing somethimes!
The root cmakelists should set up all the variables, checking compiler support and library presence.
Then you go to each subfolder and create the libraries and executables based on the source code and the detected libraries. You should also set up all the linked libraries there.
Then cmake will figure out what depends on what.
I am custom compiling libcurl , libssl and some other library. I don't want to replace system library, because if I am changing it system wise, it is going to create lib conflict and I need to compile all other component depending on these libs.
So I started using RPATH and started structuring like this:
|-- bin
| |-- app.out
|-- lib
| |-- libboost_program_options.so -> libboost_program_options.so.1.49.0
| |-- libboost_program_options.so.1.49.0
| |-- libboost_system.so -> libboost_system.so.1.49.0
| |-- libboost_system.so.1.49.0
| |-- libboost_thread.so -> libboost_thread.so.1.49.0
| |-- libboost_thread.so.1.49.0
| |-- libcares.so -> libcares.so.2.0.0
| |-- libcares.so.2 -> libcares.so.2.0.0
| `-- pkgconfig
`-- sbin
`-- nginx
This approach worked. Now problem is that , We started using PHP and node which require same application version.
|-- bin
| |-- a.out
|-- lib
| |-- libboost_program_options.so -> libboost_program_options.so.1.49.0
| |-- libboost_program_options.so.1.49.0
| |-- libboost_system.so -> libboost_system.so.1.49.0
| |-- libboost_system.so.1.49.0
| |-- libboost_thread.so -> libboost_thread.so.1.49.0
| |-- libboost_thread.so.1.49.0
| |-- libcares.so -> libcares.so.2.0.0
| |-- libcares.so.2 -> libcares.so.2.0.0
| `-- pkgconfig
|-- php_ext
| `-- sqlite3.so
|-- node
| `-- node_modules
| |-- bin
| | |-- node
`-- sbin
`-- nginx
Now , this svn repo is becoming bigger and bigger after every release. Is there a better way to structure this ? without duplicating lib folder in each app ?
As someone who has used both git and svn extensively for years, I'd seriously consider moving to git and using git submodules. Git is hugely more space-efficient (among many, many other benefits). There are also git-svn bridges that you can make if you're stuck using svn at your company.
Failing that, I'd make an svn externals for each group of shared libraries. If you have something that changes often or is logically grouped together, it can go in one svn repo while other libraries that may not change very often.
One of the advantages of git over svn is that git protects you from file corruption. I can painfully remember several occurrences of svn corrupting files (something that wasn't noticed until the client submitted a bug report).
Seriously, save yourself a world of headaches, ditch svn in favour of git.
I was taking a totally different approach when I was working with C++.
I don't treat libs as part of source code. I only want the "dependency" to exist with my source. (It is not appropriate to have "version control" for the lib binary anyway)
I have a separate directory to store all libs in a organized manner, something like libname/version/arch.
In the build script, I am referring to something like $LIB_DIR/libname/version/arch/lib-ver.so.
You can have different way to store/distribute the Lib directory, either put it in a network volume, put that in SVN etc.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
My source file pane is quickly growing (in terms of the number of files in my project) and it is getting a bit cumbersome to quickly locate the specific source file I need to access at any given time. I'm using Embarcadero's C++Builder, but I have encountered this issue in other C++ IDEs as well.
In Java, I often utilize packages for creating logical divisions of my source code, especially when dealing with a large number of source files in a single project. While this, of course, isn't the only purpose of Java packages, they are very handy in this regard.
Does anyone have any ideas on how I can achieve similar functionality in C++? Should I separate my source into physical folders? Does C++Builder offer some kind of virtual folder/grouping functionality that I'm just not seeing? Any ideas are appreciated and thank you.
I generally recommend not to use (only) the IDE or the language syntax for organizing your source code. For one, you tie yourself to the environment: well organized in the IDE, unorganized on file, and then comes the day when you might want to use a different environment...
Because of this, I usually use all three ways of organizing my source at the same time: I separate my source into functional modules, i.e. related classes. Each module gets its own namespace, physical folder, and IDE folder. (In my case, using CMake and source_group() to generate IDE project files if needed -- personally preferring the command line, Vim, and "make".)
Hence, whether I look at the project from within the IDE, from the command line, or from a compiler log, foo/some_class.hpp is foo/some_class.cpp is foo::some_class, minimizing confusion all around.
Actually, my currently preferred setup further subdivides each module directory into <project>/<module>/<class>.hpp or <project>/<module>/src/<class>.hpp depending on whether the class is used outside its own module or not, <project>/<module>/src/<class>.cpp, and <project>/<module>/test/<class>_tu.cpp. Namespace is <project>::<module>::<class>, of course.
project
|-- foo
| |-- some_class.hpp
| |-- src
| | |-- internal_class.hpp
| | |-- internal_class.cpp
| | `-- some_class.cpp
| `-- test
| |-- internal_class_tu.cpp
| `-- some_class_tu.cpp
|-- bar
| |-- ...
The idea here is that the "external" interface of each module (foo) is documented by the headers in that subfolder, with implementation details and tests "hidden" in the respective subfolders.
But in the end, it very much depends -- on your taste, on that of your co-developers, and the scope of your project.
This is how I roll:
PROJECT_NAME
|-- build // This is DVCS ignored but has all the built intermediates and final binaries
| |-- release // These are the different build profiles
| |-- debug
| |-- profile
| `-- coverage
|-- bin // For binary source code
| `-- hello_world
| |-- doc
| |-- inc
| |-- src
| |-- tests
| `-- build_script // Builds binary into the build folder
|-- include // Public headers for the library
| `-- these
| `-- folders
| `-- represent
| `-- namespaces
| `-- my_awesome_class.hpp
|-- lib // library source code
| |-- these
| | `-- folders
| | `-- represent
| | `-- namespaces
| | |-- inc // Private headers
| | | `-- my_private_class.hpp // internal class
| | |-- src // Source code for this namespace
| | | |-- posix
| | | | `-- my_awesome_class.cpp // posix specific source code
| | | |-- nt
| | | | `-- my_awesome_class.cpp // nt specific source code
| | | |-- my_private_class.cpp // non-visibile class
| | | `-- my_awesome_class.cpp // cross platform source code
| | |-- tests // Unit tests
| | | `-- my_awesome_class.cpp // builds a test executable for the library
| | `-- doc // Documentation for this namespace
| | `-- namespace.dox
| `-- build_script // Builds binary into the build folder
|-- doc // Documentation files
| |-- main_page.dox
| `-- namespace.dox
`-- build_script // Builds the source code into the build folder
This represents the these::folders::represent::namespaces::MyAwesomeClass class which has posix and NT specific source code (as well as generic source code) plus there is a private these::folders::represent::namespaces::MyPrivateClass that is used internally in the library, the headers are not public and the visibility of the class symbols are hidden.
This has scaled very well and provides easy locating of files.
I make projects in order to keep all my files easily accesible. It is the easiest way to organize, along with clear class/file names.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
now i need to design/organize the source-code structure and makefiles for the next project. This is a software implemented largely by C++ and supposed to be used normally on Linux. It will include following components:
projecthome/3rd_party_lib_boost/ hdr and src
projecthome/3rd_party_lib_from_vendor/ hdr and src
# libraries that will use 3rd party libs
projecthome/lib_base_struct/ hdr and src
projecthome/lib_utilites/ hdr and src
# applications that will generate binaries, they depend on the above libs, these applications will be developed on after another, while the number of libraries are likely to be fixed.
projecthome/app_1/ hdr and src
projecthome/app_2/ hdr and src
# shell scripts that run the above binaries
projecthome/sh
# python scripts that analyze the logs
projecthome/py
# the configuration files that need to configure the binaries
projecthome/config
# how to build? this is the most difficult party now I need to address
projecthome/build
Now I need a way to organize the above files, and most importantly, the makefiles.
This is the first time that I design such an 'architecture' by myself. So I come here for advices.
I think the most convenient way is to download an open-source project's source and use it as a model. Can any one recommend a mid-size project who has similar structure as above?
oh, my project is not very large, i think it should have 10k-20k lines of c++ code
another thing is that, I hope the above components will not depend too much on each other, because at least one application will be sub-contracted to people outside of the company, I
don't want him/her checkout the whole projecthome directory to compile.
Can anyone give me a clue ? thanks a lot!
Rather than using an open source project as a template, I would define your own structure and build scripts so that you take full control and adapt it to your needs.
For the structure, choose a logical structure that separates generic libraries (to be shared between different applications), third party libraries and applications. For example:
.
|-- app
| |-- app1
| | `-- src
| `-- app2
| `-- src
|-- common_build
|-- lib
| |-- lib1
| | `-- src
| `-- lib2
| `-- src
`-- third_party
`-- boost
For the build tool, have a look at some advanced build tools such as:
Boost Build
Scons
Cmake
... etc
The advantage of these tools is that they handle complexity that you would have to implement yourself with Make.
I have experience with Boost Build. It is used to build the Boost C++ libraries (although note that they are currently experimenting with Cmake). Boost Build automatically handles library dependencies, building of different toolchains, building of different variants (debug, release, static/dynamic linking, etc). It is a very powerful tool.
Here are some proposals regarding the file system layout. We use a similar approach for our own big-sized multi-platform builds.
Generally I would try to group the directory hierarchy by "logical means", not by means of vendor or open-source.
the src directory contains the sources (which was called "build" in your example before). You should also contain some samples there if you are shipping libraries.
regarding 3rd party libs I would not differentiate between vendor and non-vendor 3rd party libs.
the tools directory should should contain your helpers. I wouldn't be to picky if it is for building, or for analyzing only the logfiles.
the tmp_build directory should be used for building only, so that you can quickly remove it in case that a complete re-build is necessary.
projecthome
|
|---_src # your own libs and applications and examples
| |-- config
| |-- app
| | |-- app1
| | `-- app2
| |-- lib
| | |-- base_struct
| | `-- utilites
| `-- samples
| |-- usage1
| `-- usage2
|
|-- 3rd_party # all vendor and non-vendor 3rd party libs
| |-- boost
| |-- vendor1
| `-- vendor2
|
|-- build_tools # all of your tools. Even it is only used for log analyze
| |-- build_helper_1
| |-- build_helper_2
| |-- py
| `-- sh
|
`-- tmp_build # *temporary* build directory. Initially empty
Edit: it might also be a good convention to start the most important directories with an undersore (e.g. _src). This has the advantage that on most OS these directories are listed first (Explorer, or ls command).
Edit: the Makefiles in the "final directories" below the _src directory should all have the same distance to _src. E.g.:
_src/app/app1/Makefile -> distance to _src is 2
_src/app/app2/Makefile -> distance to _src is 2
_src/lib/utilities/Makefile -> distance to _src is 2
_src/lib/base_struct/Makefile -> distance to _src is 2
_src/samples/usage1/Makefile -> distance to _src is 2
_src/samples/usage2/Makefile -> distance to _src is 2
The reason for this "same distance" rule is that it allows you to resolve inter-directory dependencies in a generic way using relative-paths, e.g. with the following Makefile snippet for src/app/app1/Makefile:
PROJECT_HOME?=$(shell cd ../../.. && pwd)
Postfix and Subversion use Makefiles.