C++ project organisation - c++

I've become a maintainer of a shared library project. The library is split into a few modules, each of them compiled as static library, then linked together. Eclipse is used as IDE, code stored at SVN server. So far the building process was handlet by hand - building libraries, moving all the .a and .h into shared folder, then building the shared library. The code needs to be compiled for linux, ARM and windows.
The problem is that I need to split the current modules a little bit more, for better testing (multiple test and example simple programs, just one .cpp file with main) and inter-module code sharing (both module A and B use C, but I don't want to connect A and B). This results into more complex dependency tree which is going to be difficult to handle by hand. I also need to be able to build more configurations of one project, possibly linking to different version of dependent projects.
How would you organise the code and set up the development environment?
EDIT: the concrete things I need from the DE:
IDE with GUI (I like vim and shell, but the others don't)
Separate projects, each creating static library, set of headers to include and example programs
Different configurations for each project, linking/including different versions and/or configurations of dependencies
Code completion and SVN support

make and Makefiles are the established and very well-thought-out method for such building and linking jobs, especially in combination with automake and libtool. These tools integrate excellently with SVN, and probably also with Eclipse.

So I solved it for now. I created a folder called Pool. Directory tree:
Pool
- inc
- arm
- proj1 public headers directory
- proj2 public headers directory
- proj3 public headers directory
- lin
- proj1 public headers directory
- proj2 public headers directory
- proj3 public headers directory
- win
- proj1 public headers directory
- proj2 public headers directory
- proj3 public headers directory
-lib
- arm
- libproj1.a
- libproj2.a
- libproj3.a
- lin
- libproj1.a
- libproj2.a
- libproj3.a
- win
- libproj1.a
- libproj2.a
- libproj3.a
The libraries are copied here automatically using makefile. Including header:
#include "proj1/someheader.h"
Linking it:
-L${POOL}/lib/arm -lproj1
Note: beware of -l library parameters order.

Related

How to manage multiple build configurations with cmake

I am trying to think through how to design a new project build system.
I want to use CMake to compile for Windows and Linux. I want to have Release and Debug builds, which are pretty straight forward.
The problem is that I will want to deploy this software for multiple hardware projects over the course of the next few years. This means that I will have pre-processor definitions to turn on/off and or alter functionality in my C++ program. I will also want to link configuration specific libraries. A current legacy project uses a header with a list of defines and then links all possible project dependencies.
An example header:
// projects.h
//#define project1
//#define project2
#define project3
Which means that to change hardware/project configurations you have to edit this file to make sure the correct hardware configuration is selected.
What I want to do is to have a configuration for each project which can also be configured for Release or Debug build.
Is there any advice on how to more optimally deal with this in CMake?
for multiple hardware projects
For me it got messy and got messy real quick.
My goals were:
I don't like a single big config.h file. I don't want to have to recompile 100% of files because I added a space in a central config.h file
All boards expose an API that user application can use. Boards don't mix together.
I can easy add new boards and have a board to test the application.
What I discovered is that:
cmake is used to configure your project for different configurations. So cmake does not store a configuration, it is used to choose a configuration.
So you need "external" place to store the configurations for current project
I have multiple applications and multiple boards, the chosen application configuration options are stored inside a makefile. This makefile then configures cmake, which in turn configures the build system, that can then be used to build the application. This is so, because cmake supports one configuration per toolchain. So for architectures with different compilers/compiler options I have to re-run cmake.
I have basically such a directory structure:
- app
- CMakeLists.txt
- main.c
- lib.c
- boards # this is actually git submodule shared between multiple projects
- boardapi # generic board api library
- CMakeLists.txt
- uart_api.h # exposes an api to access uart
- spi_api.h
- timer_api.h
- board_api.h
- some_defines.h # includes some_defines_impl.h
- BOARD1
- toolchain.cmake
- CMakeLists.txt
- implementation1.c
- implementation2.c
- some_defines_impl.h
- BOARD2
- toolchain.cmake
- CMakeLists.txt
- implementation1.c
- implementation2.c
- some_defines_impl.h
- linux
- ... as above ...
- armv5te
- ... as above ...
- CMakeLists.txt
- CMakeLists.txt
- Makefile
I have a boards directory with multiple folders per one board.
Each folder has it's own toolchain.cmake file and CMakeLists.txt file
In each CMakeLists.txt in the board folder a library with the name of the folder is added.
I have a makefile that iterates for all the boards I want to compile this application for and runs cmake + make for each of the boards. Basically BOARDS=BOARD1 BOARD2 and then all: $(foreach board,$(BOARDS),cmake;make;)
cmake is configured with cmake -DCMAKE_TOOLCHAIN_FILE=board/$(BOARD)/toolchain.cmake -DBOARD=$(BOARD)
The board/CMakeLists.txt file all does is add_subdirectory(${BOARD}) and add_library(board INTERFACE ${BOARD}).
The main CMakeLists.txt does add_subdirectory(boards) and after that link_libraries(boards). That should link all libraries with the board library.
Each board directory in turn controls the specific board configuration.
If you have multiple project configurations, just act as if they were normal projects and don't care about board. Example: iterate over them inside app/CMakeLists.txt, like so:
foreach(i IN LISTS project1 project2 project3)
add_executable(main+${i} main.c app.c)
target_add_definitions(${i})
endforeach()
Alternatively, if ex. there are many macros, you could create another structure projects/{project1,project2,project3} and recreate the same structure as for boards, but without the toolchain.cmake file. Then just iterate foreach project and do target_link_libraries(main+${i} ${i}) that will effectively bring all the macro definitions to main application.

preferred cmake project structure

I would like to have the following structure A -> B -> C, where:
C is boilerplate code, wrappers for third-party libraries, very
basic code etc.
B is the common classes, functions and data
structures specific to the project's domain.
A is the project itself.
I would like to make it easy to reuse C or B(+C) in future in my other projects. In addition, I have the following requirements:
As all three projects are in-progress, I would like to have an ability to build C, C+B and C+B+A in one shot.
I would prefer the static linkage over dynamic, so that C and C+B would be static libraries, and C+B+A would be the executable
I would like to keep cmake lists and config files simple and clean. Examples which I found in the official wiki and over the internet are pretty big and monstrous.
It would be great if it won't require changing more than a couple of lines if I'd change the locations of A, B or C in the filesystem.
All these three components are using google-test, but I'm not sure if it is important for the project layout.
I am pretty new to cmake and I don't even understand is it better to write XXXConfig.cmake or FindXXX.cmake files. Also, I am not sure, how should I pass relative paths from subcomponent to the parent component using X_INCLUDE_DIRS.
First I have to admit that I agree with #Tsyvarev. Your CMake environment should fit to your processes/workflow and should take project sizes and team structure into account. Or generally speaking the environment CMake will be used in. And this tends to be - in a positive way - very alive.
So this part of your question is difficult to answer and I'll concentrate on the technical part:
CMake has to know the location of the dependencies - relative or absolute - by
having a monolithic source tree (the one you don't want anymore)
CMake share library with multiple executables
CMake: How to setup Source, Library and CMakeLists.txt dependencies?
a common directory location for includes/libraries/binaries
Custom Directory for CMake Library Output
cmake install not installing libraries on windows
getting the paths via config files/variable definitions
How can I get cmake to find my alternative boost installation?
How to add_custom_command() for the CMake build process itself?
using registration in or installation from a database provided on the host
Making cmake library accessible by other cmake packages automatically
cmake wont run build_command in ExternalProject_Add correctly
To keep your CMake files as simple as possible I would recommend to group your CMake code into separate dedicated files:
Prefer toolchain files over if(SomeCompiler) statements
Move common/repeating code parts as function() bodies into a shared CMake include file
Move complex non-target specific code parts into their own (CMake) script files
Example Code
Since you have specifically asked for the find_package() variant, taking Use CMake-enabled libraries in your CMake project and the things listed above:
MyCommonCode.cmake
cmake_policy(SET CMP0022 NEW)
function(my_export_target _target _include_dir)
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Config.cmake"
"
include(\"\$\{CMAKE_CURRENT_LIST_DIR\}/${_target}Targets.cmake\")
set_property(
TARGET ${_target}
APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES \"${_include_dir}\"
)
"
)
export(
TARGETS ${_target}
FILE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Targets.cmake"
EXPORT_LINK_INTERFACE_LIBRARIES
)
export(PACKAGE ${_target})
endfunction(my_export_target)
C/CMakeLists.txt
include(MyCommonCode.cmake)
...
my_export_target(C "${CMAKE_CURRENT_SOURCE_DIR}/include")
B/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(C REQUIRED)
...
target_link_libraries(B C)
my_export_target(B "${CMAKE_CURRENT_SOURCE_DIR}/include")
A/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(B REQUIRED)
...
target_link_libraries(A B)
This keeps all 3 build environments separate, only sharing the relatively static MyCommonCode.cmake file. So in this approach I have so far not covered your first point, but would recommend the use of a external script to chain/trigger your build steps for A/B/C.

Why does qmake put all object (.o) files to one directory?

Let's say I have a Qt application where I have two classes with the same name in two different namespaces:
namespace namespace1
{
class SomeClass;
}
namespace namespace2
{
class SomeClass;
}
and I have a project directory structure according to it:
-->src/
-->namespace1/
-->someclass.cpp
-->namespace2/
-->someclass.cpp
When I compile the application with qmake, it puts all object (.o) files to one directory - so it creates someclass.o file first and then it rewrites it with the second someclass.o - which is a name collision so it is bad.
Why does qmake not take into account the directory structure of the source files and why does it not create something like namespace1_someclass.o and namespace2_someclass.o?
Yes, I can put my classes to one directory and name them namespace1_someclass.cpp and namespace2_someclass.cpp and there will be no name collisions, but this causes little inconvenience while looking at the source files in the project explorer in Qt Creator because when there are lot of source files in the project, it is much less readable than if there was the directory structure which I can expand or collapse.
One more extreme is to have the directory structure like this:
-->src/
-->namespace1/
-->namespace1_someclass.cpp
-->namespace2/
-->namespace2_someclass.cpp
which solves name collision but it redundantly duplicates the namespace names - and therefore again less readable.
Why does qmake not have at least an option to put the object files to the directory structure according to the source files? Do creators of Qt not see that this is an important feature?
And one more thing - you could recommend me to use cmake tool instead of qmake but I see the use of cmake much much much more difficult than qmake and qmake does its job excellent for me so far - except object files placement.
You can actually put object files alongside source files by using:
CONFIG += object_parallel_to_source
or
CONFIG += object_with_source
depending on your qmake version.
Source: https://wiki.qt.io/Undocumented_QMake#Config_features
Depending on what you are trying to build, you may be able to use the subdirs template in qmake to do this. You'll need to put a project file in each of your namespace directories, and in this you can specify different output directories for your object files.
-->src/main.pro
-->namespace1/n1.pro
-->someclass.cpp
-->namespace2/n2.pro
-->someclass.cpp
main.pro:
TEMPLATE = subdirs
SUBDIRS = namespace1 namespace2
n1.pro and n2.pro:
include("../common.pri")
OBJECTS_DIR = $${PWD}
TARGET = some_target
TEMPLATE = some_qmake_template
common.pri: configurations common to both projects.
Concerning your fears that CMake might be too complicated: I have been working on projects using both build systems. While I agree that qmake is probably easier to begin with, CMake definitely has its merits, too:
It makes out-of-source builds very easy. Just execute cmake <Path to source> in your build directory. This is great when your sources are on an NFS share, for example, and you want the object files to be placed on a local file system.
Its support for finding additional libraries is very powerful. Lots of FindXXX.cmake files are already shipped with your CMake distribution, making the inclusion of "heavy" libraries such as OpenCV as easy as FIND_PACKAGE(OpenCV REQUIRED).
It even has out-of-the-box support for Qt. In fact, I use it for a larger software project where Qt is used for the GUI part. We decided on CMake because we required platform independence and multiple libraries which we could not easily add via qmake.
All in all, use the build system you are comfortable with (as long as your build system does not inhibit your software development).

Autotools: Including a prebuilt 3rd party library

I'm currently working to upgrade a set of c++ binaries that each use their own set of Makefiles to something more modern based off of Autotools. However I can't figure out how to include a third party library (eg. the Oracle Instant Client) into the build/packaging process.
Is this something really simple that I've missed?
Edit to add more detail
My current build environment looks like the following:
/src
/lib
/libfoo
... source and header files
Makefile
/oci #Oracle Instant Client
... header and shared libraries
Makefile
/bin
/bar
... source and header files
Makefile
Makefile
/build
/bin
/lib
build.sh
Today the top level build.sh does the following steps:
Runs each lib's Makefile and copies the output to /build/lib
Runs each binary's Makefile and copied the output to /build/bin
Each Makefile has a set of hardcoded paths to the various sibling directories. Needless to say this has become a nightmare to maintain. I have started testing out autotools but where I am stuck is figuring out the equivalent to copying /src/lib/oci/*.so to /build/lib for compile time linking and bundling into a distribution.
I figured out how to make this happen.
First I switched to a non recursive make.
Next I made the following changes to configure.am as per this page http://www.openismus.com/documents/linux/using_libraries/using_libraries
AC_ARG_WITH([oci-include-path],
[AS_HELP_STRING([--with-oci-include-path],
[location of the oci headers, defaults to lib/oci])],
[OCI_CFLAGS="-$withval"],
[OCI_CFLAGS="-Ilib/oci"])
AC_SUBST([OCI_CFLAGS])
AC_ARG_WITH([oci-lib-path],
[AS_HELP_STRING([--with-oci-lib-path],
[location of the oci libraries, defaults to lib/oci])],
[OCI_LIBS="-L$withval -lclntsh -lnnz11"],
[OCI_LIBS='-L./lib/oci -lclntsh -lnnz11'])
AC_SUBST([OCI_LIBS])
In the Makefile.am you then use the following lines (assuming a binary named foo)
foo_CPPFLAGS = $(OCI_CFLAGS)
foo_LDADD = libnavycommon.la $(OCI_LIBS)
ocidir = $(libdir)
oci_DATA = lib/oci/libclntsh.so.11.1 \
lib/oci/libnnz11.so \
lib/oci/libocci.so.11.1 \
lib/oci/libociicus.so \
lib/oci/libocijdbc11.so
The autotools are not a package management system, and attempting to put that type of functionality in is a bad idea. Rather than incorporating the third party library into your distribution, you should simply have the configure script check for its existence and abort if the required library is not available. The onus is on the user to satisfy the dependency. You can then release a binary package that will allow the user to use the package management system to simplify dependency resolution.

organizing external libraries and include files

Over the years my projects use more and more external libraries, and the way I did it starts feeling more and more awkward (although, that has to be said, it does work flawlessly). I use VS on Windows, CMake on others, and CodeComposer for targetting Digital Signal Processors (DSPs) on Windows. Except for the DSPs, both 32bit and 64bit platforms are used.
Here's a sample of what I am doing now; note that as shown, the different external libraries themselves are not always organized in the same way. Some have different lib/include/src folders, others have a single src folder. Some came ready-to-use with static and/or shared libraries, others were built
/path/to/projects
/projectA
/projectB
/path/to/apis
/apiA
/src
/include
/lib
/apiB
/include
/i386/lib
/amd64/lib
/path/to/otherapis
/apiC
/src
/path/to/sharedlibs
/apiA_x86.lib -->some libs were built in all possible configurations
/apiA_x86d.lib
/apiA_x64.lib
/apiA_x64d.lib
/apiA_static_x86.lib
/apiB.lib -->other libs have just one import library
/path/to/dlls -->most of this directory also gets distributed to clients
/apiA_x86.dll and it's in the PATH
/apiB.dll
Each time I add an external libary, I roughly use this process:
build it, if needed, for different configurations (release/debug/platform)
copy it's static and/or import libraries to 'sharedlibs'
copy it's shared libraries to 'dlls'
add an environment variable, eg 'API_A_DIR' that points to the root for ApiA, like '/path/to/apis/apiA'
create a VS property sheet and a CMake file to state include path and eventually the library name, like include = '$(API_A_DIR)/Include' and lib = apiA.lib
add the propertysheet/cmake file to the project needing the library
It's especially step 4 and 5 that are bothering me. I am pretty sure I am not the only one facing this problem, and would like see how others deal with this.
I was thinking to get rid of the environment variables per library, and use just one 'API_INCLUDE_DIR' and populating it with the include files in an organized way:
/path/to/api/include
/apiA
/apiB
/apiC
This way I do not need the include path in the propertysheets nor the environment variables. For libs that are only used on windows I even don't need a propertysheet at all as I can use #pragmas to instruct the linker what library to link to.
Also in the code it will be more clear what gets included, and no need for wrappers to include files having the same name but are from different libraries:
#include <apiA/header.h>
#include <apiB/header.h>
#include <apiC_version1/header.h>
The withdrawal is off course that I have to copy include files, and possibly** introduce duplicates on the filesystem, but that looks like a minor price to pay, doesn't it?
** actually once libraries are built, the only thing I need from them is the include files and thie libs. Since each of those would have a dedicated directory, the original source tree is not needed anymore so can be deleted..
Why not use file system links?
ln -s /path/to/apis/apiA/include /path/to/api/include/apiA
Voilá. Similar can be done on Windows, but I don't have the command line handy right now.