In my team we made some applications using C++ as main language and when a new project arrives we always end in copy-pasting other project's files when needed . It's frequent and we had a discuss to make an improvement.
So, in order to change that we decided to make an unique library (or many little libraries) that contains everything that is not of the business itself. And we decided to use cmake for that.
But, my question is if is there a way to import this library or these little libraries without compiling them everytime we commit a change.
For example if we have two libraries and two projects, where:
Project A depends on -> library A and Library B
Project B depends on -> library B only
Having our source directory like this:
LIB A
include
src
CMakeLists.txt
LIB B
include
src
CMakeLists.txt
Project A
include
src
CMakeLists.txt
Project B
include
src
CMakeLists.txt
How can we set CmakeLists in project A and B so, when we change something in Library A or B, and re-run cmake and then make in project B for example, all of changes apper in it. And the same for the other project ?.
Is it possible ?
I had the same issue in the past while working on some personal project.
I can offer some suggestions, some of which use your approach towards solving the problem, and some of which don't:
Method 1 (A different approach, multiple source control repositories)
Don't split the code using different CMake files.
Instead, use a single CMake file and split the codebase into smaller repositories.
For example, all the shared utility libraries altogether could've been a single repository, while application A and B, get a repository each.
(You could of course split the utility libraries into multiple repositories as well).
This makes sure that you don't have to hold/update/work with all the projects at once, but only those you actually need. The only downside, is that you have a constrain on the way you checkout these projects, but I don't think it's an issue.
Method 2 (Same approach, using CMake's add_dependencies)
You could define dependencies on the compilation of applications A and B so that the relevant libraries are automatically built if they were updated.
Here is a link to CMake's add_dependencies manual.
Related
clarification: If the below setup is a reasaonable way to do it, then my question is mainly, how do I best add the public headers with the .lib when I run ninja install as my solution does not do this currently.
There is a myriad of examples of how to do stuff in different ways, and many seem to contradict eachother to just confuse me even more.
I'm trying to change out a in-house build system and replace it with CMake. The codebase consists of many different components/libraries, like OS/Platform abstraction, networking, functionality etc. Before this was always compiled together as one big codebase, so running clean would remove everything and rebuild all dependencies.
Now it comes to the part where I have trouble understanding on how a modern smart way to do it in CMake is.
Ignoring the fact that a dependency-chain tool on top is needed, my idea was to build the components/libraries in order of libraries with no dependency to the topmost as isolated build runs.
Then each isolated build run would be built with its dependencies, like only the windows API for the OS abstraction library, and then run install and export on it.
Is this the general prefered way, to compile a library to a .lib on windows or .a on linux and export it to a known shared location?
Im guessing it would be required then to copy all public header files for this library to the same location? If so, how is this most correctly done?
If I have my OS abstraction folder structure
OSabstraction/
includes/
OSabstraction/
abstraction.h
src/
abstraction.cpp
CMakeLists.txt
and my CMakeLists is
cmake_minimum_required(VERSION 3.12)
project(OSabstraction LANGUAGES CXX)
add_library(abstraction)
target_sources(abstraction
PRIVATE
src/abstraction.cpp
)
target_include_directories(abstraction
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/includes/>
$<INSTALL_INTERFACE:include>
)
install(
TARGETS abstraction
EXPORT abstraction-export
ARCHIVE DESTINATION lib
)
How would I best go around to use this library in many other inhouse projects? This library is in practice many times larger, so what would be the correct way to do it for a large project?
edit: I tried to follow Pablo Arias' guide on how to do CMake Right, but I am not sure how the build_interface and install_interface generators work in regards to getting my header files with the export of the .lib file.
Recommended solution is to use a package manager together with CMake.
The link below is a full C++ project setup template which can use the package manager https://conan.io/ to support splitting up dependencies into individual libraries.
https://github.com/trondhe/cmake_template
I want to know what is the best way to make directory structure for multiple dependent c++ projects which is built using cmake and how cmake script should look like in this case. (I do it in linux with gcc if it is important.)
I have multiple independent projects A, B. Also I have multiple libraries X, Y.
There are no problem when those libraries is made by other developers. I can use existing FindX.cmake or write own to track them.
The problem arises when X, Y are my own libraries. Let A depends on X and Y, project B depends on Y. Furthermore, let Y depends on X (because I reuse some common, not task related code). However I assume that there can be more libraries, projects, and greater depth of their nesting. The second assumption is that libraries (X for example) can provide not only libraries like *.so, but also can provide some executables (not only for testing).
When I rebuild my project I want to rebuild all dependencies, because I can make modifications in projects and libraries simultaneously. Moreover, I want to separate libraries from each other and create one directory code with all my projects, which will contain 4 subdirs: ~/code/A, ~/code/B, ~/code/X, ~/code/Y. Every project will have own tests and will be pushed in separate code repositories.
The best links I found about building proper directory structure is this and this. However authors suggest to create nested directory structure, which seems difficult to do in my example, because if I do this way, I will end up with directory structure like this:
A/X
A/Y/X
B/Y/X
which I can get by using symlinks. But then in my separate directory with compiled binaries for each project (A/build, B/build, X/build, Y/build) I will have copy of libraries and executables from different libraries. If I call make install for all builds, those copies will create mess due to duplication of binaries and libraries.
UPDATE (SOLUTION):
The answer happens to be rather simple, I found it after few days of experimenting with cmake and reading documentation.
Libray. Add next two lines:
export(PACKAGE <pkgName>)
export(TARGETS <libTarget> FILE <pkgName>Config.cmake)
First line will create entry in ~/.cmake/packages/ which will tell where build dir of library is situated. Cmake invocations in other projects start search from that path.
Second line will create Find<pkgName>.cmake in build directory.
Executable. Add next lines:
find_package(<pkgName> QUIET)
if(${<pkgName>_FOUND})
message(STATUS "Found <pkgName>.")
else(${<pkgName>_FOUND})
message(FATAL_ERROR "Could not locate <pkgName>!")
endif(${<pkgName>_FOUND})
So, if library wasn't built we will see error message, otherwise we will be able to use target <libTarget>.
Of course, building executable don't trigger rebuilding library if some changes in library sources are made. To overcome this problem I wrote bash script to call make firstly in library, and next in executable build dir.
P. S. If you know better solution, I am still waiting for it.
What I personally use is something like this
~/my_project
~/my_project/src
~/my_project/src/A
~/my_project/src/B
~/my_project/src/X
~/my_project/src/Y
The main CMakeLists.txt uses add_subdirectory to add the src directory. The src/CMakeLists.txt in turn uses add_subdirectory to add A, B, X and Y.
With properly setup dependencies in the CMakeLists.txt files, CMake will handle the dependencies correctly, and rebuild everything needed (so if you make a change in X then Y, A and B should be rebuilt correctly).
I am trying to use gnuplot++, but this is really a more general question about downloaded source code. I have downloaded the gnuplot++ source code and it consists of multiple .h and .cc files. I would like to use it in othercopy projects in the future so I am reluctant to add all the files into my project directory.
From what I understand gcc will look in /usr/local/include for header files, so I have put the code there for now. But what is the best way to compile and link the code?
Should I use the makefile to include the directory of the source code?
Should I keep it somewhere easy to find like /usr/local/include?
How do I know the best way to compile the code in gnuplot++?
Typically, if the project itself doesn't come with install instructions, I usually add it somewhere "public", e.g. /usr/local/project/{lib,include,src,...} where "project" in this case would be gnuplot++.
In this case, there doesn't appear to be any support for building this into a library, which makes it a little more awkward, as you need the sources included in your project itself. I'd still keep those sources separate, but you may prefer to just put them into a separate directory within the project [or spend an hour or three making a library of it].
For general practice, yes, keep the source for gnuplot++ (or any other similar 3rd-party project) separate from your own application source code. This makes it much easier to manage updates to the 3rd party projects, etc.
Yes, I would use the makefile for your application to also include the path to the headers for gnuplot++ and I would not copy those files directly into /usr/local/include. Instead, I would consider a couple options: do nothing and point your include path in your makefile to the gnuplot++ directory, or put symbolic links in /usr/local/include to point to the gnuplot++ files.
As for the best way to compile gnuplot++, I would have to look at gnuplot++ myself and see what it has to say, perhaps in a README file or similar.
In general, when using third-party libraries, you build and install those libraries according to the installation description that comes with the downloaded source.
If there is no installation guideline, it is typically a set of steps like
./configure
make
make install
Then it is the responsibility of the library to ensure the relevant headers and library files are easily locatable for use in your project.
gnuplot++ is an exception here, because it does not seem to come with its own build structure.
The best advice in cases such as this is to put the source from gnuplot++ in a directory within your project (possibly parallel to your own sources) and include the files in your own build setup.
Imagine an overall project with several components:
basic
io
web
app-a
app-b
app-c
Now, let's say web depends on io which depends on basic, and all those things are in one repo and have a CMakeLists.txt to build them as shared libraries.
How should I set things up so that I can build the three apps, if each of them is optional and may not be present at build time?
One idea is to have an empty "apps" directory in the main repo and we can clone whichever app repos we want into that. Our main CMakeLists.txt file can use GLOB to find all the app directories and build them (not knowing in advance how many there will be). Issues with this approach include:
Apparently CMake doesn't re-glob when you just say make, so if you add a new app you must run cmake again.
It imposes a specific structure on the person doing the build.
It's not obvious how one could make two clones of a single app and build them both separately against the same library build.
The general concept is like a traditional recursive CMake project, but where the lower-level modules don't necessarily know in advance which higher-level ones will be using them. Yet, I don't want to require the user to install the lower-level libraries in a fixed location (e.g. /usr/local/lib). I do however want a single invocation of make to notice changed dependencies across the entire project, so that if I'm building an app but have changed one of the low-level libraries, everything will recompile appropriately.
My first thought was to use the CMake import/export target feature.
Have a CMakeLists.txt for basic, io and web and one CMakeLists.txt that references those. You could then use the CMake export feature to export those targets and the application projects could then import the CMake targets.
When you build the library project first the application projects should be able to find the compiled libraries automatically (without the libraries having to be installed to /usr/local/lib) otherwise one can always set up the proper CMake variable to indicate the correct directory.
When doing it this way a make in the application project won't do a make in the library project, you would have to take care of this yourself.
Have multiple CMakeLists.txt.
Many open-source projects take this appraoch (LibOpenJPEG, LibPNG, poppler &etc). Take a look at their CMakeLists.txt to find out how they've done this.
Basically allowing you to just toggle features as required.
I see two additional approaches. One is to simply have basic, io, and web be submodules of each app. Yes, there is duplication of code and wasted disk space, but it is very simple to implement and guarantees that different compiler settings for each app will not interfere with each other across the shared libraries. I suppose this makes the libraries not be shared anymore, but maybe that doesn't need to be a big deal in 2011. RAM and disk have gotten cheaper, but engineering time has not, and sharing of source is arguably more portable than sharing of binaries.
Another approach is to have the layout specified in the question, and have CMakeLists.txt files in each subdirectory. The CMakeLists.txt files in basic, io, and web generate standalone shared libraries. The CMakeLists.txt files in each app directory pull in each shared library with the add_subdirectory() command. You could then pull down all the library directories and whichever app(s) you wanted and initiate the build from within each app directory.
You can use ADD_SUBDIRECTORY for this!
https://cmake.org/cmake/help/v3.11/command/add_subdirectory.html
I ended up doing what I outlined in my question, which is to check in an empty directory (containing a .gitignore file which ignores everything) and tell CMake to GLOB any directories (which are put in there by the user). Then I can just say cmake myrootdir and it does find all the various components. This works more or less OK. It does have some side drawbacks though, such as that some third-party tools like BuildBot expect a more traditional project structure which makes integrating other tools with this sort of arrangement a little more work.
The CMake BASIS tool provides utilities where you can create independent modules of a project and selectively enable and disable them using the ccmake command.
Full disclosure: I'm a developer for the project.
Let's say you have several bespoke C++ projects in separate repositories or top-level directories in the same repository. Maybe 10 are library projects for stuff like graphics, database, maths, etc and 2 are actual applications using those libraries.
What's the best way to organise those 2 application projects to have the .libs they need?
Each lib project builds the .lib in its own directory, developers have to copy these across to the application area manually and make sure to get the right version
Application projects expect lib projects to be in particular paths and look for .libs inside those locations
A common /libs directory is used by all projects
Something else
This is focused on C++, but I think it's pretty similar with other languages, for instance organising JARs in a Java project.
I'd suggest this approach:
Organise your code in a root folder. Let's call it code.
Now put your projects and libraries as subfolders (e.g. Projects and Libraries).
Build your libraries as normal and add a post-build step that copies the resulting headers and .lib files into a set of shared folders. For example, Libraries\include and Libraries\lib. It's a good idea to use subfolders or a naming convention (myLib.lib, myLib_d.lib) to differentiate different builds (e.g. debug and release) so that any lib reference explicitly targets a single file that can never be mixed up. It sucks when you accidentally link against the wrong variant of a lib!
You can also copy third-party libraries that you use into these folders as well.
Note: To keep them organised, include your files with #include "Math\Utils.h" rather than just "Utils.h". And put the headers for the whole Math library into include\Math, rather than dropping them all in the root of the include folder. This way you can have many libraries without name clashes. It also lets you have different versions of libraries (e.g. Photoshop 7, Photoshop 8) which allows you to multi-target your code at different runtime environments.
Then set up your projects to reference the libraries in one of two ways:
1) Tell your IDE/compiler where the libs are using its global lib/include paths. This means you set up the IDE once on each PC and never have to specify where the libs are for any projects.
2) Or, set each project to reference the libs with its own lib/include paths. This gives you more flexibility and avoids the need to set up every PC, but means you have to set the same paths in every new project.
(Which is best depends on the number of projects versus the number of developer PCs)
And the most important part: When you reference the includes/libs, use relative paths. e.g. from Projects\WebApp\WebApp.proj, use "..\..\Libraries\include" rather than "C:\Code\Libraries\Include". This will allow other developers and your buildserver to have the source code elsewhere (D:\MyWork instead of C:\Code) for convenience. If you don't do this, it'll bite you one day when you find a developer without enough disk space on C:\ or if you want to branch your source control.