I am wondering how I should manage a growing C++ project. Now, I am developing a project with Netbeans and it's dirty work generating makefiles. The project has become too big and I have decided to split it up into a few parts. What is the best way of doing this?
I am trying to use Scons as my build system. I have had some success with it, but should I edit the build scripts every time I append or delete files. It's too dull.
So I need your advice.
P.S. By the way, how does a large project like google chrome do this? Does everybody use some kind of IDE to build scripts generated only for software distribution?
I also use Netbeans for C++ and compile with SCons. I use the jVi Netbeans plugin which really works well.
For some reason the Netbeans Python plugin is no longer official, which I dont understand at all. You can still get it though, and it really makes editing the SCons build scripts a nice experience. Even though Netbeans doesnt have a SCons plugin (yet?) you can still configure its build command to execute SCons.
As for maintaining the SCons scripts automatically by the IDE, I dont do that either, I do that by hand. But its not like I have to deal with this on a daily basis, so I dont see that its that important, especially considering how easy to read the scripts are.
Here's the build script in SCons that does the same as mentioned previously for CMake:
env = Environment()
env.EnsurePythonVersion(2, 5)
env.EnsureSConsVersion(2, 1)
libTarget = env.SharedLibrary(target = 'foo', source = ['a.cpp', 'b.cpp', 'c.pp'])
env.Program(target = 'bar', source = ['bar.cpp', libTarget])
The SCons Glob() function is a nice option, but I tend to shy away from automatically building all the files in a directory. The same goes for listing sub-directories to be built. Ive been burned enough times by this, and prefer explicitly specifying the file/dirs to be built.
In case you hear those rumors that SCons is slower than other alternatives, the SCons GoFastButton has some pointers that can help out.
Most large projects stick with a build system that automatically handles all the messy details for them. I'm a huge fan of CMake (which is what KDE uses for all their components) but scons is another popular choice. My editor (KDevelop) supposedly handles CMake projects itself, but I still edit the build scripts myself because it's not that hard.
I'd recommend learning one tool really well and sticking with it (plenty of documentation is available for any tool you'll be interested in). Be sure you also look into version control if you haven't already (I have a soft spot for git, but Mercurial and Subversion are also very popular choices).
A simple CMake example:
project("My Awesome Project" CXX)
cmake_minimum_required(VERSION 2.8)
add_library(foo SHARED a.cpp b.cpp c.cpp) #we'll build an so file
add_executable(bar bar.cpp)
target_link_libraries(bar foo) #link bar to foo
This is obviously a trivial case, but it's very easy to manage and expand as needed.
I am trying to use Scons as build system. I have some success with it, but I should edit
build scripts every time I append or delete file. It's too dull.
Depending on how your files are organized, you can use, for example, Scon's Glob() function to get source files as a list without having to list all files individually. For example, to build all c++ source files into an executable, you can do:
Program('program', Glob('*.cpp'))
You can do the same in CMake using its commands.
And, if you're using SCons, since it's Python you can write arbitrary Python code to make your source file lists.
You can also organize files into multiple folders and have subsidiary SCons (or CMakeList.txt) build files that the master build script can call.
Related
First, i know that make is used for building the code. But which code?
But what does it mean by building a code, and after executing the make command, what is presented to the user?
Second, how is it different from make build_for_e2e?
What Wikipedia tells about make
Make is a build automation tool that automatically builds executable programs and libraries from source code
Compilation process becomes big and complex in big projects, where numbers of files need to be compiled, with flags and libraries. Where it will become hard for people to compile it one by one. So these types of tools were introduced, there are more similar tools available for same use like cmake, gradle, maven. e2e's Build is also a form of build process, with different form of specifications.
For C people mostly use make. It is helpful for porting software packages in different systems.
How make is used:
As said make is a tool, which will be available in our system, we can execute it by giving command make in the directory which needs to be compiled. Then make looks for Makefile, which is provided in the package directory and it contains information about compilation of the project. Then make as per info gathered from Makefile, it compiles the package.
You can also create Makefile for your project, so that it can be also supported and compiled with make. Simple tutorial for it can be found here. For big projects you can use gnu autotools contains autoconf and automake which will help you to create your all files required by make automatically. You can find tutorial regarding it here and here . These contains some basic information, you can find some advance tutorial regarding autotools, use google for more information on it.
I'd like to create a Makefile for a moderately sized C++ project (~110 .cpp and .h files) that was created in Xcode. I don't own a mac, so I can't just go into xcode and export a Makefile.
From my understanding of using Makefiles for simpler projects, I could go through and create a target in my Makefile for each .cpp file, including its prerequisites... but that would require manually looking through each file and sorting out what its dependencies are.
It's my understanding that autoconf is a good tool for this sort of problem, but I haven't been able to figure out how to set it up and I'm not even sure that it's the right tool for the job. So before I go even further down this rabbit hole --
What is the best way to create a Makefile for an existing project without one?
It is rather subjective as to the best tool. Perhaps look at SCONS or gradle to compare for your particular needs.
I think Pbxbuild may help you.
Pbxbuild (not to be confused with the OS X tool of the same name) is used to take Xcode project files and build the code. It does this by creating GNUmakefiles, and using Make to build those.
Any build system (generator) will require some learning on your part. I would suggest using CMake, if you do not mind that CMake will be required wherever anybody tries to build your source. Otherwise I would suggest using the Autoconf/Automake/Libtool combo.
I need some pointers/advice on how to automatically generate CMakeLists.txt files for CMake. Does anyone know of any existing generators? I've checked the ones listed in the CMake Wiki but unfortunately they are not suitable for me.
I already have a basic Python script which traverses my project's directory structure and generates the required files but it's really "dumb" right now. I would like to augment it to take into account for example the different platforms I'm building for, the compiler\cross-compiler I'm using or different versions of the libraries dependencies I might have. I don't have much\expert experience with CMake and an example I could base my work or an already working generator could be of great help.
I am of the opinion that you need not use an automated script for generating CMakeLists.Txt as it is a very simple task to write one, after you have understood the basic procedure. Yeah I do agree that understanding the procedure to write one as given in CMake Wiki is also difficult as it is too much detailed.
A very basic example showing how to write CMakeLists.txt is shown here, which I think will be of use to everyone, even someone who is going to write CMakeLists.txt for the first time.
Well i dont have much of an experience in Cmake either, but to perform a cross platform make a lot of files need to be written and modified including the CMakeLists.txt file, i suggest that you use this new tool called the ProjectGenerator Tool, its pretty cool, it does all the extra work needed and makes it easy to generate such files for 3'rd party sources with little effort.
Just read the README carefully before using it.
Link:
http://www.ogre3d.org/forums/viewtopic.php?f=1&t=54842
I think that you are doing this upside down.
When using CMake, you are supposed to write the CMakeLists.txt yourself. Typically, you don't need to handle different compilers as CMake has knowledge about them. However, if you must, you can add code in the CMakeFiles to do different things depending on the tool you are using.
CLion is an Integrated development environment that is fully based on CMake project file.
It is able to generate itself the CMakeLists.txt file when using the import project from source
However this is quite probable that you have to edit this file manually as your project grows and for adding external dependency.
I'm maintaining a C++ software environment that has more than 1000 modules (shared, static libraries, programs) and uses more than 20 third parties (boost, openCV, Qt, Qwt...). This software environment hosts many programs (~50), each one picking up some libraries, programs and third parties. I use CMake to generate the makefiles and that's really great.
However, if you write your CMakeLists.txt as it is recommended to do (declare the module as being a library/program, importing source files, adding dependencies...). I agree with celavek: maintaining those CMakeLists.txt files is a real pain:
When you add a new file to a module, you need to update its CMakeLists.txt
When you upgrade a third party, you need to update the CMakeLists.txt of all modules using it
When you add a new dependency (library A now needs library B), you may need to update the CMakeLists.txt of all programs using A
When you want a new global settings to be changed (compiler setting, predefined variable, C++ standard used), you need to update all your CMakeLists.txt
Then, I see two strategies to adress those issues and likely the one mentioned by OP.
1- Have CMakeLists.txt be well written and be smart enough not to have a frozen behaviourto update themselves on the fly. That's what we have in our software environment. Each module has a standardized file organization (sources are in src folder, includes are in inc folder...) and have simple text files to specify their dependencies (with keywords we defined, like QT to say the module needs to link with Qt). Then, our CMakeLists.txt is a two-line file and simply calls a cmake macro we wrote to automatically setup the module. As a MCVE that would be:
CMakeLists.txt:
include( utl.cmake )
add_module( "mylib", lib )
utl.cmake:
macro( add_module name what )
file(GLOB_RECURSE source_files "${CMAKE_CURRENT_SOURCE_DIR}/src/*.cpp")
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/inc)
if ( what STREQUEL "lib" )
add_library( ${name} SHARED ${source_files} )
elseif ( what STREQUEL "prg" )
add_executable( ${name} ${source_files} )
endif()
# TODO: Parse the simple texts files to add target_link_libraries accordingly
endmacro()
Then, for all situations exposed above, you simply need to update utl.cmake, not the thousand of CMakeLists.txt you have...
Honestly, we are very happy with this approach, the system becomes very easy to maintain and we can easily add new dependencies, upgrade third parties, change some build/dependency strategies...
However, there remains a lot of CMake scripts to be written. And CMake script language sucks...the tool's very powerful, right, but the script's variable scope, the cache, the painful and not so well documented syntax (just to check if a list is empty you must ask for it's size and store this in a variable!), the fact it's not object oriented...make it a real pain to maintain.
So, I'm now convinced the real good approach may be to:
2- completly generate the CMakeLists.txt from a more powerful language like Python. The Python script would do things similar to what our utl.cmake does, instead it would generate a CMakeLists.txt ready to be passed CMake tool (with a format as proposed in HelloWorld, no variable, no function....it would only call standard CMake function).
I doubt such generic tool exists, because it's hard to produce the CMakeLists.txt files that will make everyone happy, you'll have to write it yourself. Note that gen-cmake does that (generates a CMakeLists.txt), but in a very primitive way and it apparently only supports Linux, but it may be a good start point.
This is likely to be the v2 of our software environment...one day.
Note : Additionally, if you want to support both qmake and cmake for instance, a well written Python script could generate both CMakeLists and pro files on demand!
Not sure whether this is a problem original poster faced, but as I see plenty of „just write CMakefile.txt” answers above, let me shortly explain why generating CMakefiles may make sense:
a) I have another build system I am fairly happy with
(and which covers large multiplatform build of big collection
of interconnected shared and static libraries, programs, scripting
language extensions, and tools, with various internal and external
dependencies, quirks and variants)
b) Even if I were to replace it, I would not consider cmake.
I took a look at CMakefiles and I am not happy with the syntax
and not happy with the semantics.
c) CLion uses CMakefiles, and Cmakefiles only (and seems somewhat interesting)
So, to give CLion a chance (I love PyCharm, so it's tempting), but to keep using my build system, I would gladly use some tool which would let me
implement
make generate_cmake
and have all necessary CMakefiles generated on the fly according to the current
info extracted from my build system. I can gladly feed the tool/script with information which sources and headers my app consists of, which libraries and programs it is expected to build, which -I, -L, -D, etc are expected to be set for which component, etc etc.
Well, of course I would be much happier if JetBrains would allow to provide some direct protocol of feeding the IDE with the information it needs
(say, allowed me to provide my own command to compile, to run, and to
emit whatever metadata they really need - I suppose they mainly need incdirs and defines to implement on the fly code analysis, and libpaths to setup LD_LIBRARY_PATH for the debugger), without referring to cmake. CMakefiles as protocol are somewhat complicated.
Maybe this could be helpful:
https://conan.io/
The author has given some speeches about cmake and how to create modular projects using cmake into CPPCon. As far as I know, this tool require cmake, so that I suppose that generate it when you integrate new packages, or create new packages. Recently I read something about how to write a higher level description of the C/C++ project using a YAML file, but not sure if it is part of conan or not (what I read was from the author of conan). I have never used, and it is something pending for me, so that, please if you use it and fit your needs, comment your opinions about it and how it fit your scenario.
I was looking for such a generator but at the end I decided to write my own (partly because I wanted to understand how CMake works):
https://github.com/Aenteas/cmake-generator
It has a couple of additional features such as creating python wrappers (SWIG).
Writing a generator that suits everyone is impossible but I hope it will give you an idea in case you want to make your customized version.
I am trying to open an existing C++ open-source library in Xcode to publish it with my own modification/additions. The library is Tesseract-OCR, which does not include a .xcodeproj file.
Since Xcode can function as an IDE, is it possible to open a bunch of files as a single project in Xcode? Is there an easy way to produce an Xcode project?
There are several ways you could do it, depending on the level of IDE integration you want. There's no direct way of importing a Makefile-based project into Xcode. You can create a project that builds via the Makefile, but you wouldn't get many of the benefits of using an IDE, since the editor features such as word completion rely on Xcode being able to parse the files in the project. You will be able to use the debugger though. To do this, create a new project and add a custom target with a script build phase that just calls down to Makefile.
If however the project you're building compiles very easily, ie without requiring a lot of macros to be set up, include paths, etc, then it may be simple to just create an empty project and merely add all source files to it. I've used this method extensively for building boost libraries. If this is a configure && make type project then you will probably have to run the configure step first, and ensure any top level config.h files are included in the project.
If the project has a complex makefile then it is likely to be an involved task to create a useful Xcode project
I realise you asked explicitly for Xcode, but in case you were actually trying to solve the problem of "I have existing C++ code which builds and runs fine from the command line, and I'd like to code and debug it in an IDE, what should I do?" my firm recommendation would be to avoid Xcode and go for Eclipse.
The reason is that as far as I can tell, Xcode has no way of ingesting the command line build environment and effectively requires you to recreate the make process inside Xcode from scratch. Fine for tiny projects, but anything with more than a few source files and it quickly becomes painful. Whereas in Eclipse everything is built around Makefiles. So in my case I got to the "step through code with working code completion" in Eclipse a lot quicker vs. never in Xcode. This of course could be because I'm an Xcode noob, but my 2c.
To create an Xcode project from an existing cmake project, you can run cmake -G Xcode. It produces some folders and files apart from the project file, so it might be better to create a folder for it first. For example:
mkdir -p build/xcode
cd build/xcode
cmake -G Xcode ../..
Xcode is a useable IDE for library creation.
Of course a good first step is to see if the one source code will build on its own with configure scripts that are included.
If not, it becomes a question of how many libraries you need to link in.
There are resources online (or at least there used to be) for using Xcode (or perhaps it's forerunner Product builder) for porting Unix projects to Mac.
Good tutorial at: http://www.macresearch.org/tutorial-introducing-xcode-30-organizer
Another good reference is Darwin Ports.
As for doing this on your own. You can build c++ based libraries in XCode. People do that every day. You can even use one of the Xcode templates to get you started.
However, library dev requires more experience with Xcode then say a simple Cocoa "Hello World" app.
The remaining questions will be assuring that the source code's dependencies are already built into the Mac's SDK. (Don't hold your breath for linking to MFC)
It's a general question... So it's a general answer.
In Xcode8,there is "Xcode->file->add files to...",then choose your files.If you want to add several files at a time,press "Cmd" when you are choosing.
I'm in the middle of setting up an build environment for a c++ game project. Our main requirement is the ability to build not just our game code, but also its dependencies (Ogre3D, Cegui, boost, etc.). Furthermore we would like to be able build on Linux as well as on Windows as our development team consists of members using different operating systems.
Ogre3D uses CMake as its build tool. This is why we based our project on CMake too so far. We can compile perfectly fine once all dependencies are set up manually on each team members system as CMake is able to find the libraries.
The Question is if there is an feasible way to get the dependencies set up automatically. As a Java developer I know of Maven, but what tools do exist in the world of c++?
Update: Thanks for the nice answers and links. Over the next few days I will be trying out some of the tools to see what meets our requirements, starting with CMake. I've indeed had my share with autotools so far and as much as I like the documentation (the autobook is a very good read), I fear autotools are not meant to be used on Windows natively.
Some of you suggested to let some IDE handle the dependency management. We consist of individuals using all possible technologies to code from pure Vim to fully blown Eclipse CDT or Visual Studio. This is where CMake allows use some flexibility with its ability to generate native project files.
In the latest CMake 2.8 version there is the new ExternalProject module.
This allows to download/checkout code, configure and build it as part of your main build tree.
It should also allow to set dependencies.
At my work (medical image processing group) we use CMake to build all our own libraries and applications. We have an in-house tool to track all the dependencies between projects (defined in a XML database). Most of the third party libraries (like Boost, Qt, VTK, ITK etc..) are build once for each system we support (MSWin32, MSWin64, Linux32 etc..) and are commited as zip-files in the version control system. CMake will then extract and configure the correct zip file depending on which system the developer is working on.
I have been using GNU Autotools (Autoconf, Automake, Libtool) for the past couple of months in several projects that I have been involved in and I think it works beautifully. Truth be told it does take a little bit to get used to the syntax, but I have used it successfully on a project that requires the distribution of python scripts, C libraries, and a C++ application. I'll give you some links that helped me out when I first asked a similar question on here.
The GNU Autotools Page provides the best documentation on the system as a whole but it is quite verbose.
Wikipedia has a page which explains how everything works. Autoconf configures the project based upon the platform that you are about to compile on, Automake builds the Makefiles for your project, and Libtool handles libraries.
A Makefile.am example and a configure.ac example should help you get started.
Some more links:
http://www.lrde.epita.fr/~adl/autotools.html
http://www.developingprogrammers.com/index.php/2006/01/05/autotools-tutorial/
http://sources.redhat.com/autobook/
One thing that I am not certain on is any type of Windows wrapper for GNU Autotools. I know you are able to use it inside of Cygwin, but as for actually distributing files and dependencies on Windows platforms you are probably better off using a Windows MSI installer (or something that can package your project inside of Visual Studio).
If you want to distribute dependencies you can set them up under a different subdirectory, for example, libzip, with a specific Makefile.am entry which will build that library. When you perform a make install the library will be installed to the lib folder that the configure script determined it should use.
Good luck!
There are several interesting make replacements that automatically track implicit dependencies (from header files), are cross-platform and can cope with generated files (e.g. shader definitions). Two examples I used to work with are SCons and Jam/BJam.
I don't know of a cross-platform way of getting *make to automatically track dependencies.
The best you can do is use some script that scans source files (or has C++ compiler do that) and finds #includes (conditional compilation makes this tricky) and generates part of makefile.
But you'd need to call this script whenever something might have changed.
The Question is if there is an feasible way to get the dependencies set up automatically.
What do you mean set up?
As you said, CMake will compile everything once the dependencies are on the machines. Are you just looking for a way to package up the dependency source? Once all the source is there, CMake and a build tool (gcc, nmake, MSVS, etc.) is all you need.
Edit: Side note, CMake has the file command which can be used to download files if they are needed: file(DOWNLOAD url file [TIMEOUT timeout] [STATUS status] [LOG log])
Edit 2: CPack is another tool by the CMake guys that can be used to package up files and such for distribution on various platforms. It can create NSIS for Windows and .deb or .tgz files for *nix.
At my place of work (we build embedded systems for power protection) we used CMake to solve the problem. Our setup allows cmake to be run from various locations.
/
CMakeLists.txt "install precompiled dependencies and build project"
project/
CMakeLists.txt "build the project managing dependencies of subsystems"
subsystem1/
CMakeLists.txt "build subsystem 1 assume dependecies are already met"
subsystem2/
CMakeLists.txt "build subsystem 2 assume dependecies are already met"
The trick is to make sure that each CMakeLists.txt file can be called in isolation but that the top level file can still build everything correctly. Technically we don't need the sub CMakeLists.txt files but it makes the developers happy. It would be an absolute pain if we all had to edit one monolithic build file at the root of the project.
I did not set up the system (I helped but it is not my baby). The author said that the boost cmake build system had some really good stuff in it, that help him get the whole thing building smoothly.
On many *nix systems, some kind of package manager or build system is used for this. The most common one for source stuff is GNU Autotools, which I've heard is a source of extreme grief. However, with a few scripts and an online depository for your deps you can set up something similar like so:
In your project Makefile, create a target (optionally with subtargets) that covers your dependencies.
Within the target for each dependency, first check to see if the dep source is in the project (on *nix you can use touch for this, but you could be more thorough)
If the dep is not there, you can use curl, etc to download the dep
In all cases, have the dep targets make a recursive make call (make; make install; make clean; etc) to the Makefile (or other configure script/build file) of the dependency. If the dep is already built and installed, make will return fairly promptly.
There are going to be lots of corner cases that will cause this to break though, depending on the installers for each dep (perhaps the installer is interactive?), but this approach should cover the general idea.
Right now I'm working on a tool able to automatically install all dependencies of a C/C++ app with exact version requirement :
compiler
libs
tools (cmake, autotools)
Right now it works, for my app. (Installing UnitTest++, Boost, Wt, sqlite, cmake all in correct order)
The tool, named «C++ Version Manager» (inspired by the excellent ruby version manager), is coded in bash and hosted on github : https://github.com/Offirmo/cvm
Any advices and suggestions are welcomed.