I found similar topic: What are the differences between Autotools, Cmake and Scons? , but my question is a little bit other and I think the answers could be other too.
I found a lot of articles telling that waf is unstalbe (API changes), is not yet ready for production etc (but all of these articles are 2 or 3 years old).
Which of these build tools should be used if I want to:
create big C++ (11) project - lets say a complex compiler
use with LLVM
be sure it will be flexible and simple to use
be sure it will be fast enought
compile under all standard platforms (the base platform is Linux, but I want to compile under Windows and MacOSX also)
Reading a lot of articles I found out Cmake and waf the "best" tools available, but I have no expirence with them and it is really hard to find out any comparison, which is not very biased (like comparison of the scons author) and not very old.really
waf covers nearly all your requirements ...
API change: not a problem as waf shall be included in the source tarball (<100Ko)
big project: you can split your configuration in subdirectories (the contexts could be inherited). I've worked on projects with more than 10k files in C/C++/fortran77/fortran90/python/Cython including documentation in doxygen/sphinx.
flexibility and easyness: you can add extra modules en python (http://code.google.com/p/waf/wiki/AddingNewToolsToWaf)
fast: the tasks could be run in parallel: http://www.retropaganda.info/~bohan/work/psycle/branches/bohan/wonderbuild/benchmarks/time.xml
multi plat-form: you can run Waf everywhere python is available, that includes Windows and MacOs. Waf is compatible with mscvc, gcc, icc, and other compilers. You can produce visual/eclipse projects.
... but waf seems to have an issue with llvm: http://code.google.com/p/waf/issues/detail?id=1252
EDIT: as said by Wojciech Danilo, LLVM issue has been fixed
I'm currently using CMake for my own language implementation via C++11 and LLVM.
I like CMake for it's easy to use syntax. LLVM can be loaded with an easy 'load_package' command. After that you can use all the headers and libraries you need. CMake lets child scripts inherit variables from parent scripts. So you do not need to set variables and load packages in every sub directory.
The C++11 support depends on your compiler you want to use. All in all CMake is just a layout to create your 'real' build script.
When you're using make you can use make's --jobs=N to speed up compilation on multicore-platforms. On Windows you could generate Visual Studio 2012-project files and use Microsoft's build system and use their build-jobs to speed up the compilation process.
You should always create a subfolder for build-files (myproject/build or something). This way you keep your source tree clean (cd build; cmake ..; cd ..).
I can't speak for all the other tools out there.
Related
First, i know that make is used for building the code. But which code?
But what does it mean by building a code, and after executing the make command, what is presented to the user?
Second, how is it different from make build_for_e2e?
What Wikipedia tells about make
Make is a build automation tool that automatically builds executable programs and libraries from source code
Compilation process becomes big and complex in big projects, where numbers of files need to be compiled, with flags and libraries. Where it will become hard for people to compile it one by one. So these types of tools were introduced, there are more similar tools available for same use like cmake, gradle, maven. e2e's Build is also a form of build process, with different form of specifications.
For C people mostly use make. It is helpful for porting software packages in different systems.
How make is used:
As said make is a tool, which will be available in our system, we can execute it by giving command make in the directory which needs to be compiled. Then make looks for Makefile, which is provided in the package directory and it contains information about compilation of the project. Then make as per info gathered from Makefile, it compiles the package.
You can also create Makefile for your project, so that it can be also supported and compiled with make. Simple tutorial for it can be found here. For big projects you can use gnu autotools contains autoconf and automake which will help you to create your all files required by make automatically. You can find tutorial regarding it here and here . These contains some basic information, you can find some advance tutorial regarding autotools, use google for more information on it.
Seems most open source projects in C/C++ only provide the source code,i.e. nginx
Is this a convention that anyone interested in joining the developing team should figure out the .sln/.project files himself to qualify??
most open source projects are coming from the linux side of computing. thus, they are mainly using unix style build tools, as well as open source compilers.
the main build tool is make, which uses a makefile to know how to build a project. on Windows, the main open source compiler is MinGW which is a win32 port of gcc. the use of those tools allows to keep a maximum of common things between unix and windows.
note that .sln files are specific to microsoft compilers which are not free to use (and are rather costly), they are not portable and so are not suitable for multi-platform programming.
Some projects use CMake, which can generate project files for Your Favourite Build System, but as mentioned above, you don't need to use .sln and pro files, even if a project is built with the MSVC compiler, MinGW + makefiles, or scons, or CMake, or any other number of scripty methods to invoke the right commands to compile the program will work just fine.
Don't confuse the IDE with the compiler!
No, most opensource project do not use MSVC solutions as they not portable and very weak in terms of features.
In most cases they use what is called "build-system" like autotools, CMake or SCons.
These build systems include information about:
source code and how to build it
various system checks that should be done (like find various 3rd part libraries)
How to build and run unit-tests
How to install application
They also allow important tasks like cross compilation and packaging for deploy.
These task done via specific build system scripting language that allow you big flexibility.
So, these build systems generally much more powerful then typical "project files" and they
generally work with multiple compilers on different platforms and operating systems.
Some of build systems (like CMake) allow as one of the options to generate MSVC solutions as well as one of optional ways to build application.
I currently maintain a few boxes that house a loosely related cornucopia of coding projects, databases and repositories (ranging from a homebrew *nix distro to my class notes), maintained by myself and a few equally pasty-skinned nerdy friends (all of said cornucopia is stored in SVN).
The vast majority of our code is in C/C++/assembly (a few utilities are in python/perl/php, we're not big java fans), compiled in gcc. Our build toolchain typically consists of a hodgepodge of make, bash, grep, sed and awk. Recent discovery of a Makefile nearly as long as the program it builds (as well as everyone's general anxiety with my cryptic sed and awking) has motivated me to seek a less painful build system.
Currently, the strongest candidate I've come across is Boost Build/Bjam as a replacement for GNU make and python as a replacement for our build-related bash scripts. Are there any other C/C++/asm build systems out there worth looking into? I've browsed through a number of make alternatives, but I haven't found any that are developed by names I know aside from Boost's.
(I should note that an ability to easily extract information from svn commandline tools such as svnversion is important, as well as enough flexibility to configure for builds of asm projects as easily as c/c++ projects)
We've started using CMake here at work recently. I've been pretty happy with it so far.
scons or cmake.
Both "cross platform" - enable compiling on Windows and Linux.
Cmake now very popular (for example boost uses it). It creates native build files for each platform - Makefiles (for Linux), VC++ projects (for Visual Studio) from custom files (CMakeList.txt). In can also create Eclipse projects, KDevelop projects etc. Since it creates native build files, you can continue using all features you're used for. For example, ccache/colorgcc/distcc for Makefiles or Visual Assist X for Visual Studio.
We use it our project and are happy with it - automatic dependencies, easy syntax, robust builds.
Scons is python bases system, which perform the builds by itself. It's IMHO less popular, and still slow for large project. But for msmall to medium project maybe good alternative.
You could use python-based build system, too -- http://code.google.com/p/waf/
What we need in our firm is a sort of release management tool for Linux/C++. Our products consist of multiple libraries and config files. Here I will list the basic features we want such system to have:
Ability to track dependencies, easily increase major versions of libraries whose dependencies got their major version increased. It should build some sort of dependency graph internally so it can know who is affected by an update.
Know how to build the products it handle. Either a specific build file or even better - ability to read and understand makefiles.
Work with SVN so it can check for new releases from there and does the build.
Generate some installers - in rpm or tar.gz format. For that purpose it should be able to understand the rpm spec file format.
Currently we are working on such tool which is already pretty usable. However I believe that our task is not unique and there should be some tool out there which does the job.
You should look into using a mix between Hudson, Maven (for build management), Ivy (for dependencies management) and Archiva (for artifacts archival).
Also, if you are looking into cross.compilation, take a look at Make Project Creator (MPC) and Bakefile.
Have fun!!
In the project I'm currently working on we use cmake and other Kitware tools to handle most of this issues for native code (C++). Answering point by point:
The cmake scripts handle the dependencies for our different projects. We have a dependency graph but I don't know if is a home-made script or it is a functionality that cmake provides.
Well cmake generates the makefiles regarding the platform. I generates projects for eclipse cdt and visual studio if it is asked to do so in case of developing.
Cmake has a couple of tools, ctest and cdash that we use to do the daily build and see how the test are doing.
In order to create the installer cmake has cpack. From just one script it can generate tar.gz, deb or rpm files in Linux or an automatically generated NSIS script to generate installers in windows.
For Java code we use maven and hudson that have been already mentioned here.
Take a look at this article from DDJ, in which a more robust build system concept (than make) is presented and implemented. Not sure it will fit well to your requirements, but it's the closest I've ever seen. I was looking for the same thing months ago, and then I discovered the article.
http://www.drdobbs.com/architect/218400678
Maven has a native code plugin. I don't think it'll do everything you want, but it's good at tracking version numbers of dependencies, will build artefacts and it'll work with your VCS.
No idea
cmake/scons: I have used cmake but I don't exactly love it, but I have heard really good things about scons. But scons is python-based, so you need to have python installed on the build/dev machines.
I use Hudson, which has a plugin to fetch from svn. It performs intelligently in general, and in particular builds only if some file has changed in an svn update. Hudson is easy to get started with. Hudson is java-based and is pretty popular with the Java community. This means it is quite cross-platform, but you need to have JRE installed on the build machine.
Probably can call some rpm tool within hudson.
I'm in the middle of setting up an build environment for a c++ game project. Our main requirement is the ability to build not just our game code, but also its dependencies (Ogre3D, Cegui, boost, etc.). Furthermore we would like to be able build on Linux as well as on Windows as our development team consists of members using different operating systems.
Ogre3D uses CMake as its build tool. This is why we based our project on CMake too so far. We can compile perfectly fine once all dependencies are set up manually on each team members system as CMake is able to find the libraries.
The Question is if there is an feasible way to get the dependencies set up automatically. As a Java developer I know of Maven, but what tools do exist in the world of c++?
Update: Thanks for the nice answers and links. Over the next few days I will be trying out some of the tools to see what meets our requirements, starting with CMake. I've indeed had my share with autotools so far and as much as I like the documentation (the autobook is a very good read), I fear autotools are not meant to be used on Windows natively.
Some of you suggested to let some IDE handle the dependency management. We consist of individuals using all possible technologies to code from pure Vim to fully blown Eclipse CDT or Visual Studio. This is where CMake allows use some flexibility with its ability to generate native project files.
In the latest CMake 2.8 version there is the new ExternalProject module.
This allows to download/checkout code, configure and build it as part of your main build tree.
It should also allow to set dependencies.
At my work (medical image processing group) we use CMake to build all our own libraries and applications. We have an in-house tool to track all the dependencies between projects (defined in a XML database). Most of the third party libraries (like Boost, Qt, VTK, ITK etc..) are build once for each system we support (MSWin32, MSWin64, Linux32 etc..) and are commited as zip-files in the version control system. CMake will then extract and configure the correct zip file depending on which system the developer is working on.
I have been using GNU Autotools (Autoconf, Automake, Libtool) for the past couple of months in several projects that I have been involved in and I think it works beautifully. Truth be told it does take a little bit to get used to the syntax, but I have used it successfully on a project that requires the distribution of python scripts, C libraries, and a C++ application. I'll give you some links that helped me out when I first asked a similar question on here.
The GNU Autotools Page provides the best documentation on the system as a whole but it is quite verbose.
Wikipedia has a page which explains how everything works. Autoconf configures the project based upon the platform that you are about to compile on, Automake builds the Makefiles for your project, and Libtool handles libraries.
A Makefile.am example and a configure.ac example should help you get started.
Some more links:
http://www.lrde.epita.fr/~adl/autotools.html
http://www.developingprogrammers.com/index.php/2006/01/05/autotools-tutorial/
http://sources.redhat.com/autobook/
One thing that I am not certain on is any type of Windows wrapper for GNU Autotools. I know you are able to use it inside of Cygwin, but as for actually distributing files and dependencies on Windows platforms you are probably better off using a Windows MSI installer (or something that can package your project inside of Visual Studio).
If you want to distribute dependencies you can set them up under a different subdirectory, for example, libzip, with a specific Makefile.am entry which will build that library. When you perform a make install the library will be installed to the lib folder that the configure script determined it should use.
Good luck!
There are several interesting make replacements that automatically track implicit dependencies (from header files), are cross-platform and can cope with generated files (e.g. shader definitions). Two examples I used to work with are SCons and Jam/BJam.
I don't know of a cross-platform way of getting *make to automatically track dependencies.
The best you can do is use some script that scans source files (or has C++ compiler do that) and finds #includes (conditional compilation makes this tricky) and generates part of makefile.
But you'd need to call this script whenever something might have changed.
The Question is if there is an feasible way to get the dependencies set up automatically.
What do you mean set up?
As you said, CMake will compile everything once the dependencies are on the machines. Are you just looking for a way to package up the dependency source? Once all the source is there, CMake and a build tool (gcc, nmake, MSVS, etc.) is all you need.
Edit: Side note, CMake has the file command which can be used to download files if they are needed: file(DOWNLOAD url file [TIMEOUT timeout] [STATUS status] [LOG log])
Edit 2: CPack is another tool by the CMake guys that can be used to package up files and such for distribution on various platforms. It can create NSIS for Windows and .deb or .tgz files for *nix.
At my place of work (we build embedded systems for power protection) we used CMake to solve the problem. Our setup allows cmake to be run from various locations.
/
CMakeLists.txt "install precompiled dependencies and build project"
project/
CMakeLists.txt "build the project managing dependencies of subsystems"
subsystem1/
CMakeLists.txt "build subsystem 1 assume dependecies are already met"
subsystem2/
CMakeLists.txt "build subsystem 2 assume dependecies are already met"
The trick is to make sure that each CMakeLists.txt file can be called in isolation but that the top level file can still build everything correctly. Technically we don't need the sub CMakeLists.txt files but it makes the developers happy. It would be an absolute pain if we all had to edit one monolithic build file at the root of the project.
I did not set up the system (I helped but it is not my baby). The author said that the boost cmake build system had some really good stuff in it, that help him get the whole thing building smoothly.
On many *nix systems, some kind of package manager or build system is used for this. The most common one for source stuff is GNU Autotools, which I've heard is a source of extreme grief. However, with a few scripts and an online depository for your deps you can set up something similar like so:
In your project Makefile, create a target (optionally with subtargets) that covers your dependencies.
Within the target for each dependency, first check to see if the dep source is in the project (on *nix you can use touch for this, but you could be more thorough)
If the dep is not there, you can use curl, etc to download the dep
In all cases, have the dep targets make a recursive make call (make; make install; make clean; etc) to the Makefile (or other configure script/build file) of the dependency. If the dep is already built and installed, make will return fairly promptly.
There are going to be lots of corner cases that will cause this to break though, depending on the installers for each dep (perhaps the installer is interactive?), but this approach should cover the general idea.
Right now I'm working on a tool able to automatically install all dependencies of a C/C++ app with exact version requirement :
compiler
libs
tools (cmake, autotools)
Right now it works, for my app. (Installing UnitTest++, Boost, Wt, sqlite, cmake all in correct order)
The tool, named «C++ Version Manager» (inspired by the excellent ruby version manager), is coded in bash and hosted on github : https://github.com/Offirmo/cvm
Any advices and suggestions are welcomed.