I'm looking to implement a build server for Qt-based C++ projects. The server needs to check out the necessary code / assets from Subversion, build the executable files, assemble the artifacts for installation projects, and build the installation media files. The target platforms and (rough) toolchains are:
Windows (32- and 64-bit): qmake, nmake, msbuild, wix toolchain. The end result is an installer EXE and DVD image.
Mac OS X: qmake, make, custom bash scripts to assemble package. The end result is an application bundle within a disk image and a DVD image.
Ubuntu (32- and 64-bit): qmake, make, debuild-based scripts. The end result is a collection of DEB files and a DVD image.
Fedora (32- and 64-bit): qmake, make, rpmbuild-based scripts. The end result is a collection of RPM files and a DVD image.
So that's at least 4 build agents (maybe more if 32- and 64-bit can't be done on the same box) and 7 configurations. Open-source projects are preferred, but that is not an absolute requirement.
Most of the tools I'm seeing seem to be catered to Java (Jenkins, CruiseControl, etc.) or .Net (CruiseControl.net, etc.) Can those be used with a C++ toolchain, or will I constantly be fighting the system? Anything that you have used in the past and found works well with Qt / C++?
I use Jenkins for building and packaging many C++ projects, based on qmake, cmake, and makefiles.
There are plugins for cmake, qmake, and msbuild, but any command line scripts can be run as well.
I have done packaging using Jenkins with no problems, as it is just another command line step in a project.
There are good plugins for monitoring the number of warnings/errors produced by the compiler (I normally use GCC).
It also has matrix builds which allow you to build a project several times with different combinations of compiler flags, pre-processor variables, platform, etc. One project I set up is a matrix build with 5 boolean preprocessor flags on two platforms, which then does 2^6=64 builds. These can take a bit of setting up to get correct.
Here you can read a quick example:
Continuous Integration Server - Hudson
I think that Hudson, jenkins and builbot are worth a try. Wasting a day or two evaluating and trying them with a quick example will help you to choose confidently.
Most of the tools I'm seeing seem to be catered to Java (Jenkins, CruiseControl, etc.) or .Net (CruiseControl.net, etc.) Can those be used with a C++ toolchain, or will I constantly be fighting the system? Anything that you have used in the past and found works well with Qt / C++?
Any reasonably capable CI system will have a piece that will allow you to execute any program you want for your build command.
Here's what I would consider:
Does the CI system run on your system(s) of choice
Does it allow you an easy way to view your logs
Does it integrate with your test runner
Does it integrate with your code coverage reports (e.g. BullseyeCoverage w/C++ & Qt)
Will it publish your files in a manner sensible for your needs
Will at provide an archive/store of files, if necessary (e.g. pdbs & lib*.so.debug)
If the CI system doesn't support feature X, will you have to write it for each supported OS/system
Is the CI system / UI easy for you to use.
I did the above using CruiseControl and most things were pretty easy. I wrote everything in make or qmake and simply called out to the command that I needed executed. For unit test and code coverage integration I output stuff to XML and transformed it to something supported by CruiseControl.
My recommendation, take a look at the recommended CI systems and examine them based on the criteria above.
I'm using buildbot for this. I've been using it for 4 years, and I feel very happy with it.
It is an application written in python, that runs on a server and can manage multiple clients on various OSes. I'm currently using Windows XP, Windows 7, Debian, Ubuntu and CentOS build slaves. My projects are C++, and one of them (the end user GUI) is made in Python. But we've also integrated with other frameworks, for other features than GUI.
What is really good about buildbot is that it works by running command lines on slaves. With this, you can do whatever you want. Even on Windows systems to compile using Visual Studio! From these command lines, you get all the output centralized on the server, and accessible.
You may also find alternatives on this site that references many of them.
Disclaimer: I looked at it 3 years ago, I don't know if it is still accurate.
Hudson or Jenkins is pretty good.
Jenkins is indeed pretty popular for developing such a custom service, even after all these years, considering the question is already 7 years old.
Felgo also offers a Continuous Integration and Delivery (CI/CD) service for Qt. It supports desktop platforms as well as iOS, Android and embedded targets. The full feature set is described in the blog post.
Disclaimer: I am a software developer at Felgo
Related
We have a complex C++ codebase targeting multiple mobile platforms. We currently have Windows CE (4.2 to 6.5 both raw CE and Mobile based on those), Android (2.1+), iPhone (4+), almost working Bada (2.0+) and if anything comes of the new C++/CX thing, are likely to add Windows Phone (8+). Plus a testing version on Win32 and service application on Win64 that shares some code. We also already tried to compile unit tests on Linux and questions already came (too small business volume so far, but that might change) to get it working on some other Linux platforms.
We currently compile the code with native tools for each platform. Each of them is pretty complex and has some hacks in or around it to achieve reasonably single-click builds. And for Bada, we didn't solve builds outside of the Samsung-tweaked Eclipse yet, which we'll have to do for production.
So far it works, but is becoming more and more of a maintenance problem. The biggest problem currently is the iPhone build, because unlike Visual Studio project files and plain makefiles, it's not possible to add/remove/rename files in XCode project by hand and only two people have MacOS boxen and any experience with XCode (while everybody has Windows and knows Visual Studio). We also need to create some makefiles for the Bada target (it's plain GNU toolchain, so anything capable of cross-compiling with those should do) and I wouldn't mid getting rid of some of the kludges in Android build: fix for bug in dependency handling when building under cygwin, some hacks on top of the ant build script and shell glue to massage manifest and hold it all together.
So I am looking for advice on ways to unify the build process for this set of diverse platforms.
It absolutely has to handle building the iPhone and Bada executables and the native part of the Android build (because nobody has all three platforms to test separately).
Has to handle large project consisting of several shared libraries, one main binary, one test binary for the same platforms and some auxiliary binaries built only on some platforms (well, currently Win32 only).
Has to be able to generate build configuration header and Java file with version from version control system and some user-defied variables, since we do 18 and counting slightly different builds for various customers.
And of course it has to automatically handle dependencies (header files) and generally be reliable.
For the other things I am prepared to hack around any deficiencies, but I would obviously like to keep the amount of shell duct tape and spit to minimum, so it should:
Be able to integrate with Visual Studio, Eclipse and XCode enough that each environment can trigger build, upload the build product to the respective target and attach debugger to it there.
Be able to build Java and call the custom packaging tools for Android, so we don't have to hack around the ant build script that Google gratuitously and incompatibly changed twice in last two years already (and old SDK can't be downloaded).
Be able to install various data files and call random packagers and random other scripts and tools and stuff, so it does not have to be mixed with too much shell script.
We have so far started trying CMake (didn't have much time, so didn't get far, but will have to do something shortly) and also thought about SCons. However I have already tried building with SCons for Windows CE a few years ago, but since it generates makefile-type projects for Visual Studio and those didn't work for embedded platforms in VS2005, I gave up. CMake can generate native makefiles, but a custom, bit out of date, branch is needed for CE. So I'd like to ask whether there are any other tools we might want to look into or any known stumbling blocks with these tools we should be aware of.
Updates: I've found instructions for android native binaries in several places and pixellight even has cmake script to generate apk by directly calling the packaging tools. Also this shell script shows it. iPhone usage seems to be documented here.
The same CMake build files can be used to generate and build projects across platforms for atleast Win32 (VS2005), Linux(ubuntu), and OS X (10.5.8 Leopard), in my personal experince. There are multiple projects for using CMake in the android app build process, eg http://code.google.com/p/android-cmake/. With all these options, I would consider CMake a good option. Further CMake is easy to pick up, and even easier if you have prior experience with writing configure.ac and Makefile.ac files.
Note: How CMake works:
Developer/Maintainer creates CMakefile.
Builder runs cmake, which uses the CMakefile to generate platform specific build files.(linux makefiles/OSX makefiles/VS project files)
Builder fires off the build command.
CMake is the best choise.
Starting with CMake 2.8.11 Windows CE is supported out of the box. Until the release of that version a ​Nightly Binary of CMake can be used.
EDIT 2012.03.02: updated build tool section with some other requirements. Addding premake.
EDIT 2012.03.05: updated build tool section. Focused on CMake. Added Ant + CMakeAnt.
I have a lot of small Visual C++ projects and I want to start using continuous integration for that. A lot of tools are involved in the process. I start reading articles here and there and it is hard for me to see which to tool could integrate nicely with the others. So I am looking for an already tested continuous integration workflow that could respect my needs:
Current Context
C++/Qt applications developed with Visual Studio;
Less than 1'000 source files;
Small dev team;
Small budget;
Preferably open source/free software;
Looking for easy to use and simple tools. No need of something heavy and "powerful".
Finally packages software (installers ready for download) should work for windows and OSX. Linux (most popular distributions) is a plus.
Tools needed
Continuous Integration Server:
Needs to be Jenkins. I know it and I like it. I could change for another one if it is really needed.
Source Code Management:
GIT.
Documentation Generator:
Doxygen.
Build Tool:
We will need to generate solutions and configurations for all the platforms we need to support. I have been strongly advised to use CMake for that. Ant + CMakeAnt (http://code.google.com/p/cmakeant/) seems to be a nice addition too.
Unit Testing Framework:
CPPunit, C++Test, Googletest?
Installers Builder
CPack (part of CMake seems great too).
Hardware:
Preferably, one single machine with VirtualBox or VMWare. is it only possible? Would it be better to use Linux for such things? Or would a MAC mini be sufficient?
So, has anyone here already built something similar? With which tools exactly?
Thanks for any comments and suggestions.
I have successfully used CMake/CTest/CPack with Jenkins, there is a CMake builder plugin available.
I found the ctest --output-on-failure option to be useful as the output from the test is shown in the Jenkins log on failure.
I use the Boost.Test framework for testing C++, it works fine with ctest, I run it with the -l all option to get all the logging output when something goes wrong.
You should be able to bootstrap the entire process using a bunch of scripts.
Write them in Perl or Python. There are a million tutorials on how to bootstrap these sorts of processes.
If you're stuck on implementing a specific party of this build-process, ask a specific question about it.
I am working on my free time on a multiplatform/multi-architecture library written in C++.
Before every release, I have to boot up several computers (One on Windows, one on Linux, another one on Mac OS, ...) just to make sure the code compiles and runs fine on every platform.
So I decided to create my own buildmachine but I really don't know what tools exist to do this. I'd like my buildmachine to run on Linux but any other solution will be accepted.
Ideally, I would just have to click on a "Build all" button, and it would compile my library for the different platforms/architectures, generate archives from the result and/or report potential errors.
My project "constraints" are:
It is written in C++
It compiles on Windows using SConstruct/MinGW and Visual Studio 2010
It compile on Linux and Mac OS using SConstruct/g++
The sources are stored into Subversion (svn)
Do you know any tool/set of tools that could help me achieving my goal ?
Thank you very much.
I would setup 3 VMs (VirtualBox is free), one for each platform.
Install TeamCity (or Hudson) on Linux and agents on the other VMs and then it's just a matter of configuring the build system.
At the very basic level you should have 2 tasks: one to checkout the sources from Subversion and another to invoke scons.
I'm not too familiar with Hudson but TeamCity is certainly capable of generating reports of a build, display progress etc.
I am currently researching a good setup for a continuous integration server which would build various C++ applications for several Linux distributions.
My primary question is how other users here have handled the differences in system libraries between Linux distributions?
While it might be relatively easy to build direct dependencies such as UI libraries along with an application, "indirect" dependencies such as glibc look like a big pain if they had to be built alongside the application every time. I am therefore thinking of moving the actual build execution into a separate virtual machine for each distribution, e.g. using rlogin to run the commands. My goal is to prevent binary incompatibilities between build-machine library versions and those deployed in the target distributions.
Does anyone here have any experience with such a process and could tell if the above sounds like a feasible approach?
We use Jenkins (Contiguous Integration) and CMake (build system) for this purpose. Jenkins is similar to Buildbot, i.e. it also has buildmaster and buildslaves. Currently I have setup 8 slaves to build for 4 different platforms (FC8, FC10, FC12 and Windows 7). We build both debug and release binaries, so I dedicated one slave for each platform and build type.
As for the third party libraries like Qt & Boost, I compiled them on each platform and checked them into a separate repository.
#esavard: We use CMake 2.8 to do cross compilation, I have not used minigw but a quick google search indicates that it is possible. Here is a link to a tutorial to cross compile for Windows on Linux using CMake and miniGW.
I have not used Buildbot and cannot comment on its features but thought I should mention an alternative that we are currently using.
Hope this helps.
Buildbot has the notion of buildmasters and buildslaves.
A buildmaster takes care of displaying the web GUI, sending email, triggering builds, and other housekeeping. The buildslaves wait on the buildmaster and when commanded perform builds.
We have buildbot set up to build on a number of different platforms, some of them VMs, and it's working well for us.
Certainly buildbot and many virtual machines is the way to go with this. We have VMWare ESX server hosting many build slaves which overnight compile our application. The application is then tested on another virtual machine (not the build slave and just having a default OS install) to verify that it works and all dependency's are packaged.
The nice thing I would like to do is make the testing run time phase an automated step but I haven't been given the time to do that yet.
I'm involved in C++ project targeted for Windows and Linux (RHEL) platforms. Till now the development was purely done on Visual Studio 2008. For Linux compilation we used 3rd party Visual Studio plugin, which read VS solution/perojects files and remotely compiled on Linux machine.
Recently the decision was to abandon the 3rd party plugin.
Now my big concern is a build system. I was looking around for cross platform build tools. This way I don't need to maintain two set of build files (e.g. vcproj/solution for Windows and make files for Linux).
I found the following candidates:
a. Scons
b. cmake
What do you think about the tools for cross-platfrom development?
Yet another point that bothers me is that Visual Studio (+ Visual Assist) will loose a lot functionality without vcproj files - how you handle the issue with the tools?
Thanks
Dima
PS 1: Something that I like about Scons is that it
(a) uses python and hence it's flexible, while cmake uses propriety language (I understand that it's not a winner feature for a build-system) (b) self contained (no need to generate makefiles on Linux as with cmake).
So why not Scons? Why in your projects the decision was to use cmake?
CMake will allow you to still use Visual Studio solutions and project files. Cmake doesn't build the source code itself, rather it generated build-files for you. For Linux this can be Code::Blocks, KDevelop or plain makefiles or still other more esoteric choices . For Windows it can be among others Visual Studio project files and still others for MacOS.
So Visual Studio solutions and projects are created from your CMakeLists.txt. This works for big projects just fine. E.g. current Ogre3d uses CMake for all platforms (Windows, Linux, MacOS and IPhone) and it works really well.
I don't know much about scons in that regard though, I only used to build one library and only in Linux. So I can't compare these two on fair ground. But for our multi-platform projects CMake is strong enough.
I haven't used Scons before, so can't say how that works, but CMake works pretty well.
It works by producing the build files needed for the platform you're targeting.
When used to target VC++, it produces solution and project files so from VS, it appears as if they were native VS projects. The only difference is, of course, that if you edit the project or solution directly through VS, the changes will be erased the next time you run CMake, as it overwrites your project/solution files.
So any changes have to be made to the CMake files instead.
We have a big number of core libraries and applications based on those libraries. We maintain a Makefile based build system on Linux and on Windows using the Visual Studio solution for each project or library.
We find it works well for our needs, each library or app is developed either on linux or windows with cross compilation in mind (e.g. don't use platform specific api's). We use boost for stuff like file paths, threads and so on. In specific cases we use templates/#defines to select platform specific solution (for example events). When is ready we move to the other system (linux or windows), recompile, fix warnings/errors and test.
Instead of spending time figuring out tools that can cross compile on both platforms we use system that is best for each platform and spend time fixing specific issues and making the software better.
We have GUI apps only on Windows atm. so there's no GUI to cross compile. Most of our development that is shared between Windows and Linux is server side networking (sockets, TCP/IP, UDP ...) and then client side tools on Linux and GUI apps on Windows.
Using with perforce for source code version management we find in quite many cases that the Linux Makefile system is much more flexible for what we need then Windows VS. Especially for using multiple workspaces (views of source code versions) where we need to point to common directories and so on. On Linux this can be done automatically running a script to update environment variables, on Visual Studio referencing environment variables is very inflexible because it's hard to update automatically between views/branches.
Re sync question:
I assume you are asking how to make sure that the two build systems get synchronized between linux and windows. We are actually using Hudson on Linux and CruiseControl on Windows (we had windows first with cruise control, when I went to setup linux version I figured Hudson is better so now we have mixed environment). Our systems are running all the time. When something is updated it is tested and released (either windows or linux version) so you would know right away if it does not work. During testing we make sure all the latest features are there and fully functional. I guess that's it, no dark magic involved.
Oh you mean build scripts ... Each application has it's own solution, in solution you setup up dependencies. On Linux side I have a makefile for each project and a build script in project directory that takes care of all dependencies, this mostly means build core libraries and couple of specific frameworks required for given app. As you can see this is different for each platform, it is easy to add line to build script that changes to directory and makes required project.
It helps to have projects setup in consistent way.
On Windows you open project and add dependency project. Again no magic involved. I see this kind of tasks as development related, for example you added new functionality to a project and have to link in the frameworks and headers. So from my point of view there is no reason to automate these - as they are part of what developers do when they implement features.
Another options is premake. It's like cmake in that it generates solutions from definition files. It's open source and the latest version is very highly customizable using Lua scripting. We were able to add custom platform support without too much trouble. For your situation it has support for both Visual Studio and GNU makefiles standard.
See Premake 4.0 Homepage
CruiseControl is a good choice for continuous integration. We have it running on Linux using Mono with success.
Here is an article about the decision made by KDE developers to choose CMake over SCons. However I've to point that this article is almost three years old, so scons should have improved.
Here is comparison of SCons with other building tools.
Had to do this a lot in the past. What we did is use gnu make for virtually everything including windows at times.
You can use the project files under windows if you prefer and use gnu make for Linux.
There isn't really a nice way to write cross platform makefiles because the target file will
be different among other things (and pathname issues, \ vs / etc). In general, you'll probably be tweaking the code across the various platforms to take subtle differences into account, so a tweak to a make file and checking on the other platforms would have to happen
anyway.
Many OS projects maintain Makefiles for different platforms such as zlib where they are named like Makefile.win, Makefile.linux etc. You could follow their lead.