We have a complex C++ codebase targeting multiple mobile platforms. We currently have Windows CE (4.2 to 6.5 both raw CE and Mobile based on those), Android (2.1+), iPhone (4+), almost working Bada (2.0+) and if anything comes of the new C++/CX thing, are likely to add Windows Phone (8+). Plus a testing version on Win32 and service application on Win64 that shares some code. We also already tried to compile unit tests on Linux and questions already came (too small business volume so far, but that might change) to get it working on some other Linux platforms.
We currently compile the code with native tools for each platform. Each of them is pretty complex and has some hacks in or around it to achieve reasonably single-click builds. And for Bada, we didn't solve builds outside of the Samsung-tweaked Eclipse yet, which we'll have to do for production.
So far it works, but is becoming more and more of a maintenance problem. The biggest problem currently is the iPhone build, because unlike Visual Studio project files and plain makefiles, it's not possible to add/remove/rename files in XCode project by hand and only two people have MacOS boxen and any experience with XCode (while everybody has Windows and knows Visual Studio). We also need to create some makefiles for the Bada target (it's plain GNU toolchain, so anything capable of cross-compiling with those should do) and I wouldn't mid getting rid of some of the kludges in Android build: fix for bug in dependency handling when building under cygwin, some hacks on top of the ant build script and shell glue to massage manifest and hold it all together.
So I am looking for advice on ways to unify the build process for this set of diverse platforms.
It absolutely has to handle building the iPhone and Bada executables and the native part of the Android build (because nobody has all three platforms to test separately).
Has to handle large project consisting of several shared libraries, one main binary, one test binary for the same platforms and some auxiliary binaries built only on some platforms (well, currently Win32 only).
Has to be able to generate build configuration header and Java file with version from version control system and some user-defied variables, since we do 18 and counting slightly different builds for various customers.
And of course it has to automatically handle dependencies (header files) and generally be reliable.
For the other things I am prepared to hack around any deficiencies, but I would obviously like to keep the amount of shell duct tape and spit to minimum, so it should:
Be able to integrate with Visual Studio, Eclipse and XCode enough that each environment can trigger build, upload the build product to the respective target and attach debugger to it there.
Be able to build Java and call the custom packaging tools for Android, so we don't have to hack around the ant build script that Google gratuitously and incompatibly changed twice in last two years already (and old SDK can't be downloaded).
Be able to install various data files and call random packagers and random other scripts and tools and stuff, so it does not have to be mixed with too much shell script.
We have so far started trying CMake (didn't have much time, so didn't get far, but will have to do something shortly) and also thought about SCons. However I have already tried building with SCons for Windows CE a few years ago, but since it generates makefile-type projects for Visual Studio and those didn't work for embedded platforms in VS2005, I gave up. CMake can generate native makefiles, but a custom, bit out of date, branch is needed for CE. So I'd like to ask whether there are any other tools we might want to look into or any known stumbling blocks with these tools we should be aware of.
Updates: I've found instructions for android native binaries in several places and pixellight even has cmake script to generate apk by directly calling the packaging tools. Also this shell script shows it. iPhone usage seems to be documented here.
The same CMake build files can be used to generate and build projects across platforms for atleast Win32 (VS2005), Linux(ubuntu), and OS X (10.5.8 Leopard), in my personal experince. There are multiple projects for using CMake in the android app build process, eg http://code.google.com/p/android-cmake/. With all these options, I would consider CMake a good option. Further CMake is easy to pick up, and even easier if you have prior experience with writing configure.ac and Makefile.ac files.
Note: How CMake works:
Developer/Maintainer creates CMakefile.
Builder runs cmake, which uses the CMakefile to generate platform specific build files.(linux makefiles/OSX makefiles/VS project files)
Builder fires off the build command.
CMake is the best choise.
Starting with CMake 2.8.11 Windows CE is supported out of the box. Until the release of that version a ​Nightly Binary of CMake can be used.
Related
I would like to be able to compile C and C++ code under the Windows environment without using an IDE, just by using the Windows Command prompt (cmd.exe).
I come from Linux, where you are be able to install the gcc package with just a command in the terminal:
$ sudo apt install gcc
I wonder if there is a C/C++ compiler collection in a package inside the Windows install folders, just like the ones in Linux, I just need to install.
What also gives me a reason to ask this question is:
Since the kernel of Windows and the Windows API are written in C, and many of the high-level applications of the actual Windows 10 release are written in C++, it would be reasonable to also directly provide a suitable compiler suite. This is my thought modell, does not need to match reality.
Thanks for your help.
Since the kernel of Windows and the Windows API are written in C
Microsoft doesn't ship a compiler, or the required Windows SDK headers/libs (also includes a bunch of other useful development tools) for Windows in the installation.
Microsoft Visual C++ (part of Visual Studio) would be the equivalent "built in" choice although I am not sure if Microsoft ever specify exactly which version they use for a given Windows build and it is common to have lots of software built with different compilers/versions (including the various non-Microsoft ones).
As well as the full Visual Studio package with the IDE and other tools. Microsoft provide some components separately, such as the Build Tools for Visual Studio 2019.
I am sure this is for many reasons like most users not being interested in compiling their own software, and Microsoft still sells Visual Studio separately to larger organisations (historically to most serious users, but "Community" edition is now pretty nonrestrictive for individuals and small business).
Strangely enough it doesn't come with a C compiler, we need to install one, Mingw-w64 is allways my choice, you will need to add the path in environment variables (step 12), if you want to use it anywhere. You can then use the gcc command where it's more convenient like in Linux, don't forget to open a new cmd after the changes for them to take effect.
Unfortunately, Windows doesn't have the command line tools for installing/removing stuff and the great repository infrastructure we know and love from Linux.
You will need two things:
The command-line build tools. These can be found on the Visual Studio Downloads page under Tools for Visual Studio -> Build Tools for Visual Studio. This will include the compiler (cl.exe) and linker (link.exe) for the MSVC build toolchain.
The Windows SDK. The latest version of the SDK can currently be found here. This page has a tendency to move around, but googling for Windows SDK usually gives you the right page immediately. The SDK contains all the headers and libraries required to build Windows applications and make use of the Windows native API. The Windows SDK contains a lot of stuff which you may or may not need. You will almost certainly want to install the Windows SDK for Desktop C++ x86 Apps and Windows SDK for Desktop C++ amd64 Apps components. Most of the other stuff should be optional, but some of it is nonetheless highly useful.
You may want to download additional packages such as the Windows Debugger (which is an entirely different application than the Visual Studio debugger) or the Driver SDK, depending on what kind of things you want to develop.
Note that even if you don't intend to use the IDE in the end, installing the full Community Edition of Visual Studio is a far more convenient way to get a working build environment, so unless you have a really good reason not to, just go with the full package and choose to never open the IDE.
The built in compilers available on Windows 10 are for VisualBasic, C#, JScript.
To improve speed & performance of apps, "ngen.exe creates native images, which are files containing compiled processor-specific machine code, and installs them into the native image cache on the local computer. The runtime can use native images from the cache instead of using the just-in-time (JIT) compiler to compile the original assembly".
For low level programmers, ilasm.exe (IL Assembler) ships with Windows, which also facilitates "tool and compiler" development; so you could even create your own language or build a better compiler for a current one, or "debug your code at low level and understand how .NET deals with your high level code", or "write your own compiler for a new .NET language."
For web programmers, AspNetCompiler precompiles server-side ASP.NET web-applications, therefore helps application performance because end users do not encounter a delay on the first request to the application.
All Compilers & assemblers come as builtin with Windows without IDE and can be run from "the Windows Command prompt (cmd.exe)", so no extra downloads necessary; located in folder: C:\Windows\Microsoft.NET\Framework\vx.x.xxxxx\ .
Note: C# is based on JScript.
Compilers:
vbc.exe
csc.exe
jsc.exe
ilasm.exe
ngen.exe
aspnet_compiler.exe
Addendum:
If you still are looking for a C compiler to handle some C source codes you already have spent your life on, then (without downloading) you can make/write a C compiler, in a high level language, then optimize it with a low level language.
This guide will "introduce you to the high-level architecture, theory, and .NET Framework APIs that are required to build your own .NET compiler" in C#.
You can to use gcc for windows. For e.g. mingw, tdm-gcc, mingw-68, Cygwin etc. Each of them allows you to work on c/c++. But if you are not sure about the installation process, and don't want to do all the work by yourself, the easiest solution is to download something like dev-c++ or code-blocks.
cygwin: https://cygwin.com/
mingw-64: http://mingw-w64.org/doku.php
Downloadable file can be found here (for mingw, dev-c++): https://sourceforge.net/
You can also install Microsoft compiler. I usually do it by installing the whole visual studio.
I'm looking to implement a build server for Qt-based C++ projects. The server needs to check out the necessary code / assets from Subversion, build the executable files, assemble the artifacts for installation projects, and build the installation media files. The target platforms and (rough) toolchains are:
Windows (32- and 64-bit): qmake, nmake, msbuild, wix toolchain. The end result is an installer EXE and DVD image.
Mac OS X: qmake, make, custom bash scripts to assemble package. The end result is an application bundle within a disk image and a DVD image.
Ubuntu (32- and 64-bit): qmake, make, debuild-based scripts. The end result is a collection of DEB files and a DVD image.
Fedora (32- and 64-bit): qmake, make, rpmbuild-based scripts. The end result is a collection of RPM files and a DVD image.
So that's at least 4 build agents (maybe more if 32- and 64-bit can't be done on the same box) and 7 configurations. Open-source projects are preferred, but that is not an absolute requirement.
Most of the tools I'm seeing seem to be catered to Java (Jenkins, CruiseControl, etc.) or .Net (CruiseControl.net, etc.) Can those be used with a C++ toolchain, or will I constantly be fighting the system? Anything that you have used in the past and found works well with Qt / C++?
I use Jenkins for building and packaging many C++ projects, based on qmake, cmake, and makefiles.
There are plugins for cmake, qmake, and msbuild, but any command line scripts can be run as well.
I have done packaging using Jenkins with no problems, as it is just another command line step in a project.
There are good plugins for monitoring the number of warnings/errors produced by the compiler (I normally use GCC).
It also has matrix builds which allow you to build a project several times with different combinations of compiler flags, pre-processor variables, platform, etc. One project I set up is a matrix build with 5 boolean preprocessor flags on two platforms, which then does 2^6=64 builds. These can take a bit of setting up to get correct.
Here you can read a quick example:
Continuous Integration Server - Hudson
I think that Hudson, jenkins and builbot are worth a try. Wasting a day or two evaluating and trying them with a quick example will help you to choose confidently.
Most of the tools I'm seeing seem to be catered to Java (Jenkins, CruiseControl, etc.) or .Net (CruiseControl.net, etc.) Can those be used with a C++ toolchain, or will I constantly be fighting the system? Anything that you have used in the past and found works well with Qt / C++?
Any reasonably capable CI system will have a piece that will allow you to execute any program you want for your build command.
Here's what I would consider:
Does the CI system run on your system(s) of choice
Does it allow you an easy way to view your logs
Does it integrate with your test runner
Does it integrate with your code coverage reports (e.g. BullseyeCoverage w/C++ & Qt)
Will it publish your files in a manner sensible for your needs
Will at provide an archive/store of files, if necessary (e.g. pdbs & lib*.so.debug)
If the CI system doesn't support feature X, will you have to write it for each supported OS/system
Is the CI system / UI easy for you to use.
I did the above using CruiseControl and most things were pretty easy. I wrote everything in make or qmake and simply called out to the command that I needed executed. For unit test and code coverage integration I output stuff to XML and transformed it to something supported by CruiseControl.
My recommendation, take a look at the recommended CI systems and examine them based on the criteria above.
I'm using buildbot for this. I've been using it for 4 years, and I feel very happy with it.
It is an application written in python, that runs on a server and can manage multiple clients on various OSes. I'm currently using Windows XP, Windows 7, Debian, Ubuntu and CentOS build slaves. My projects are C++, and one of them (the end user GUI) is made in Python. But we've also integrated with other frameworks, for other features than GUI.
What is really good about buildbot is that it works by running command lines on slaves. With this, you can do whatever you want. Even on Windows systems to compile using Visual Studio! From these command lines, you get all the output centralized on the server, and accessible.
You may also find alternatives on this site that references many of them.
Disclaimer: I looked at it 3 years ago, I don't know if it is still accurate.
Hudson or Jenkins is pretty good.
Jenkins is indeed pretty popular for developing such a custom service, even after all these years, considering the question is already 7 years old.
Felgo also offers a Continuous Integration and Delivery (CI/CD) service for Qt. It supports desktop platforms as well as iOS, Android and embedded targets. The full feature set is described in the blog post.
Disclaimer: I am a software developer at Felgo
I'm working on a cross-platform C/C++ code base that has Visual C++ (super majority) & XCode developers. It also needs to compile on Linux, because that's where it's deployed. We are currently using a complicated Unix makefile that's called from Visual C++, XCode, & Unix command line.
However, the makefile project causes several productivity losses for Visual C++ developers:
Slower build times
Intellisense & text search don't work well for files not directly referenced in project
No .h dependency generation (not clear how to do on Windows)
Adding a native Visual C++ project has the following downsides:
lots of work to manage all those separate platform configuration as mentioned here Maybe VC++ 2010's hierarchical property sheets will help.
more work due to syncing makefile with native project
Currently, I'm considering to add a native project for Windows developers. Can anyone offer their experience on what's best or suggest how those problems with either approach can be reduced.
I have considered CMake and personally would use it, but it's going to be hard to convince other people to learn cmake & syncing it with the native projects would be an issue.
I have considered CMake and personally would use it, but it's going to be hard to convince other people to learn cmake & syncing it with the native projects would be an issue.
The nice thing about CMake is that it builds ALL of the configurations for you. You would setup a single CMake project, and then use it to generate VS solutions, XCode projects, and unix makefiles for you.
It's a huge improvement - everybody gets to work in their "native" environment, whether they're on Windows, Mac, or Unix.
I've handled this by writing my own converter that would synch scripts for various embedded development environments. If something like the above answer of using CMake does all that for you, then that would be the way to go. But if its too complex, rolling your own custom tool is not that hard. Just has to synch both ways.
Also, if you don't have it. Visual Assist for visual studio is awesome :)
I am not a fan of makefile generators, cmake, qmake and the like.
I am a fan of make. We have a single makefile (well, several actually but they include each other and there is no recursive make involved) which knows all of the project dependencies on all platforms (windows, wince, linux, mac, ...). This gives:
Makefile is really no more than a list of sources (include dependencies are auto-generated during the build)
Same build command on each platform, apart from specifying the tool chain
Build uses all my CPUs
Very short time-to-do-nothing
Developers can use whatever IDE they fancy
vim, emacs, qtcreator, eclipse, XCode all in use
I often use VC++ when on Windows (for the debugger)
Scriptable. Great way to automate your tests
Nice.
I am working on my free time on a multiplatform/multi-architecture library written in C++.
Before every release, I have to boot up several computers (One on Windows, one on Linux, another one on Mac OS, ...) just to make sure the code compiles and runs fine on every platform.
So I decided to create my own buildmachine but I really don't know what tools exist to do this. I'd like my buildmachine to run on Linux but any other solution will be accepted.
Ideally, I would just have to click on a "Build all" button, and it would compile my library for the different platforms/architectures, generate archives from the result and/or report potential errors.
My project "constraints" are:
It is written in C++
It compiles on Windows using SConstruct/MinGW and Visual Studio 2010
It compile on Linux and Mac OS using SConstruct/g++
The sources are stored into Subversion (svn)
Do you know any tool/set of tools that could help me achieving my goal ?
Thank you very much.
I would setup 3 VMs (VirtualBox is free), one for each platform.
Install TeamCity (or Hudson) on Linux and agents on the other VMs and then it's just a matter of configuring the build system.
At the very basic level you should have 2 tasks: one to checkout the sources from Subversion and another to invoke scons.
I'm not too familiar with Hudson but TeamCity is certainly capable of generating reports of a build, display progress etc.
I'm involved in C++ project targeted for Windows and Linux (RHEL) platforms. Till now the development was purely done on Visual Studio 2008. For Linux compilation we used 3rd party Visual Studio plugin, which read VS solution/perojects files and remotely compiled on Linux machine.
Recently the decision was to abandon the 3rd party plugin.
Now my big concern is a build system. I was looking around for cross platform build tools. This way I don't need to maintain two set of build files (e.g. vcproj/solution for Windows and make files for Linux).
I found the following candidates:
a. Scons
b. cmake
What do you think about the tools for cross-platfrom development?
Yet another point that bothers me is that Visual Studio (+ Visual Assist) will loose a lot functionality without vcproj files - how you handle the issue with the tools?
Thanks
Dima
PS 1: Something that I like about Scons is that it
(a) uses python and hence it's flexible, while cmake uses propriety language (I understand that it's not a winner feature for a build-system) (b) self contained (no need to generate makefiles on Linux as with cmake).
So why not Scons? Why in your projects the decision was to use cmake?
CMake will allow you to still use Visual Studio solutions and project files. Cmake doesn't build the source code itself, rather it generated build-files for you. For Linux this can be Code::Blocks, KDevelop or plain makefiles or still other more esoteric choices . For Windows it can be among others Visual Studio project files and still others for MacOS.
So Visual Studio solutions and projects are created from your CMakeLists.txt. This works for big projects just fine. E.g. current Ogre3d uses CMake for all platforms (Windows, Linux, MacOS and IPhone) and it works really well.
I don't know much about scons in that regard though, I only used to build one library and only in Linux. So I can't compare these two on fair ground. But for our multi-platform projects CMake is strong enough.
I haven't used Scons before, so can't say how that works, but CMake works pretty well.
It works by producing the build files needed for the platform you're targeting.
When used to target VC++, it produces solution and project files so from VS, it appears as if they were native VS projects. The only difference is, of course, that if you edit the project or solution directly through VS, the changes will be erased the next time you run CMake, as it overwrites your project/solution files.
So any changes have to be made to the CMake files instead.
We have a big number of core libraries and applications based on those libraries. We maintain a Makefile based build system on Linux and on Windows using the Visual Studio solution for each project or library.
We find it works well for our needs, each library or app is developed either on linux or windows with cross compilation in mind (e.g. don't use platform specific api's). We use boost for stuff like file paths, threads and so on. In specific cases we use templates/#defines to select platform specific solution (for example events). When is ready we move to the other system (linux or windows), recompile, fix warnings/errors and test.
Instead of spending time figuring out tools that can cross compile on both platforms we use system that is best for each platform and spend time fixing specific issues and making the software better.
We have GUI apps only on Windows atm. so there's no GUI to cross compile. Most of our development that is shared between Windows and Linux is server side networking (sockets, TCP/IP, UDP ...) and then client side tools on Linux and GUI apps on Windows.
Using with perforce for source code version management we find in quite many cases that the Linux Makefile system is much more flexible for what we need then Windows VS. Especially for using multiple workspaces (views of source code versions) where we need to point to common directories and so on. On Linux this can be done automatically running a script to update environment variables, on Visual Studio referencing environment variables is very inflexible because it's hard to update automatically between views/branches.
Re sync question:
I assume you are asking how to make sure that the two build systems get synchronized between linux and windows. We are actually using Hudson on Linux and CruiseControl on Windows (we had windows first with cruise control, when I went to setup linux version I figured Hudson is better so now we have mixed environment). Our systems are running all the time. When something is updated it is tested and released (either windows or linux version) so you would know right away if it does not work. During testing we make sure all the latest features are there and fully functional. I guess that's it, no dark magic involved.
Oh you mean build scripts ... Each application has it's own solution, in solution you setup up dependencies. On Linux side I have a makefile for each project and a build script in project directory that takes care of all dependencies, this mostly means build core libraries and couple of specific frameworks required for given app. As you can see this is different for each platform, it is easy to add line to build script that changes to directory and makes required project.
It helps to have projects setup in consistent way.
On Windows you open project and add dependency project. Again no magic involved. I see this kind of tasks as development related, for example you added new functionality to a project and have to link in the frameworks and headers. So from my point of view there is no reason to automate these - as they are part of what developers do when they implement features.
Another options is premake. It's like cmake in that it generates solutions from definition files. It's open source and the latest version is very highly customizable using Lua scripting. We were able to add custom platform support without too much trouble. For your situation it has support for both Visual Studio and GNU makefiles standard.
See Premake 4.0 Homepage
CruiseControl is a good choice for continuous integration. We have it running on Linux using Mono with success.
Here is an article about the decision made by KDE developers to choose CMake over SCons. However I've to point that this article is almost three years old, so scons should have improved.
Here is comparison of SCons with other building tools.
Had to do this a lot in the past. What we did is use gnu make for virtually everything including windows at times.
You can use the project files under windows if you prefer and use gnu make for Linux.
There isn't really a nice way to write cross platform makefiles because the target file will
be different among other things (and pathname issues, \ vs / etc). In general, you'll probably be tweaking the code across the various platforms to take subtle differences into account, so a tweak to a make file and checking on the other platforms would have to happen
anyway.
Many OS projects maintain Makefiles for different platforms such as zlib where they are named like Makefile.win, Makefile.linux etc. You could follow their lead.