Related
I'm currently using jenkins/hudson for continuous integration a large mostly C++ project. We have separate projects for trunk and every branch. Also, there are some related projects for the Java code, but the setup for those are fairly basic right now (we may do more later though). The C++ projects do the following:
Builds everything with options for whether to reconfigure, do a clean build, or use a fresh checkout
Optionally builds and runs all tests
Optionally runs all tests using Valgrind's memcheck
Runs cppcheck
Generates doxygen documentation
Publishes reports: unit tests, valgrind, cppcheck, compiler warnings, SLOC, open tasks, and code coverage (using gcov, gcovr, and the cobertura plugin)
Deploys code nightly or on demand to a test environment and a package repository
Everything is configurable for automatic builds and optional for on demand builds. Underneath, there's a bash script that controls much of this, which farther depends on our build system, which uses automake and autoconf along with custom bash scripts.
We started using Hudson (at the time) because that's what the Java guys were using and we just wanted nightly builds. Since then, we've added a lot more and continue to add more. In some ways Hudson is great, but certainly isn't ideal.
I've looked at other solutions and the only one that looks like it could be a replacement is buildbot. Would buildbot be better for this situation? Is the investment worth it since we're already using Hudson? Why?
EDIT: Someone asked why I haven't found Hudson/Jenkins to be ideal. The short answer is that everything can be improved. I'm simply wondering if Jenkins is the best current solution for my use case or whether there is something better (buildbot?) that would be easier to maintain in the long run even as new requirements come up.
Both are open source projects, but you do not need to change buildbot code to "extend" it, it is actually quite easy to import your own packages in its configuration in which you can sub-class most of the features with your own additions. Examples: your own compilation or test code, some parsing of outputs/errors to be given to the next steps, your own formating of alert emails etc. there are lots of possibilities.
Generally I would say that buildbot is the most "general purpose" automatic builds tools. Jenkins however might be the best related to running tests, especially for parsing and presenting results in nice ways (results, details, charts.. some clicks away), things that buildbot does not do "out-of-the-box". I'm actually thinking of using both to have sexier test result pages.. :-)
Also as a rule of thumb it should not be difficult to create a new tool's config: if the specification of what to do (configs, builds, tests) is too hard to switch from one tool to another, it is a (bad) sign that not enough configuration scripts are moved to the sources. Buildbot (or Jenkins) should only call simple commands. If it is simple to run tests, then developers will do it as well and this will improve the success rate, whereas if only the continuous integration system runs the tests, you will be running after it to fix the new code failures, and will loose its non-regression value, just my 0.02€ :-)
Hope it'll help.
The 'result integration' is also in jenkins/hudson, and you can relatively easily capture build products without having to 'copy them elsewhere'.
For our instance, the coverage reports and unit test metrics and javadoc for the java code is all integrated. For our C++ code, the plugins are a little lacking, but you can still get most of it.
we ran buildbot since pre 0.7, and are now running 0.8 and are only now seeing any real reason to switch, as buildbot 0.8 forgot about windows slaves for an extended period of time and the support was pretty poor.
There are many other solutions out there, besides Jenkins/Hudson/BuildBot:
TeamCity by Jetbrains
Bamboo by Atlassian
Go by Thoughtworks
Cruise Control
OpenMake Meister
The specifics about what you are doing are not so important, in fact, as long as the agents (aka nodes) that you are doing them on support those tasks.
The beauty of a CI server is noticing when the build changes to trigger a new build (and test), publish the artifacts, and publish test results.
When you compare CI tools like those we mentioned, consider features like the usability of its interface, how easy is branching (and features it might offer like automatic merging), notifications (like XMPP/jabber), or an information-radiator (like hooking up a monitor to always show status). Product support is another thing to consider - Jenkins' support is only as good as who is responding to community questions at the time you have questions.
My personal favorite is Bamboo, but it comes with a license fee.
I'm a long-time Jenkins user in the middle of evaluating Buildbot and would like to offer a few items for folks considering using Buildbot for multi-module solutions:
*) Buildbot doesn't have any out-of-the-box concept of file artifacts related to each build. It's not in the UI and it's not in any of the builtin "steps" modules as far as I can see:
http://docs.buildbot.net/current/manual/configuration/buildsteps.html
...and I see no third party plugin:
https://github.com/buildbot/buildbot/wiki/PluginList#steps
Buildbot does collect all the console output from a given build, but critically, you can't collect files related to it.
*) Given that artifacts are not supported, it's not easy to create "collector" projects that bring multiple modules into say, a single installer. Jenkins has a great feature that lets you parameterize a build with builds from other modules (the parameter type is a run).
*) Establishing dependencies between modules is trickier in Buildbot. Say you have a library that three binaries depend on, and you want those binaries to rebuild each time the library changes. Jenkins has triggers built into the UI. If you want to do triggers in Buildbot you have to script them using schedulers.Dependent, and it causes a lot of item congestion in the Schedulers UI.
*) When you're working in Buildbot, it seems that pretty much all of the configuration is done in master.cfg in code. This is awesome and frustrating.
*) Buildbot forces you to create a worker in addition to a master server. This is annoying for beginners and systems for which a single build server is sufficient.
My impression after two days of Buildbot evaluation is that we'll stick with Jenkins, primarily due to it having artifacts. Buildbot is a tool we'd only use if we had more extensive customization needs, and the time to do it.
On the subject of buildbot and artifacts -- I don't have enough user score to make a comment -- you can get artifacts from buildbot 2.x series pretty easy with built-in file/directory upload actions. However you rarely want to just move files. Typically you make a triggered buildstep that does deployment directly off the worker for best results. eg push to cloud storage, containers, thirdparty (steam uploads), etc.
This way you can get metrics on the uploads and conditionally control them better (or even mix and match artifacts across worker machines).
I have a bat script which calls out to an exe and starts this off - I dont know what language it is written in (Probably C++)
I want to find a profiler that allows me to run a command a few times and gathers IO/CPU and other stats
It needs to be open source where possible
Any suggestions
You may try Performance Analysis of Logs (PAL) - http://pal.codeplex.com
It has predefined templates for major system metrics (see System Overview template).
It may automatically create quite impressive report including graphs and brief explanations on detected 'hotspots'.
I have used Glowcode (www.glowcode.com) in the past. It's not open source, but quite easy to use, can profile an executable, and is available on an evaluation basis as well.
I recall it was quite cheap as well.
For Windows, XPERF is your friend, and is a free download.
Check out the Windows Performace Analysis Tools site for more information on this and other tools.
Have fun!
I just downloaded and built the libraries/executables of Google Performance Tools. Before I run the CPU profiler on the application that I want to investigate, I want to learn how to use the tools properly perhaps on a sample application. What would be a good example to run the Google CPU profiler on? Thanks in advance.
The following paragraph appears in the README.windows file distributed with perftools 1.3:
The heap-profiler has had a preliminary port to Windows. It has not been well tested, and probably does not work at all when Frame Pointer Optimization (FPO) is enabled -- that is, in release mode. The other features of perftools, such as the cpu-profiler and leak-checker, have not yet been ported to Windows at all.
In my experience, for performance tuning, stack-sampling is the method of choice.
Google perftools contains a stack-sampler, and I believe its visual analyzer can be made to show the cost of individual statements, not just functions.
What you need to know is the percent of time the stack contains that statement, because that is how much time would be saved if the statement were removed.
I think everyone here would agree that in order to be considered a professional software house there are number fundamental things you must have in place.
There is no doubt that one of these things is a build server, the question is, how far do you need to go.
What are the minimum requirements for the build server? (Somewhere to just compile?)
What is the ultimate goal for your build server? (Scheduled, source control integration, auto deployment to test / live servers)
Where is a good place to start assuming you have nothing at the moment?
It would be great if we could list out a few simple tasks that an amateur developer could take on board in order to set them on the right track to a fully functional build server.
It would also be good to hear about people that feel they have a "complete" system setup that performs all the functionality they require and how they went about setting it all up from scratch.
You can start by looking into Cruise Control.
There's also CruiseControl.net if that's your poison.
Essentially though, you need the following ingredients:
A dedicated environment (Virtual Machine/server. Don't use a developer's machine, unless it's just you. Even then, run a VM if you can. Much easier to move it to a server when/if one becomes available in your organisation)
A source control system that supports labelled/tagged revisions (for example, Subversion+TortoiseSVN)
Build scripts. These can be batchfiles that start the devenv.exe or msbuild.exe applications with a command line, or you can use something like Ant or NAnt.
In this scenario, CruiseControl acts as the Continous Integration server, and can make sure that you have builds done as you check in your code. This means you know whether the build is broken quicker than if you just had nightly builds. You should probably also have nightly builds, though.
Hudson is a great CI.
We run farm locally, but we started by downloading hudson.war and doing
java -jar hudson.war
It integrates with SCM, bug trucking systems it is really awesome.
You'll need some disk space if you want to keep old build.
Enjoy it is most straightforward CI solution so far.
HTH,
Hubert.
If you're using Cruise Control, the place to start is an Ant build.xml that does the job manually.
You need a version control system that can do labeled check-outs.
You need JUnit tests to run using the Ant task and generate HTML reports.
Id say you'd have to start by implementing a build strategy so you can build your code in a structured way - I use NANT.
For a basic build server - use one of the CI offerings out there that monitors your source control and triggers a build whenever a change is detected. eg: cruiseControl.
Once you get the basic build together - add the running of your unit tests after a successfuly build.
The most successful system i've had in place had 3 different builds :-
- one that fired on a check in - all this did was build the code.
- an on demand one that would build the application, generate the installer and then put
the installer into a shared drive for the testers to pick up
- a daily build that fired at 10pm. This:
- ran some code generation to build DB and C# code from a UML model
- build the code
- created a new build verification test user on a test oracle instance
- ran the application schema into the db
- fired off a bunch of unit tests
- cleaned up the db user (if the tests were successful)
- ran coverage analysis to build a report of the unit code coverage
Software we used for this was NANT, CruiseControl.NET, a custom code generation system, custom app to build an oracle schema, and NCover for the code analysis.
Start by having a read of Martin Fowler's excellent paper on Continuous Integration.
We built such a system for a major project >2,000 kSLOC and it proved itself to be invaluable.
HTH
cheers,
Rob
Cruise, Maven, Hudson etc are all great but its always worth having a stopgap solution.
You should have a batch file, shell script or simply written instructions that will allow you to run a build from any machine. We have had build servers unavailable in the past and the ability to switch quickly to another machine was invaluable!
The spec of the build machine need not be important unless you have a monster project. We try and keep our build times down to 10 minutes (including unit tests) and we have a pretty big project.
Don't be tempted to create or write your own build system because "none of the tools out there are good enough". All modern build systems allow you to write plugins to do custom stuff.
I'm using Cruisecontrol.NET and an msbuild buildscript.
I can use the buildscript manually so that I can get the latest version of the codebase, built the codebase very easily using the commandline. (This is very interesting if you are working on an application that consists of multiple solutions).
Next to that, my CruiseControl.NET buildserver uses this buildscript as well. It checks on a regular interval if there have been changes committed to the source-control.
If that happens, CC.NET performs the 'get-latest' task that I've defined in the buildscript, builds everything, executes unit-tests and performs a statical code analysis (fxcop).
My 'buildserver' is just an old workstation. It's a PIV, 3Ghz with 1gb RAM, and it does its job perfectly.
One additional thing that I would find interesting, is to have the ability to automatically deploy a new version, or build a setup.
I haven't done that yet, since I'm not sure whether it is a good idea, nor have I found a good strategy yet to do so ...
I mean; is deploying a new version of some components into production for a mission-critical application a good idea ? I don't think so ...
I think this is a good place to start:
[http://confluence.public.thoughtworks.org/display/CC/Home;jsessionid=5201DA7E8D361EB164C40E519DA0F0DE][1]
At least, that's where I started looking when setting up my build server. :)
[1]: Home of CruiseControl
Roughly in order - minimal/least sophisticated through more sophisticated
able to get a specific set of source onto any machine
able to build that source (with no problems)
able to (schedule) build each night/or some other defined period with no user intervention
One (or more) dedicated build server (not shared as qa or dev machine)
able to do a build after each check-in/commit
Notify interested parties of the build status after a build
Provide build status at any time
Create installers as part of the build
ability to deploy/live if build is good
Run unit tests
Run tests on the product
Report the results of those tests
Static code analysis and reporting
...
And the list goes on and on
Don't be afraid to just start with batch files or shell scripts or other ad-hoc means. People made perfectly good software before the CI craze. there were plenty of good processes before Hudson and Cruise Control - ( I am not knocking those or others - I use Hudson among others) - but don't miss the point - these things are here to help you - not become overbearing process)
I couldn't give you all the details about how we set our build server up (I was only involved at the start), but:
We started with an in-house system, implemented in ASP.NET and a .NET Windows Service, using NAnt to do the actual builds. Actually, most of the workflow was implemented in NAnt (e.g. emailing people, copying stuff around, etc.).
We moved to JetBrains TeamCity (there's a free cut-down version available), which is still serving us well.
We use it for builds triggered by a commit: these just build the binaries and run the unit tests. From here, we can do a complete build, which does the MSI as well. From there, we have system test builds that run more in-depth tests, across an environment built with virtual machines (with a separate domain controller, SQL Server box, etc.). When the system tests pass, the build is made available to our QA department for manual testing and some regression tests that we've not automated yet.
In the java space I've tested most of the available build environments. The issue with automatic build is that you quite often end up spending a fair amount of time following it up. After we switched to the commercial bamboo from atlassian, we found that we have to spend a lot less time pampering the build box, which in our case turns out to be very good economy. Bamboo also supports clustering, so you can add inexpensive boxes as needs evolves.
Try & find something that fits in with your existing practices in terms of building - e.g. it's not going to be a good fit to try & use an Ant-based buildserver if you're using Maven, for instance!
Ideally, it should just be able to monitor your source-control system, checkout the code, build, run some tests & publish the results without you being aware of it, or at least not 'till it's reporting a failure. Personally, I'd suggest Hudson (https://hudson.dev.java.net/) as a good starting point as it's easy to get installed & running & has a decent UI.
We start by writing batch scripts that will run on the developers machine. Once we have all the processes automated, we move them to the build server.
On the tools side we are currently moving from Cruise Control to TFS.
Can anyone list the steps needed to programatically install an application on Windows. Aside from copying the files where they need to be, what are the additional steps needed so that your app will be a first-class citizen in Windows (i.e. show up in the programs list, uninstall list...etc.)
I tried to google this, but had no luck.
BTW: This is for an unmanaged c++ application (developed in Qt), so I'd rather not involve the .net framework if I don't have to.
I highly recommend NSIS. Open Source, very active development, and it's hard to match/beat its extensibility.
To add your program to the Add/Remove Programs (or Programs and Features) list, add the following reg keys:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\PROGRAM_NAME]
"DisplayName"="PROGRAM_NAME"
"Publisher"="COMPANY_NAME"
"UninstallString"="PATH_TO_UNINSTALL_PROGRAM"
"DisplayIcon"="PATH_TO_ICON_FILE"
"DisplayVersion"="VERSION"
"InstallLocation"="PATH_TO_INSTALLATION_LOCATION"
I think the theme to the answers you'll see here is that you should use an installation program and that you should not write the installer yourself. Use one of the many installer-makers, such as Inno Setup, InstallSheild, or anything else someone recommends.
If you try to write the installer yourself, you'll probably do it wrong. This isn't a slight against you personally. It's just that there are a lot of little details that an installer should consider, and a lot of things that can go wrong, and if you want to write the installer yourself, you're just going to have to get all those things right. That means lots of research and lots of testing on your part. Save yourself the trouble.
Besides copying files, installation tasks vary quite a bit depending on what your program needs. Maybe you need to put an icon on the Start menu; an installer tool should have a way to make that happen very easily, automatically filling in the install location that the customer chose earlier in the installation, and maybe even choosing the right local language for the shortcut's label.
You might need to create registry entries, such as for file associations or licensing. Your installer tool should already have an easy way to specify what keys and values to create or modify.
You might need to register a COM server. That's a common enough action that your installer tool probably has a way of specifying that as part of the post-file-copy operation.
If there are some actions that your chosen installer tool doesn't already provide for, the tool will probably offer a way to add custom actions, perhaps through a scripting language, or perhaps through linking external code from a DLL you would write that gets included with your installer. Custom actions might include downloading an update from a specific Web site, sending e-mail, or taking an inventory of what other products from your company are already installed.
A couple of final things that an installer tool should provide are ways to apply upgrades to an existing installation, and a way to uninstall the program, undoing all those installation tasks (deleting files, restoring backups, unregistering COM servers, etc.).
I've used Inno Setup to package my software for C++. It's very simple compared to heavy duty solutions such at InstallShield. Everything can be contained in a single setup.exe without creating all these crazy batch scripts and so on.
Check it out here: http://www.jrsoftware.org/isinfo.php
It sounds like you need to check out the Windows Installer system. If you need the nitty-gritty, see the official documentation. For news, read the installer team's blog. Finally, since you're a programmer, you probably want to build the installer as a programmer would. WiX 3.0 is my tool of choice - open source code, from Microsoft to boot. Start with this tutorial on WiX. It's good.
The GUI for innosetup (highly recommended) is Istool
You can also use the MSI installer built into Visual Studio, it's a steeper learning curve (ie is a pain) but is useful if you are installing software in a corporate environment.
To have your program show up in the Start program menu,
You would need to create folder
C:\Documents and Settings\All Users\Start Menu\Programs
and added a short cut to the program you want to launch.
(If you want your application be listed
directly in the Start menu, or in the programs submenu,
you would put your short cut in the respective directory)
To programically create a short cut you can use IShellLink
(See MSDN article).
Since you want to uninstall, that gets a lot more involved because you don't want to simply go deleting DLLs or other common files without checking dependencies.
I would recommend using a setup/installation generator, especially nowadays with Vista being so persnickety, it is getting rather complicated to roll your own installation
if you need anything more than a single executable and a start menu shortcut.
I have been using Paquet Builder setup generator for several years now.
(The registered version includes uninstall).
You've already got the main steps. One you left out is to install on the Start Menu and provide an option to create a desktop and/or quick launch icon.
I would encourage you to look into using a setup program, as suggested by Jeremy.