I need information regarding distributed build with Jenkins. The distribution i need is not the normal Jenkins distributed build (Master/slave config) where it acts like a load balancer so that the job will get executed on the available node.
For cpp projects , there are tools like distcc,netcc etc to distribute build across several machines on network so that the compilation will be fast. Is there any similar tools or way that we can use inorder to reduce the build timing?
thanks in advance
Jenkins is not a compiler - it is merely a coordinator for software build activities.
There is nothing stopping you from using distcc or similar in a build script that Jenkins starts, and the compiler nodes does not need to be aware of the fact that Jenkins started it.
If you have a distributed compiler and can make use of it from your command prompt, it can be called from a Jenkins job as well.
Related
We are building several project with TeamCity. In addition to an agent on the main server which runs on Linux, we also have three additional agents that run on separate boxes. One on Linux, one on Mac and one on Windows.
If all agents are idle, the first available agent, as they are listed, is chosen for the build. This means that if the load on TeamCity is small, the same agent is always used. We have had situations where a project had been built successfully by the same Linux agent for more than 50 builds, then when it finally had a run on the Windows agent, a test failed, due to code that was committed fairly early in the run of the 50 builds.
Since many of our tests may be affected by the environment, we are looking for ways to spread the builds on the agents, automatically. Is there any way of setting up a round robin agent selection policy? Or any other way to spread the builds on the agents?
You can have a schedule trigger that runs a build on all agents.
Or configure a build for each platform (linux,windows, mac, ...) that will run on a specific agent, selected with some specific requirements.
The answer to my question, at least for TeamCity 8 and earlier, is NO.
See JetBrains own TeamCity Developer forum: https://devnet.jetbrains.com/message/5533629
I would like to ask about your experience with build server for embedded systems. What are you using (if any), and what are good and bad sides.
We are developing mainly for microcontrollers without operating system.
At this moment I'm trying to use Jenkins and my build is running. But I have some problem with projects structure. When I want all plugins working, than I need flat job structure. But we have few projects that are developed in parallel, and then job view start to be messy.
I've tried folders, but than some plugins stopped working.
I would like to build a pipeline, that is running sequential, but have parallel jobs inside. eg. Commit stage have: compile, lint check, style check, unit tests. all of them can run in parallel and when all are successful next stage is executed.
What I need from Build server at this moment:
build pipeline support
user authorization based on LDAP
parallel job execution
hierarchical projects (projects/configurations groups)
reports from xUnit, Lint, Compiler warnings, Robot framework.
slave/agents support, tags for slave
privileges based on ldap groups
privileges per group/project
I'm opened for any suggestions, open source and commercial.
I was looking at Bamboo on videos look very nice but I didn't try it yet.
We have two development teams, that are developing different projects. It could be nice to have projects grouped for teams and privileges for group. Members of one group shouldn't modify builds of other. But it is more "nice to have" than "must have".
TeamCity
I tried to use TeamCity. Building build pipeline is easier than in Jenkins, just click add Step.
One thing that I found difficult is making steps in parallel in one configuration. For example after commit I would like to run in parallel Lint, Unit tests, Compile to save some time. I found solution, but it make pipeline harder to view and maintain.
TeamCity support multiple configuration in projects which solve problem with jobs grouping. I didn't found option to group projects.
TeamCity is a free, Java-based CI server from JetBrains. We've been using it very successfully (for very different kinds of projects) and I would unreservedly recommend it to you. To each of your requirements:
Build pipelines are configured as a series of steps within a build configuration. A project can have an arbitrary number of configurations, which in turn can have an arbitrary number of steps.
LDAP integration is fully supported.
Build pipelines can be executed in parallel. TeamCity delegates work to Build Agents, which are typically distinct servers that have all the necessary tools (frameworks, etc.) to perform the steps of a build configuration. The free version of TeamCity comes with licenses for three agents, so you could have up to three builds running in parallel. Additional agents can be licensed for a nominal fee.
By 'hierarchical projects' I understand you to mean that the completion of one build pipeline will automatically trigger the start of a subsequent pipeline. This is supported, and build/version numbers can be passed between the stages for consistency.
XUnit has first-class support. Lint/compiler reports can be saved as 'artifacts' of the build for easy review later. Essentially, a lot of frameworks have built-in support in TeamCity, and for everything else you can execute arbitrary shell commands, the output of which can be saved as artifacts or used in subsequent build steps.
Slave/agent support is central to the TeamCity model, as noted above.
All of this is highly configurable and customizable. We've been able to do a lot of diverse, complex things with TeamCity, and it has been totally solid and stable for us. And it looks good, too -- the server dashboard arranges information in an easily-understood way.
Disclaimer: I work for Atlassian so I'm a bit biased.
Configuring your build pipeline in Bamboo is pretty easy to do. Bamboo operates based on a Plan → Stage → Job structure, listed from higher to lower order. Check out the Bamboo Plan Structure.
Every Project in Bamboo holds a collection of Plans. Plans are comprised of one or more Stages. Stages run sequentially and are comprised of one or more Jobs. Jobs run in parallel and are comprised of one or more Tasks (tasks run sequentially, but can be placed in separate Jobs so that they run in parallel and speed up build time). Agents in Bamboo are machines or services that perform your build steps. An entire Job will execute on a single Agent. You can read more about Agents here. As for slave tags, the ability to make certain agents exclusively tied to certain builds or projects is on the short-list for new features.
To answer your other points:
user authorization based on LDAP/privileges based on ldap groups/project: You can connect to an external LDAP server to manage users and permissions. Bamboo has a groups feature or if your team is using JIRA you can take advantage of JIRA groups to set up global permissions, plan permissions, and also indicate which users will receive notifications on a plan’s build results. Global permissions control who has access to build plans and the Bamboo server whereas Plan permissions control who can perform specific operations on a Plan and its Jobs.
hierarchical projects (projects/configurations groups): Bamboo does support parent & child plan structure. There are several ways you can set up triggering for builds. One of them is to base triggers on other builds, that is, Plan builds are triggered by preceding successful builds of other plans or if other specified plans are building successfully. Example: If Plan A builds successfully it will automatically trigger builds of Plans B & C.
reports from xUnit, Lint, Compiler warnings, Robot framework: Bamboo can run any build process that can be started from a command line. Support includes Maven/Maven2, Ant, make, MSBuild, NAnt, Grails, devenv.exe, and any xUnix-compiant framework (JUnit, Selenium, JWebUnit, NUnit, PHPUnit, etc...).
We use Jenkins for our CI build system. We also use 'concurrent builds' so that Jenkins will build each change independently. This means we often have 5 or 6 builds of the same job running simultaneously. To accommodate this, we have 4 slaves each with 12 executors.
The problem is that Jenkins doesn't really 'load balance' among its slaves. It tries to build a job on the same slave that it previously built on (presumably to reduce the time syncing from source control). This is a problem because Jenkins will build all 6 instances of our build on the same slave (or more likely between 2 slaves). One build machine gets bogged down and runs very slowly while the rest of them sit idle.
How do I configure the load balancing behavior of Jenkins, and how it controls its slaves?
We were facing a similar issue. So I've put together a plugin that changes the Load Balancer in Jenkins to select a node that currently has the least load - https://plugins.jenkins.io/leastload/
Any feedback is appreciated.
If you do not find a plugin that does it automatically, here's an idea of what you can do:
Install Node Label Parameter plugin
Add SLAVE parameter to your jobs
Restrict jobs to run on ${SLAVE}
Add a trigger job that will do the following:
Analyze load distribution via a System Groovy Script and decide on which node to start next build.
Dispatch the build on that node with Parameterized Trigger
plugin
by assigning appropriate value to SLAVE parameter.
In order to analyze load distribution you need to install Groovy plugin and familiarize yourself with Jenkins Main Module API. Here are some useful initial pointers.
If your build machines cannot comfortably handle more than 1 build, why configure them with 12 executors? If that is indeed the case, you should reduce the number of executors to 1. My Jenkins has 30 slaves, each with 1 executor.
You may also use the Throttle Concurrent Builds plugin to restrict how many instances of a job can run in parallel on the same node
I have two labels -- one for small tasks and one for big tasks. I have one executor for the big task and 4 for the small tasks. This does balance things a little.
I am a Qt/C++ developer. I would like to setup a continuous integration environment whereby after committing the source code, it triggers a build process that build the code for the 3 platforms I'm using:
Linux
OS X
Win32
If possible, how do I setup such environment. Any hints or links are welcome.
I've read around about Jenkins, but I can't find any good tutorial for it.
I also suggest Jenkins for several reasons:
It will run on all of the platforms you listed.
It can be configured to start a build when the repository is updated (hint: configure the Job to "Poll SCM" and you won't have to muck with your SCM tool to get it to tell Jenkins to start building).
It provides good support (mostly through plugins) for Unit Testing. [You're project is doing unit testing, right?]
The price is right
A bigger issue is going to have is that AFAIK, Qt doesn't really do cross-compiling for other platforms well. Using Jenkins (and the appropriate plugins), you should be able to solve this.
One method that comes quickly to mind is to have an instance of Jenkins on each platform. Each instance is responsible for building the version for its own platform. At the end of the build, the created artifacts are all put into a common, shared location.
Jenkins supports this feature via plugins for all major source control systems. If you seriously considering using Jenkins (and I would highly recommend it), consider buying John Ferguson Smart's Jenkins: The Definitive Guide.
Two solutions coming to my mind:
BuildBot
BuildBot is a highly customizable continuous integration system written in Python. The master component offers a nice web-based GUI to monitor and trigger builds; slave components are put on the target machines (usually virtual machines but they could be the Mac laptop of one of the developers). Docs are good enough to build up a basic system, customization could be a little tricky (at least it was for me). Using commit/push hooks provided by VC systems you can easily activate the master and trigger builds across the slaves. It also supports incremental builds (a must if your project is big).
CDash
Developed by the authors of CMake, CDash is a web application collecting builds coming from across the network, not exactly what you asked for but I think it's worth a try. Very powerful if you have a team of developers who could continuosly submit build result on their machines to the server (and if you use CMake it's almost transparent). You cannot trigger builds from the server as Buildbot does, but you could setup a bunch of VM with a cron which checks for changes and in case performs the build and sends results to CDash
Sure it's possible. Most of the version control systems are able to execute custom script on server side. Some of them (git, for example), has hooks to achieve the same locally. Have a look at git's post-commit hook.
All you need is to create a script that will trigger cross-platform builds.
Most version control systems allow post-commit hooks to allow you to kick off events like builds. Alternatively build systems can be configured to regularly poll a source control repository and manage their own build scheduling (this is how we use Jenkins).
Something to bear in mind is how long it will take to do a complete build across platforms and the typical number of check-ins in that interval. You might find batching check-ins a better way of doing continuous integration builds if you have an fair sized team or limited build server resources. Otherwise your build system could quickly end up trying to play catch up.
As for whether it is possible to build on all target platforms, that depends on your tool chain.
Quite a few build and CI systems support steps for pushing build output to Azure, but I haven't seen any which can actually run on Azure (or EC2). Ideally I would like to be able to spin up an arbitrary number of instances (depending on the # of pending submits) to deal with the actual build + quality gates (UTs, FXCop, other static analysis tools) + source repository checkin process.
Are there existing tools which can do this, or has anyone built something which they can discuss?
Thanks!
[Edit: I found this question which is quite similar but didn't have any informative answers, so I'll keep my question alive]
If you're using Git or Mercurial for source control, AppHarbor might be what you're looking for. It's a CI build/deploy environment that runs exclusively in the cloud (EC2), and can deploy build output to Azure.
Here are some links for reference:
http://sourcecodebean.com/archives/appharbor-heroku-for-net/987
http://lostechies.com/chrismissal/2011/03/12/using-appharbor-for-continuous-integration
http://haacked.com/archive/2011/05/12/making-let-me-bing-that-for-you-open-source.aspx
http://appharbor.com/page/pricing
The open souce Jenkins CI server has an EC2 plugin that will spin up EC2 instances automatically depending on your build load. I couldn't find anything for Azure, but I highly recommend Jenkins - it's easy to configure, well maintained and has stacks of features.
Continuous Integration on Windows Azure http://code.google.com/p/cassis/ (over Mercurial)
Disclaimer: work produced by my 1st year CS students
Also Teamcity has support for this: http://www.jetbrains.com/teamcity/features/amazon_ec2.html