I have configured Jenkins and created a job to checkout, build and run a simple project from SVN. I could configure this job to run periodically, like once every five minutes. But I'd like it to build the project only when something has changed on the SVN repository. I read the "Builds by source changes" section of this document, but could not figure out what exactly I am meant to do! Any help would be appreciated!
When you configure your job you have to do this:
In the Source Code Management specify the source management system you use (for instance SVN) and fill all required fields (url, authentication, ...) (probably already done since you are able to do a checkout)
in the Build Triggers section : choose Poll SCM with a schedule */10 * * * * for checking the repository every 10 minutes.
Go to the configuration of your project and scroll down to the Build Triggers directly under the Source Code Management section. Here you've to configure it this way:
The syntax to schedule the job is in the crontab format take a look here.
what you are looking for is Subversion post commit hook that will execute what ever you script in your hook script. Take a look at the following example by Mike West:
Mike West - Subversion Post-commit-hook
good luck!
Related
Hopefully I find here someone who has experience with Hudson and its functions.
Now . I have Hudson installed this did not reveal any problems. But now I want to create a new job and that I'm developing in C / C + +.
In addition, I am working on Subversion svn where I run on the first error. Hudson did not find my svn . He says that I need an authentication . As I learned I can at Hudson authenticate but that does not work .
Maybe one of you knows how to create a project.
The things should be done in the job of Hudson.
Hudson is on my computer (local ) delete my project.
Then Hudson to access my SVN and check out the project from there.
The whole is now compiling Hudson . ( The best would be a compiler for C / C + + for Visual Studio 2008 compiler ) . The compiler then creates a * . Exe file.
Now Hudson to start the project on the basis of the *. Exe file and run the program .
Last but not least is to Hudson case of an error or if it was all right, inform the persons working on the project via email.
So that would be it what I 've hoped of Hudson. Otherwise, I take the whole not much. I know that I can do all this via a batch file . But that's not my goal. I want Hudson to automate so that I can start at midnight my builds / tests daily.
Do you think that at Hudson are my requirement too high?
For your help I would be very grateful , as I am stuck for days.
Here is a "basic" Hudson job
Create a new free-style software project job.
Configure that job.
(Optional) Configure triggers, such as "timer", "SCM polling", or others.
(Optional) Under Source Code Management section, select your SCM source and configure your repositories and local workspace
Under Build section, select Add build step and select:
Execute Shell if on *nix
OR
Execute Windows Batch Command if on Windows
OR
Pick whatever build-step plugin you are using.
(If using either of the "execute" build steps) Write your build/make/compile command as you would from command line.
(If using another plugin build step) Configure the plugin options according to your requirements.
(Optional) Archive the artifacts of the build with Archive the artifacts under Post-build Actions
(Optional) Execute other post-build actions
(Optional) Send out an email
Now to address your specific scenario. First things first, your question is too broad, and may get locked. Don't get discouraged if that happens, create separate question for each item individually. I cannot cover in details all these items, but I will give you an overview.
The SCM part
Based on your previous question, No Credentials to try in Hudson, I am now guessing that you are not providing Hudson with an HTTP URL to your SVN server, but trying to give it your local workspace location... Please do the command line check that I asked in that question.
You need to provide it with a proper HTTP server URL. Hudson will check out the project from the SVN URL you provided, under what is called a Workspace. The location of workspace can differ, based on your Hudson configuration, but it is a folder inside Hudson installation that is dedicated to the job. It can be referenced from within the job through %WORKSPACE% environment variable.
There are ways to use a different workspace location, but that is outside the scope of this overview. The whole SCM part is also optional, you can rely on existing file system, but this is not a good approach, and again, out of scope of this overview.
The Build step
After Hudson checked-out/updated the Workspace with your SVN, comes the building step. Hudson can do Execute Windows Batch Command by default. It can also Invoke Ant by default. (It can also do Maven, but that is not applicable to your situation)
To do other types of builds, you need a Build Wrapper plugin. In your particular case, the MSBuild plugin is probably what you want. I've never used MSBuild, so cannot give you details. Again, if you have a specific question on how to use MSBuild plugin, you should probably make a separate question with specific issues.
So, using either Execute Windows Batch Command or MSBuild plugin, configure your building step.
Running the exe???
This is very vague. You want to start the .exe and then what? Will it quit and you need an exit code? Do you want to see it on the screen? Again, this is very broad, and deserves a separate question (or read existing questions). If you just want to make a call to the .exe, you can configure a second Execute Windows Batch Command step, type there call path\to\yourfile.exe. But most likely you will not see that on screen. Read my answer here, Open Excel on Jenkins CI, on details of launching an .exe from Hudson/Jenkins that would be visible on screen.
Email
If you want a simple email, Hudson Post-Build actions has a way to send an email. For better customization options, you would want Email-Ext plugin. Once again, if you need details on how to use the email-ext plugin, create a new question (after searching existing questions first), as this is too much to cover in one question.
Conclusion
Your requirements are not too high, but Hudson is not a magic tool that will do the work for you. You still need to configure every step of it. And unless you have a Maven based project (which integrate very well with Hudson), a lot of actions will need to be done through the Execute Windows Batch Command and scripting of your own.
Is it possible to trigger a Hudson/Jenkins build only when a certain string appears in a commit-message?
For instance, I want to trigger a build that rolls out my application to the dev environment by writing a commit message like:
MYPROJECT-123 Fixed NPE in MyClass.java #deploy:DEV
The general idea is described in this great talk on Continuos Deployment but I couldn't find any information on how to do this in Hudson.
I would prefer to have this behavior in Hudson itself and not in an external system like commit-hooks or web-hooks.
I don't know of an out of the box way you can parse the SCM message as part of the trigger. You have a couple of options that might achieve what you want though
Write your own Hudson SCM plugin
Chain your jobs together into a build pipeline. The first job could simply look for that message in the changelog.xml to determine if the next build is triggered or not.
If you are looking at building a pipeline of build jobs, check out the build-pipeline-plugin. http://www.centrumsystems.com.au/blog/?p=121
Anyone got a more elegant solution??
Cheers,
Geoff
There is a plugin called Commit Message Trigger Plugin, but it had just a 0.1 release.
Maybe the easiest way is to use a version control post commit (or push) trigger to start a Hudson Job. You'd one anyway to automatically start your build.
I was trying to find out who triggered the failing Hudson build. But i found Started by an SCM change instead of Started by 'UserId'. Now, what does that mean?
It means that someone checked in code changes to your version control system / software configuration management (CVS, SVN, Git, etc), and Hudson started a built based on that change.
You should be able to see who it was by clicking the "Changes" link on the left menu.
"SCM" is "software configuration management", i.e. your version control system. Hudson can be configured to poll CVS, SVN etc for changes to your source code, and trigger a build based on that change.
I was working on a script to fire off an email with a list of changeset to a developer who started the build. After spending a couple hours on the web trying to search for a solution, I found a workaround digging through hudson log files. Probably not the cleanest way of doing it, but it works. Every time when hudson fire off a build, it records the build status to a log file. The log looks something like this..
Started by user <****>
Building remotely on Slave1
$ hg clone -r test_clone https://mercuial.com/build /some/workspace/test_clone
adding changesets
adding manifests
adding file changes
added 355 changesets with 298 changes to 43 files
updating to branch default
41 files updated, 0 files merged, 0 files removed, 0 files unresolved
...
...
The log file is in ~workspace/jobs/${RELEASE}/builds/${BUILD_NUMBER}/log. You can then parse the log file for the UserId. Something like this should work.
UserID=head -1 ~workspace/jobs/${RELEASE}/builds/${BUILD_NUMBER}/log|cut -d" " -f4
Had the same problem in jenkins when checkout branch name and specified branch name where same. Any change triggered hudson to automatically create a build to catch-up.
solved by: change local branch name to something else (or don't use it, maybe)
I have a job that I want to run every time a commit is made to a repository. I want to avoid pulling this code down, I only want the notification build trigger. So, is there either a way to not pull down certain repositories in your SCM upon a build or a way to poll things that aren't in the SCM for a build?
you could use a post commit hook to trigger your hudson job.
Since you want to avoid changing SVN, you have to write a job that gets executed every so often (may be every 5 Minutes). This jobs runs a svn command using windows bach or shell script task to get the current revision for the branch in question. You can set the status of the job to unstable if there is a change. Don't use failure because you can't distinguish than between a real failure and a repository change. I think there is a plugin that sets the job status depending on the contents of you output.
You can then use the email extension plugin to send an email every time the revision changes. You can get the revision number from the last (or better the last successful or unstable) job. You can archive a file containing the revision number on the jobs or you can set the description for the job to the revision using the description setter plugin. Have a look at Hudsons remote API for ideas on how to get the information from the previous job.
Since you run your job very often during the day. don't forget to delete old job runs. But I would keep at least two days worth of history, just in case your svn is down for 24 hours.
How do people deploy/version control cronjobs to production? I'm more curious about conventions/standards people use than any particular solution, but I happen to be using git for revision control, and the cronjob is running a python/django script.
If you are using Fabric for deploment you could add a function that edits your crontab.
def add_cronjob():
run('crontab -l > /tmp/crondump')
run('echo "#daily /path/to/dostuff.sh 2> /dev/null" >> /tmp/crondump')
run('crontab /tmp/crondump')
This would append a job to your crontab (disclaimer: totally untested and not very idempotent).
Save the crontab to a tempfile.
Append a line to the tmpfile.
Write the crontab back.
This is propably not exactly what you want to do but along those lines you could think about checking the crontab into git and overwrite it on the server with every deploy. (if there's a dedicated user for your project.)
Using Fabric, I prefer to keep a pristine version of my crontab locally, that way I know exactly what is on production and can easily edit entries in addition to adding them.
The fabric script I use looks something like this (some code redacted e.g. taking care of backups):
def deploy_crontab():
put('crontab', '/tmp/crontab')
sudo('crontab < /tmp/crontab')
You can also take a look at:
http://django-fab-deploy.readthedocs.org/en/0.7.5/_modules/fab_deploy/crontab.html#crontab_update
django-fab-deploy module has a number of convenient scripts including crontab_set and crontab_update
You can probably use something like CFEngine/Chef for deployment (it can deploy everything - including cron jobs)
However, if you ask this question - it could be that you have many production servers each running large number of scheduled jobs.
If this is the case, you probably want a tool that can not only deploy jobs, but also track success failure, allow you to easily look at logs from the last run, run statistics, allow you to easily change the schedule for many jobs and servers at once (due to planned maintenance...) etc.
I use a commercial tool called "UC4". I don't really recommend it, so I hope you can find a better program that can solve the same problem. I'm just saying that administration of jobs doesn't end when you deploy them.
There are really 3 options of manually deploying a crontab if you cannot connect your system up to a configuration management system like cfengine/puppet.
You could simply use crontab -u user -e but you run the risk of someone having an error in their copy/paste.
You could also copy the file into the cron directory but there is no syntax checking for the file and in linux you must run touch /var/spool/cron in order for crond to pickup the changes.
Note Everyone will forget the touch command at some point.
In my experience this method is my favorite manual way of deploying a crontab.
diff /var/spool/cron/<user> /var/tmp/<user>.new
crontab -u <user> /var/tmp/<user>.new
I think the method I mentioned above is the best because you don't run the risk of copy/paste errors which helps you maintain consistency with your version controlled file. It performs syntax checking of the cron tasks inside of the file, and you won't need to perform the touch command as you would if you were to simply copy the file.
Having your project under version control, including your crontab.txt, is what I prefer. Then, with Fabric, it is as simple as this:
#task
def crontab():
run('crontab deployment/crontab.txt')
This will install the contents of deployment/crontab.txt to the crontab of the user you connect to the server. If you dont have your complete project on the server, you'd want to put the crontab file first.
If you're using Django, take a look at the jobs system from django-command-extensions.
The benefits are that you can keep your jobs inside your project structure, with version control, write everything in Python and configure crontab only once.
I use Buildout to manage my Django projects. With Buildout, I use z3c.recipe.usercrontab to install cron jobs in deploy or update.
You said:
I'm more curious about conventions/standards people use than any particular solution
But, to be fair, the particular solution will depend in your environment and there is no universal elegant silver bullet. Given that you happen to be using Python/Django, I recommend Celery. It is an asynchronous task queue for Python, which integrates nicely with Django. And, on top of the features that it gives as an asynchronous task queue, it also has specific features for periodic tasks.
I have personally used the django-celery-beat integration and it integrates perfectly with Django settings and behaves correctly in distributed environments. If your periodic tasks are related to Django stuff, I strongly recommend to take a look at Celery I started using it only for certain asynchronous mailing and ended up using it for a lot of asynchronous tasks + periodic sanity checks and other web application maintenance stuff.