I want to build a Maven project using Jenkins. However, the project only must be built if a certain file in the SVN repository has changed (and contains a special key)
So my plan is to create a job with two build steps:
the first step executes a shell or python script that checks that "condition".
the second step is the actual Maven build
The second step only must be invoked if the condition check in step 1 returned "true".
Is there a possibility to do so? Well, I guess I could return an exit code 1 in the first script if the condition is not met. This will stop the build at once, but the job will be marked as "failed". So this is not a good idea since the red icon makes my users panic ;-)
Any other ideas around this?
Cheers,
Frank
We do something similar with our own Jenkins setup.
We have a "trigger" job that monitors SVN on a periodic basis. When a change occurs in SVN, the trigger job executes its build steps. One of the build steps examines some aspects of the code and decides whether a build is necessary or not. If it is necessary, it uses CURL to initiate the start of a the "build" project. The "build" project gets the source code and does a build - it doesn't bother to figure out whether it needs to build or not - it always does.
Having the two tasks separate also makes it easy to trigger a manual build without worrying the should-I-build logic kicking and stopping the build.
Related
I use an onsite build agent to perform my VSTS builds. This is working fine, sort of. I have 2 build definitions, one of which is a clone of the other and the only difference between the 2 is the solution that is built, all other parameters are exactly the same.
One of my builds completes without error and creates build artifacts and compiled code zip files in the 'build/1/a' artifacts folder. My other build completes without error BUT there are no build artifacts and compiled zip files created, my 'build/3/a' directory for this build is empty and I cannot see anywhere in the logs where the tasks to create this was executed, if at all. This did used to work before I cloned the build definition though. These are the MSBuild arguments that I have defined for both build definitions;
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\" /p:PackageTempRootDir="D:\Build\SiteManagerDev"
The only difference between them is the last parameter '/p:PackageTempRootDir'.
I have tried switching between the 2 directories for both build definitions to make sure it is not a permission error and the definition that finishes correctly works against either directory. I am starting to tear my hair out now and I have even tried creating a completely new build definition for the solution that creates an incomplete build and it is the same result, it is almost as if it is the solution itself that is causing the issue?
Any help would be greatly appreciated.
UPDATE: 05/02/2017
I think I finally understand what is going on! Question, if a build is manually triggered, not by a check-in trigger, if there have been no changes made to the code, and even though the build is executed, does this prevent the build artifacts from being created again because nothing has changed? The reason I ask is because I have found a strange in-house housekeeping routine that goes and deletes the contents of the 'D:\Build\1\a' directory on our build machine on a regular basis (I have no idea why!) and this results in there being nothing to publish UNTIL there is a code change checked in and then they are generated again! What a waste of everyone time this has been, my apologies and thank you for your help.
I am using Jenkins CI as the build server on a project that I am working on, I am also using Klocwork as a static analysis tool to identify deviations from our coding standards.
At present Jenkins has two builds (being performed in separate directories), a full build on a nightly basis that wipes out the workspace and performs a fresh checkout and full rebuild of everything.
In addition to the overnight build I also have an incremental build happening within 15 mins of a check in. Both builds are using the Klocwork analysis tool.
Klockwork works by displaying a list of potential issues which can then be fixed or chosen to be ignored if they are not applicable to the project, when issues are being ignored Klocwork uses the build file paths to remember where the issues that have been ignored reside. This means that when in Klocwork once I have ignored a warning in the full build and an incremental build is triggered the warning once again returns as the build path is different.
The most sensible solution I can see to this is for Jenkins to perform its full build on a nightly basis but for the incremental build to do an update in the full build location and to then do an incremental build - in the same way that an IDE on a PC functions.
The problem is that I have Jenkins running the full build and the incremental build as two separate jobs which causes them to check out into different locations and I cannot find a way of having the two jobs share a common directory.
Also I cannot find a way of having a single job that performs a nightly full checkout and rebuild, and an incremental build with an update on check in at the same time.
Is anyone familiar with a way of making Jenkins use a common source directory across multiple jobs?
Many thanks,
Pete.
Here's what I did.
Used one job to only check out source code.
In other jobs configuration settings', I set an environment variable that pointed to the workspace directory tree that contains the first job's source code (command line access to the Jenkins server is helpful here to figure out where it is, but not necessary). Then in my config scripting in Jenkins in the regular jobs, I 'cd' to that location and use the environment variable as path to all files so these other jobs would use the first job's checked out code.
Used locks, so regular jobs would not be running at same time as the check-out code job.
Since some results files (because of the tests being run) were generated and created in the source code tree because of the specifics of some of these jobs, in the config post-action script, I copied/moved the desired results back to the workspace of the job that should have them so I could process these results in the right job.
Worked for me.
You can easily make the two builds share the same build-area,
simply by extracting the files in both build-jobs to a shared location.
I strongly advise NOT to do this, as you can quickly get to a situation where
the nightly-build is cleaning the build-area while the incremental build is still running
(or that the incremental-build is checking-out sources while the nightly is still running).
Suggest you connect Klockwork only to one of the build-jobs (the nightly, probably)
so to avoid duplicate warnings.
I would like to stop hudson from incrementing the build number for failed builds. I see no value in incrementing the build number for failed builds as they are not usable.
I expect this is a simple setting in the config but cannot find it and I can't find a solution on SO.
Could someone please let me know if this is possible?
A build is a build, whether it's successful or failed, isn't it? Build number is different from release number, or version number, or whatever your organization calls it.
Additionally, Hudson stores the entire logs from all builds so you can easily refer back to them. This aids you in your ability to go fix the problems with broken builds. If you didn't have that, and you had multiple broken builds in a row, how could you tell what the problem was with each of the builds?
IMHO, you can't find a solution because it's something both that you shouldn't be doing, and cannot currently be done.
It sounds like your trying to automate a rolling minor release #. As others have mentioned you shouldn't use the hudson build number.
If you are using Maven you can use the release plugin so that the least significant version number is bumped (ie 1.0.x where x is incremented) on successful builds.
The Maven release plugin will also checkin the new version # into source control (using wagon) and create a tag.
If you are not using Maven I'm sure there are other build tools that do something similar.
We recently moved to an automatic build system (something internal, not Hudson or Teamcity, yet).
Our version is stored in a header file and is included by some cpp and resource files. It is also used by the installer.
Its format is A.B.C.D where:
A didn't change in years.
B changes rarely (major version).
C changes with minor versions.
D changes when a new minor version (bug fix) is delivered to QA.
Up until now, the one incharge of building a new version, incremented C/D by hand (D being the more common) before starting the build, checked in the change and then started the build. The version stayed the same until that person built the app successfully.
Naturally with the move to an automatic build system I'd like to get rid of the manual step of changing the version number.
How should this be approached?
Do I increment D whenever a new build is made, whether it's a QA build or an internal-test build (i.e. I'm working on some feature and I'd like to test I haven't broke anything)?
Is the increment step a task in the automatic build system?
After incrementing, should I commit the version file?
How do I avoid having a lot of noise in my version control? I don't want tons of "version incremented" commits.
What do I do if the build failed? Still increment the version and commit?
Do I increment D whenever a new build is made, whether it's a QA build or an internal-test build (i.e. I'm working on some feature and I'd like to test I haven't broke anything)?
The Eclipse Foundation adds an E element, the date and time of the build. I think that's a good idea for the internal-test builds. It's up to you if you want to use E for the QA builds.
Is the increment step a task in the automatic build system?
Seems logical, but you have to have some way of telling that task what kind of build you're doing.
How do I avoid having a lot of noise in my version control? I don't want tons of "version incremented" commits.
Commit the version control file with the source code.
Basically, your development build process should proceed in the following order.
Build the product from the development source code.
If the build succeeds, increment the version number.
Commit the source code and the version control file.
Build the product again from your version control system.
If the build fails, back out the source code and version control file commit.
This tests your build and your build process. The second build should never fail, but if it does, there's a problem in the process.
Your production build process would start at the 2nd step, skipping the 3rd step.
What do I do if the build failed? Still increment the version and commit?
My E is auto-incrementing. :-) I'd say no for the other elements A, B, C, or D.
Do I increment D whenever a new build is made, whether it's a QA build or an internal-test build (i.e. I'm working on some feature and I'd like to test I haven't broke anything)?
Yes, change your process so that D increments with every build (successful or not) rather than with every delivery to QA.
It can be quite frustrating having several builds, some working some not and not being able to tell them apart because the failed build is the same id as the good one, well eventually.
Then you don't even have to consider if it was on the same day or in the same hour.
Is the increment step a task in the automatic build system?
I'd have the build system auto increment the build number (D) only.
After incrementing, should I commit the version file?
How do I avoid having a lot of noise in my version control? I don't want tons of "version incremented" commits.
The version control storage is all about recording the detailed noise.
I'd have the version update checked in, this can make a reasonable tag visible in SVN of what build the previous changes where included in, have the build system ignore checkins by the build system, or those identified as the version update checkin.
Then to view the version history you should have an appropriate tool that allows you to filter the history to show you the view you need, in some cases excluding the version commit tags.
If you choose not to commit the version number for each build, then it might be a good idea to maintain the version number in a separate file to avoid accidental updates.
What do I do if the build failed? Still increment the version and commit?
Still increment the version number, I wouldn't commit the version number unless it was a successful build. You can have a variety of failures outside of source change in version control that don't need to be recorded - build server out of disk, server crash, compiler got all wobbly in the knees building 32 and 64 bit, debug and release aix, linux and windows builds at the same time...
You could consider to use the convention for .NET assemblies, as described in the documentation for class System.Version. Quote:
Build [your C]: A difference in build number represents a recompilation of the same source. Different build numbers might be used when the processor, platform, or compiler changes.
Revision [your D]: Assemblies with the same name, major, and minor version numbers but different revisions are intended to be fully interchangeable. A higher revision number might be used in a build that fixes a security hole in a previously released assembly.
How are you going to automate this? I mean, what system would know that "this build is the release build!". It would seem to me that all your digits in a version is relevant. If the next release (D + 1) requires two builds, then would A.B.C.D+2 be the next version? Sounds fishy to me. I would rather add the build number on top of the version instead, if it's really necessary to have this information on your DLLs and EXEs.
I don't think the build number is a relevant piece of information to have attached to the binary, unless you distribute files of version A.B.C.D from different builds (which you shouldn't do anyway!)
I would setup the build server to store the artifacts (DLLs, EXEs, MSIs, PDBs, etc) in a directory, whose name includes the build number and version, and then burn DVD/whatever from there. If you ever need to back track from a version to a specific build, you can use this information, provided that you keep an archive of your releases (recommended!).
I would recommend the use of autorevision.
You could still keep the A.B.C.D format for your tags and use the script to create header files that are generated at build time that have the needed info in them.
I am building merge modules with WIX. The batch files which calls the WIX tools to generate the merge modules from *.wxs files are run by my daily build.
I am trying to figure out how could I automate the testing of these merge modules. Things I would like to test are, whether the merge module installs required files, whether the versions of the files are correct etc.
One idea I have is to write a script (may be VB Script) to install the merge module at a temporary location and check if it has installed everything correctly. However, I am not sure if this is a correct way to do it.
Are there any standard ways of writing unit tests for merge modules? Any ideas around how to go about this are welcome.
When you test an installer, the primary goals are to verify that
When installing the msi file, msiexec reports success (i.e. return code 0).
After running the installer, your application can be started and works as expected.
The first point should be easy enough to do, though if you want to keep the test automated you can only test the non-interactive install. Here is an example how to do that from a batch file or on the command line:
msiexec /i myinstaller.msi /passive || echo ERROR: non-zero return code!
The second point is a bit more complicated. I think the best way to achieve this is to build some integration tests into your application, and invoke those tests after the install:
"c:\program files\mystuff\app.exe" /selftest || echo ERROR: non-zero return code!
In your case you're testing a merge module rather than an entire installer. The same approach can be used. You just will have to do the additional work of building a "self test" application and its installer which consumes your merge module.
I've often thought about this but haven't come up with anything that I like. First I would call this integration testing not strictly unit testing. Second the problem of "right files" and "right versions" is difficult to define.
I'm often tempted to say WiX/MSI is just data that defines what the installer is to do. It's declarative in nature and therefore by definition correct. It's tempting to want to create yet another set of data that cross checks the implementation of the installer but what exactly does that accomplish that the first data set didn't already represent? It's hard enough sometimes to maintain what files go into an application yet alone to maintain a second list of files.
I continue to think about this and wonder if there's an approach that would make sense but at this point I just do my normal MSI validation.
You could try to use scripts or other small console program that will do the job, just like you suggested.
With your build process you could also build a basic setup that just uses the merge module. Your script could just install this, run the other script or console app that will check if all the files are in place, that they have the correct version, that all the registry keys are installed, etc. After all the output is gathered your main script would just uninstall everything. You could also run the check program after uninstalling to be sure that everything is gone and that the uninstall works correctly. I would recommend this if, for example, you have custom actions set for install and uninstall.
Ideally this whole install / uninstall process should be done on a separate machine, or a virtual one, in order to avoid messing up the build server.
You'll have some work to do with all this scripts but once you have it, you'll be able to use it with little changes for any other future merge module projects or just plain setup projects.
Hope this would help,
Adrian.