We recently moved to an automatic build system (something internal, not Hudson or Teamcity, yet).
Our version is stored in a header file and is included by some cpp and resource files. It is also used by the installer.
Its format is A.B.C.D where:
A didn't change in years.
B changes rarely (major version).
C changes with minor versions.
D changes when a new minor version (bug fix) is delivered to QA.
Up until now, the one incharge of building a new version, incremented C/D by hand (D being the more common) before starting the build, checked in the change and then started the build. The version stayed the same until that person built the app successfully.
Naturally with the move to an automatic build system I'd like to get rid of the manual step of changing the version number.
How should this be approached?
Do I increment D whenever a new build is made, whether it's a QA build or an internal-test build (i.e. I'm working on some feature and I'd like to test I haven't broke anything)?
Is the increment step a task in the automatic build system?
After incrementing, should I commit the version file?
How do I avoid having a lot of noise in my version control? I don't want tons of "version incremented" commits.
What do I do if the build failed? Still increment the version and commit?
Do I increment D whenever a new build is made, whether it's a QA build or an internal-test build (i.e. I'm working on some feature and I'd like to test I haven't broke anything)?
The Eclipse Foundation adds an E element, the date and time of the build. I think that's a good idea for the internal-test builds. It's up to you if you want to use E for the QA builds.
Is the increment step a task in the automatic build system?
Seems logical, but you have to have some way of telling that task what kind of build you're doing.
How do I avoid having a lot of noise in my version control? I don't want tons of "version incremented" commits.
Commit the version control file with the source code.
Basically, your development build process should proceed in the following order.
Build the product from the development source code.
If the build succeeds, increment the version number.
Commit the source code and the version control file.
Build the product again from your version control system.
If the build fails, back out the source code and version control file commit.
This tests your build and your build process. The second build should never fail, but if it does, there's a problem in the process.
Your production build process would start at the 2nd step, skipping the 3rd step.
What do I do if the build failed? Still increment the version and commit?
My E is auto-incrementing. :-) I'd say no for the other elements A, B, C, or D.
Do I increment D whenever a new build is made, whether it's a QA build or an internal-test build (i.e. I'm working on some feature and I'd like to test I haven't broke anything)?
Yes, change your process so that D increments with every build (successful or not) rather than with every delivery to QA.
It can be quite frustrating having several builds, some working some not and not being able to tell them apart because the failed build is the same id as the good one, well eventually.
Then you don't even have to consider if it was on the same day or in the same hour.
Is the increment step a task in the automatic build system?
I'd have the build system auto increment the build number (D) only.
After incrementing, should I commit the version file?
How do I avoid having a lot of noise in my version control? I don't want tons of "version incremented" commits.
The version control storage is all about recording the detailed noise.
I'd have the version update checked in, this can make a reasonable tag visible in SVN of what build the previous changes where included in, have the build system ignore checkins by the build system, or those identified as the version update checkin.
Then to view the version history you should have an appropriate tool that allows you to filter the history to show you the view you need, in some cases excluding the version commit tags.
If you choose not to commit the version number for each build, then it might be a good idea to maintain the version number in a separate file to avoid accidental updates.
What do I do if the build failed? Still increment the version and commit?
Still increment the version number, I wouldn't commit the version number unless it was a successful build. You can have a variety of failures outside of source change in version control that don't need to be recorded - build server out of disk, server crash, compiler got all wobbly in the knees building 32 and 64 bit, debug and release aix, linux and windows builds at the same time...
You could consider to use the convention for .NET assemblies, as described in the documentation for class System.Version. Quote:
Build [your C]: A difference in build number represents a recompilation of the same source. Different build numbers might be used when the processor, platform, or compiler changes.
Revision [your D]: Assemblies with the same name, major, and minor version numbers but different revisions are intended to be fully interchangeable. A higher revision number might be used in a build that fixes a security hole in a previously released assembly.
How are you going to automate this? I mean, what system would know that "this build is the release build!". It would seem to me that all your digits in a version is relevant. If the next release (D + 1) requires two builds, then would A.B.C.D+2 be the next version? Sounds fishy to me. I would rather add the build number on top of the version instead, if it's really necessary to have this information on your DLLs and EXEs.
I don't think the build number is a relevant piece of information to have attached to the binary, unless you distribute files of version A.B.C.D from different builds (which you shouldn't do anyway!)
I would setup the build server to store the artifacts (DLLs, EXEs, MSIs, PDBs, etc) in a directory, whose name includes the build number and version, and then burn DVD/whatever from there. If you ever need to back track from a version to a specific build, you can use this information, provided that you keep an archive of your releases (recommended!).
I would recommend the use of autorevision.
You could still keep the A.B.C.D format for your tags and use the script to create header files that are generated at build time that have the needed info in them.
Related
I'm wondering if there's a macro or a simple way to let the compiler increment either major, minor or revision of my code each time when I compile?
By the way I'm using the ARM compiler and uVision from Keil.
To set the version is not a compiler topic. This should be done in connection with a source code control / version control system like cvs/svn/git or others. Your build id should be connected to the content of your source code database to get reproducible builds from checkouts from your version control system. Or, if your code is not already committed to your database a dirty-tag should be provided and compiled in to give the user of the software a chance to see that this is not a controlled version.
Simply counting a value in a variable can be done by a Makefile or in pre- and post-build instructions which depends on the used IDE. Sorry, for keil I have no experience...
Define Post-Build Event to run small external program. The program has to modify specific .h file. In the header file define macros like VER_MAJOR, VER_MINOR, VER_BUILD. Date/time string can also be updated. I use this method and can control the version numbers as I wish.
IMO, you do not need to do this, especially increase a number every time you compile your code.
Set the major and minor revision manually in a header file; you should not have to do this often.
Build number should only be related to the source control revision number (i.e. you should be able to build and rebuild any revision under source control).
Imagine if you are a team of 5 developers, and everyone build and rebuild on their side what is the actual build number?
do they all update a header file ?
who is responsible of owning that header file ?
Some compilers do support features like "post build", which runs a program of your choice after compiling, but that would be tricky if your program is built from multiple source files. Not all compilers do though.
For that reason, I wouldn't actually do this sort of thing via the compiler. I'd do it in the build script (e.g. makefile) or by configuring the build settings in your IDE.
Assuming you're using make or similar, add a target something like setversion (you pick the name) that runs a program which modifies a header file which specifies the components of your version number. So typing make setversion will update your version numbers.
Optionally, that target can also - after updating the version numbers - do a make clean (i.e. delete all object files and executables) and make all (to recompile and link everything).
I also suggest avoiding changing version numbers after each recompile. Imagine you're busily testing and debugging code, and go through several rebuild cycles. Do you really want the version number updated every time you recompile even one source file? It can be done that way if you choose, but will make every rebuild take longer (and, in projects with multiple source files) you will need to take care if you want to preserve capability for incremental builds.
I am using Jenkins CI as the build server on a project that I am working on, I am also using Klocwork as a static analysis tool to identify deviations from our coding standards.
At present Jenkins has two builds (being performed in separate directories), a full build on a nightly basis that wipes out the workspace and performs a fresh checkout and full rebuild of everything.
In addition to the overnight build I also have an incremental build happening within 15 mins of a check in. Both builds are using the Klocwork analysis tool.
Klockwork works by displaying a list of potential issues which can then be fixed or chosen to be ignored if they are not applicable to the project, when issues are being ignored Klocwork uses the build file paths to remember where the issues that have been ignored reside. This means that when in Klocwork once I have ignored a warning in the full build and an incremental build is triggered the warning once again returns as the build path is different.
The most sensible solution I can see to this is for Jenkins to perform its full build on a nightly basis but for the incremental build to do an update in the full build location and to then do an incremental build - in the same way that an IDE on a PC functions.
The problem is that I have Jenkins running the full build and the incremental build as two separate jobs which causes them to check out into different locations and I cannot find a way of having the two jobs share a common directory.
Also I cannot find a way of having a single job that performs a nightly full checkout and rebuild, and an incremental build with an update on check in at the same time.
Is anyone familiar with a way of making Jenkins use a common source directory across multiple jobs?
Many thanks,
Pete.
Here's what I did.
Used one job to only check out source code.
In other jobs configuration settings', I set an environment variable that pointed to the workspace directory tree that contains the first job's source code (command line access to the Jenkins server is helpful here to figure out where it is, but not necessary). Then in my config scripting in Jenkins in the regular jobs, I 'cd' to that location and use the environment variable as path to all files so these other jobs would use the first job's checked out code.
Used locks, so regular jobs would not be running at same time as the check-out code job.
Since some results files (because of the tests being run) were generated and created in the source code tree because of the specifics of some of these jobs, in the config post-action script, I copied/moved the desired results back to the workspace of the job that should have them so I could process these results in the right job.
Worked for me.
You can easily make the two builds share the same build-area,
simply by extracting the files in both build-jobs to a shared location.
I strongly advise NOT to do this, as you can quickly get to a situation where
the nightly-build is cleaning the build-area while the incremental build is still running
(or that the incremental-build is checking-out sources while the nightly is still running).
Suggest you connect Klockwork only to one of the build-jobs (the nightly, probably)
so to avoid duplicate warnings.
How do I build RPMS under Red Hat?
I need to package a newer version of some software than is available from the repositories.
(I can build it locally already, its just the packaging that I need to do, so that I can use it on other machines)
I could just take the .spec file from the older version's SRPM and start from there, right? - But i'm brand new to packaging, any pointers?
Starting with the existing SRPM is a fantastic first step -- trying to build one from scratch is an exercise in frustration, especially if your users expect it to work with previously packaged versions of the same tool.
You'll need to keep the documentation at rpm.org handy at all times. The Maximum RPM book is probably a little too dated now, but at one time it was always loaded in one of my browser tabs.
One thing to be very careful of is builds that work on your machine but aren't repeatable anywhere else. You really should get a dedicated building environment of some sort set up so you can have repeatable builds.
I would like to stop hudson from incrementing the build number for failed builds. I see no value in incrementing the build number for failed builds as they are not usable.
I expect this is a simple setting in the config but cannot find it and I can't find a solution on SO.
Could someone please let me know if this is possible?
A build is a build, whether it's successful or failed, isn't it? Build number is different from release number, or version number, or whatever your organization calls it.
Additionally, Hudson stores the entire logs from all builds so you can easily refer back to them. This aids you in your ability to go fix the problems with broken builds. If you didn't have that, and you had multiple broken builds in a row, how could you tell what the problem was with each of the builds?
IMHO, you can't find a solution because it's something both that you shouldn't be doing, and cannot currently be done.
It sounds like your trying to automate a rolling minor release #. As others have mentioned you shouldn't use the hudson build number.
If you are using Maven you can use the release plugin so that the least significant version number is bumped (ie 1.0.x where x is incremented) on successful builds.
The Maven release plugin will also checkin the new version # into source control (using wagon) and create a tag.
If you are not using Maven I'm sure there are other build tools that do something similar.
For past 4 years, I have been programming with Eclipse (for Java), and Visual Studio Express (for C#). The IDEs mentioned always seemed to provide every facility a programmer might ask for (related to programming, of course).
Lately I have been hearing about something called "build tools". I heard they're used almost in all kind of real world development. What are they exactly? What problems are they designed to solve? How come I never needed them in past four years? Are they kind of command-line stripped down IDEs?
What are build tools?
Build tools are programs that automate the creation of executable
applications from source code (e.g., .apk for an Android app). Building
incorporates compiling,linking and packaging the code into a usable or
executable form.
Basically build automation is the act of scripting or automating a
wide variety of tasks that software developers do in their day-to-day
activities like:
Downloading dependencies.
Compiling source code into binary code.
Packaging that binary code.
Running tests.
Deployment to production systems.
Why do we use build tools or build automation?
In small projects, developers will often manually invoke the build
process. This is not practical for larger projects, where it is very
hard to keep track of what needs to be built, in what sequence and
what dependencies there are in the building process. Using an
automation tool allows the build process to be more consistent.
Various build tools available(Naming only few):
For java - Ant,Maven,Gradle.
For .NET framework - NAnt
c# - MsBuild.
For further reading you can refer following links:
1.Build automation
2.List of build automation software
Thanks.
Build tools are tools to manage and organize your builds, and are very important in environments where there are many projects, especially if they are inter-connected. They serve to make sure that where various people are working on various projects, they don't break anything. And to make sure that when you make your changes, they don't break anything either.
The reason you have not heard of them before is that you have not been working in a commercial environment before. There is a whole lot of stuff that you have probably not encountered that you will within a commercial environments, especially if you work in software houses.
As others have said, you have been using them, however, you have not had to consider them, because you have probably been working in a different way to the usual commercial way of working.
Build tools are usually run on the command line, either inside an IDE or completely separate from it.
The idea is to separate the work of compiling and packaging your code from creation, debugging, etc.
A build tool can be run on the command or inside an IDE, both triggered by you. They can also be used by continuous integration tools after checking your code out of a repository and onto a clean build machine.
make was an early command tool used in *nix environments for building C/C++.
As a Java developer, the most popular build tools are Ant and Maven. Both can be run in IDEs like IntelliJ or Eclipse or NetBeans. They can also be used by continuous integration tools like Cruise Control or Hudson.
Build tools are generally to transform source code into binaries - it organize source code, set compile flags, manage dependencies... some of them also integrate with running unit test, doing static analysis, a generating documentation.
Eclipse or Visual Studio are also build systems (but more of an IDE), and for visual studio it is the underlying msbuild to parse visual studio project files under the hood.
The origin of all build systems seems like the famous 'make'.
There are build systems for different languages:
C++: make, cmake, premake
Java: ant+ivy, maven, gradle
C#: msbuild
Usually, build systems either using a propriety domain specific language (make, cmake), or xml (ant, maven, msbuild) to specify a build. The current trend is using a real scripting language to write build script, like lua for premake, and groovy for gradle, the advantage of using a scripting is it is much more flexible, and also allows you the to come up with a set of standard APIs(as build DSL).
These are different types of processes by which you can get your builds done.
1. Continuous Integration build: In this mainly developers check-in their code and right after their check-in a build initiates for building of the recent changes so we should know whether the changes done by the developer has worked or not right after the check-in is done. This is preferred for smaller projects or components of the projects. In case where multiple teams are associated with the project or there are a large no. of developers working on the same project this scenario becomes difficult to handle as if there are 'n' no. of check-in’s and the build fails at certain points it becomes highly difficult to trace whether all the breakage has occurred because of one issue or with multiple issues so if the older issues are not addressed properly than it becomes very difficult to trace down the later defects that occurred after that change. The main benefit of these builds is that we get to know whether a particular check-in is successful or not.
2. Gated check-in builds: In this type of check in a build is initiated right after the check in is done keeping the changes in a shelve sets. In this case if the build succeeds than the shelve-set check-in gets committed otherwise it will not be committed to the Team Foundation Server. This gives a slightly better picture from the continuous integration build as only the successful check-in's are allowed to get committed.
3. Nightly builds: This is also referred as Scheduled builds. In this case we schedule the builds to run for a specific time in order to build the changes. All the previous uncommitted changes from the last build are built during this build process. This is practiced when we want to check in multiple times but do not want a build every time we check in our code so we can have a fixed time or period in which we can initiate the build for building of the checked-in code.
The more details about these builds can be found at the below location.
Gated-check in Builds
Continuous Integration Builds
Nightly Builds
Build Process is a Process of compiling your source code for any errors using some build tools and creating builds(which are executable versions of the project). We(mainly developers) do some modifications in the source code and check-in that code for the build process to happen. After the build process it gives two results :
1. Either build PASSES and you get an executable version of your project(Build is ready).
2. It fails and you get certain errors and build is not created.
There are different types of build process like :
1. Nightly Build
2. gated Build
3. Continuous integration build etc.
Build tools help and automates the process of creating builds.
*So in Short Build is a Version of Software in pre-release format used by the Developer or Development team to gain confidence for the final result of their Product by continuously monitoring their Product and solving any issues early during the development process.*
You have been using them - IDE is a build tool. For the command line you can use things like make.
People use command line tools for things like a nightly build - so in the morning with a hangover the programmer has realised that the code that he has been fiddling with with the latest builds of the libraries does not work!
"...it is very hard to keep track of what needs to be built" - Build tools does not help with that all. You need to know what you want to build. (Quoted from Ritesh Gun's answer)
"I heard they're used almost in all kind of real-world development" - For some reason, software developers like to work in large companies. They seem to have more unclear work directives for every individual working there.
"How come I never needed them in past four years". Probably because you are a skilled programmer.
Pseudo, meta. I think build tools do not provide any really real benefit at all. It is just there to add a sense of security arising from bad company practices, lack of direction - bad software architectural leadership leading to bad actual knowledge of the project. You should never have to use build tools(for testing) in your project. To do random testing with a lack of knowledge of the software project does not give any sort of help at all.
You should never ever add something to a project without knowing it's purpose, and how it will work with the other components. Components can be functional separate, but not work together. (This is the responsibility of the software architect I assume).
What if 4-5 components are added into the project. You add a 6th component. Together with the first added component, it might screw up everything. No automatic would help to detect that.
There is no shortcut other than to think think think.
Then there is the auto download from repositories. Why would you ever want to do that? You need to know what you download, what you add to the project. How do you detect changes in versions of the repositories? You need to know. You can't "auto" anything.
What if we were to test bicycles and baby transports blindfolded with a stick and just randomly hit around with it. That seems to be the idea of build tool testing.
I'm sorry there are no shortcut
https://en.wikipedia.org/wiki/Scientific_method
and
https://en.wikipedia.org/wiki/Analysis