In compile scripts I generally see a call to always delete object files before compiling. Does this slow down the build process? Is it really necessary with compilers that check if the object files are out of date when deciding to recompile them?
Sometimes if you revert a source file to a previous version, the .o will have a newer date that its source, and therefore won't be fed to the compiler. If you had a reason to revert the source file, you almost surely want the object rebuilt. Doing a clean build ensures you get what you think you're getting.
In some way, deleting existing object files defeats the purpose of having separate translation units in the first place. Your standard build environment should normally only rebuild those object files which are older than the corresponding source file. (You don't need to delete, you can just overwrite.)
If you have a decent revision control system, then even checking out an older version of a modified file will make the actual file on disk have a current timestamp, but indeed, if you're worried that something might be inconsistent, you can always clean up the entire build tree and start over. But as a matter of normal code writing, it would appear terribly wasteful to delete object files.
You should of course keep one set of object files for each set of build options (e.g. debug vs release). Some build environments allow you to have multiple output directories, others (like cmake) will just automatically rebuild everything if you change the global build settings, but that's something to watch out for, especially if you just add some #defines to the compiler flags in the middle of a build process.
1) Yes, and 2) no, but it's not the compiler that watches out for that, it's the build system (an IDE or well-written makefile).
Related
I am using cmake to setup my project and when I change a file in a project, I found my cmake knows to only recompile the changed file and then relink everything together for the final executable/lib.
I then read through the documentation about ccache, what I don't understand is: what is the difference between ccache's approach (that uses hash value to check if file is changed and needs recompile) and the default approach that the cmake uses (or there might be something else rather than cmake checks the file updates, but you know what I mean here). Maybe the PCH part is different, but cmake 3.18 now comes with PCH support, so, does that mean the benefits ccache provides on the PCH part is no longer unique?
Consider the case where you switch to some older branch of your project - that you did compile in the past and that ccache has cached, but CMake sees as "almost all files have changed and must be recompiled" - that's where you see a massive gain.
Another situation is where you have deleted your build directory (for some good reason) and now have to rebuild everything. ccache is also a huge help there.
Also; ccache is trivial to set up and is thenceforth completely invisible / transparent, so there really is no reason to not use it. When it helps it usually helps a lot, when it does not help it doesn't hurt.
cmake/gmake and ccache are not exclusive to each other. They are typically used together.
ccache comes into play when the entire source tree needs to be rebuilt for some reason. cmake/gmake rebuilds only changed files, but there are situations where the entire source tree needs to be recompiled. And if this happens repeatedly, ccache will wake up and short-circuit the compiler. C++ compilers are notorious for being slow, and this often helps quite a bit.
Just a couple of examples: when you need to repeatedly switch between building with and without optimizations, repeatedly. cmake/gmake won't help you when you edit the makefile and adjust the compilation flags. None of the source files actually changed, so cmake/gmake doesn't think there's anything to do, so you must explicitly make clean and recompile from scratch.
If you are doing it repeatedly, ccache will avoid having to run the compiler on the entire source code, and will simply fetch out the appropriate object module instead of compiling the source from scratch.
Another common situation is when you're running a script to prepare an installable package for your code. This typically involves using an implementation-specific tool to rebuild the source code, from scratch, into an installable package.
I'm wondering if there's a macro or a simple way to let the compiler increment either major, minor or revision of my code each time when I compile?
By the way I'm using the ARM compiler and uVision from Keil.
To set the version is not a compiler topic. This should be done in connection with a source code control / version control system like cvs/svn/git or others. Your build id should be connected to the content of your source code database to get reproducible builds from checkouts from your version control system. Or, if your code is not already committed to your database a dirty-tag should be provided and compiled in to give the user of the software a chance to see that this is not a controlled version.
Simply counting a value in a variable can be done by a Makefile or in pre- and post-build instructions which depends on the used IDE. Sorry, for keil I have no experience...
Define Post-Build Event to run small external program. The program has to modify specific .h file. In the header file define macros like VER_MAJOR, VER_MINOR, VER_BUILD. Date/time string can also be updated. I use this method and can control the version numbers as I wish.
IMO, you do not need to do this, especially increase a number every time you compile your code.
Set the major and minor revision manually in a header file; you should not have to do this often.
Build number should only be related to the source control revision number (i.e. you should be able to build and rebuild any revision under source control).
Imagine if you are a team of 5 developers, and everyone build and rebuild on their side what is the actual build number?
do they all update a header file ?
who is responsible of owning that header file ?
Some compilers do support features like "post build", which runs a program of your choice after compiling, but that would be tricky if your program is built from multiple source files. Not all compilers do though.
For that reason, I wouldn't actually do this sort of thing via the compiler. I'd do it in the build script (e.g. makefile) or by configuring the build settings in your IDE.
Assuming you're using make or similar, add a target something like setversion (you pick the name) that runs a program which modifies a header file which specifies the components of your version number. So typing make setversion will update your version numbers.
Optionally, that target can also - after updating the version numbers - do a make clean (i.e. delete all object files and executables) and make all (to recompile and link everything).
I also suggest avoiding changing version numbers after each recompile. Imagine you're busily testing and debugging code, and go through several rebuild cycles. Do you really want the version number updated every time you recompile even one source file? It can be done that way if you choose, but will make every rebuild take longer (and, in projects with multiple source files) you will need to take care if you want to preserve capability for incremental builds.
I'm working with a very old and large VC6++ project and it's all messed up. There are unused files and folders everywhere, copies of folders and it's just a mess to clean it up by hand in its current state.
It will be done eventually, but is there any simple way to check what files and folders are used when it does a clean compile?
The project settings doesnt help me at all because it simply uses copies of folders and additional include directories.
Any suggestions?
Well, if you want to parse the compiler output you can get which files are actually used. I also find this when googling around, you might want to try (I haven't tried it myself). My way would be to clean the build, list all source files, build, and for each source find its corresponding .obj. The ones without .obj are not used. Note that this only works for source files, unused header files stay undetected.
VC6 will produce a makefile for you:
http://msdn.microsoft.com/en-us/library/aa233950%28v=vs.60%29.aspx
You can use the generated makefile (and the associated .dep file) as a starting point and edit it down to the list of files that get used in a build.
This will let you see the header files that the project depends on in addition to the .c/.cpp/.lib files that might show in the build log. One thing to keep in mind is that you'll probably also want to make sure you track the .dsw and .dsp workspace and project files.
If you're a bit adventurous, you might be able to convince the makefile to actually copy the source files to some other location for you with an appropriate override of the certain macros and/or dependencies. But that would probably be more trouble than it's worth for a one-time effort.
Finally, there's a commercial product, CopyWiz by Kinook Software, that seems to have features that might do what you're looking for (and it supports VC++ 6). Note: I'm not sure if it will do what you want, but it may be worth a look.
Yes. Run Process Monitor from SysInternals. It can capture all file system events and filter them based on the path and other factors.
So, set the filter to the root of your source tree, only succesfull file reads (VC looks for headers in many places), and build your project. You'll probably still see several thousand events. So, save them to file, sort by path, and remove duplicate paths (headers especially will have many duplicate entries)
When using an autoconf/automake build system if the compiler flags or other variables in a Makefile.am (or even higher level like configure.ac) change, the C++ source files associated with that Makefile will not be automatically rebuilt. This becomes especially important as we use automake as part of a continuous build system that only recompiles as needed.
My thought was to include Makefile as a dependency for the .o files which would theoretically solve the above issue. So a couple of questions:
First, is it possible to add a rule like that? I would prefer to not have to add that custom rule to every single Makefile.am, so something that could be placed into a top-level file (like configure.ac) would be great.
Second, the downside to this approach is that in some cases the change to the Makefile did not actually affect the compilation so I will end up rebuilding when it is not really needed. I guess I'm willing to live with this (or at least try it to see how painful it is) to have a better guarantee that my builds will be correct, but is there a better way to solve this problem? I believe clearmake solves this by saving the actual compiler command (along with other dependencies) then comparing the current command with the previous to determine if a file needs to be regenerated.
If you use ccache (./configure CXX='ccache g++', or just add ccache's g++ to the path), spurious rebuilds should be very cheap and still safe. Also make sure never to use the AM_MAINTAINER_MODE autoconf macro, which makes dependency tracking optional (conditional on the --enable-maintainer-mode flag).
We're looking for a way to include some sort of build ID automatically in our builds. This needs to be portable (VC++, g++ on Linux and Mac) and automatic. VC++ is what matters most, since in the other environments we use custom Python build scripts so I can do whatever I want.
We use SVN, so we were looking at using the output of svnversion to write the revision to a header and include it. This has problems : if we put the file in SVN, it will appear as modified every time, but it would be a superfluous commit and in a sense generate an infinite loop of increasing revisions. If we don't put the file in SVN and just create it as a pre-build step, the sources wouldn't be complete, as they'd need the pre-build step or Makefile to generate that file.
We could also use __DATE__ but we can't guarantee the file that uses the __DATE__ (ie writes it to a log file) will be compiled if some other file is modified - except if we "touch" it, but then we'd cause the project to be always out of date. We could touch it as the pre-build step, so it would get touched only if the rest of the project is out of date, thus not causing a spurious compile, but if VC++ computes the dependencies before the pre-build step, this wouldn't work (the file with __DATE__ won't get compiled)
Any interesting ideas?
We're using the output of svnversion, written to a header file and included. We omit the file from the repository and create it in a pre-build step; this has worked quite well for us. (I'm not sure why you object to using a pre-build step?)
We're currently using a Perl script to convert svnversion's output into a header file; I later found out that TortoiseSVN includes a subwcrev command (which has also been ported to Linux) that can do much of the same thing.
If you don't like the idea of an include file not in source control that is required for a build, consider a batch file or other build step that programmatically creates a file/include and call the svnversion within your build process.
basically GENERATE the file so you don't have an unversioned and required file.
EDIT
Josh's subwcrev is probably the best idea.
Before that was implemented I wrote my own hacky tool to do the same thing - do replacement in a template file.
It could be as simple as:
% make -DBUILD_NUMBER=`svnlook youngest /path/to/repo`
I'd look at SvnRev. You can use it as a custom pre-build step in VS, or call it from a makefile, or whatever else you need to do, and it generates a header file that you can include in your other files that will give you what you need. There's good documentation on the site.
SubWCRev is another option, though the Linux port is newer, and I don't know that a Mac version exists. It's very useful on Windows for .NET (which I'm guessing isn't an issue for you, but I'm adding this for future reference), because it allows you to create a template file that can be used to generate, for example, the Properties file for a .NET assembly.
Automatic builds can typically be full, clean builds. In that case, you start in a clean directory and there would be no issue with __DATE__ in any case. Otherwise, see Paul Beckinham's idea.
Why not tie a GUID to it, almost every language has support for generating one, or if your's doesn't there are alot of algorithms for that around.
(Although, if you do use subversion, I personally like Josh's idea better!)