As of yet, at least to my knowledge, there is no standard way in CMake to specify the addition of a precompiled header (PCH) to a project in a cross-platform manner because the way PCHs are handled by C++ compilers is very different among vendors. For G++, this is usually this is worked around by simply adding a custom command which takes care of invoking the compiler with the appropriate input and has it generate the PCH.
My current problem is that CMake will not parse the dependencies of the dependencies you specify for the custom command. For instance, assume the following structure:
pch.h
|- dependA.h
|- dependB.h
...
Only providing pch.h as a dependency will lead to the generation of the appropriate target in the corresponding makefile, which tracks changes to pch.h. However, CMake does not parse the includes inside pch.h and will therefore not recognize changes to dependA.h and dependB.h. This extends furhter if there are dependencies for dependsA.h and so on.
Note: I'm aware that the fact that PCH dependencies can and do change regularly puts the whole process in question. However, this is just the way it is and I can't really do anything about it.
Since the task isn't too hard, there are a couple of obvious ideas that come to mind:
Solution A:
Enter all the dependencies by hand. Obviously this works, but is tedious as hell and doesn't scale at all.
Solution B:
If possible, write a CMake function that automates the process and parse the includes "manually".
Solution C:
Do something similar using a different language, for instance Python, and just provide CMake a list of dependencies to add to the custom command.
Solution D:
Use gcc/g++'s feature to parse and print out the dependency tree of the PCH and parse the output to extract the list of dependencies.
My question is: does anyone know a more convenient and faster way to get this done?
The IMPLICIT_DEPENDS option of the add_custom_command might do the trick:
add_custom_command(
OUTPUT outFile
COMMAND ...
IMPLICIT_DEPENDS CXX "pch.h")
The IMPLICIT_DEPENDS option makes the generated build system scan the implicit dependencies of the given input file at build time. It is only supported for Makefile generators, though.
Related
Question
In ninja, I can compile a single C++ file by running ninja path/to/my/object.file.o.
Is there a way to achieve the same in bazel?
Use case / Background
During refactoring, in particular when changing interfaces in .hpp files, I usually want to focus on one single complex user of the interface first. I want to iterate on that one user until my refactoring works as expected on complex_user.cpp and I am happy with the new interfaces. Only afterwards, I want to adjust all other users. I hence want to get the compiler errors / warnings only from my complex_user.cpp file while ignoring all other places where .hpp might be included
Try --save_temps. bazel build --save_temps //my:library will give you the .o, .s, and similar files for only the targets listed on the command line.
--compile_one_dependency is designed for a similar use case, if you want to specify the target to build by the .cpp file instead of specifying a particular cc_library.
You need to implement a custom-made rule cc_object_file. Since the Bazel cc_rules are open source you can use this as a starting point.
As far as I know, CMake checks the time stamp of a source file to detect if it is outdated and needs to be rebuild (and with it, all files including it). When switching branches in a large git repository, this can causes problems.
Let's say I have one source folder and two build directories (build1 and build2), which correspond to two different branches (branch1 and branch2)
project
+-- src
+-- branch1_build
+-- branch2_build
Say my two branches have few differences, in few files; mostly, they only differ for some configuration option, all encapsulated in a config.h file, generated by the CONFIGURE_FILE command in cmake. The source files for the two config.h files (the config.h.in, as it is often called) are different. For instance, one branch introduces a new subfolder, which can be activated with a config-time option, which gets put in config.h.in with something like #cmakedefine HAVE_NEW_FEATURE_FOLDER. In such a scenario, when switching branches in the source folder, this happens: cmake recognizes that something changed in the config.h.in file, so it runs again; by running again, it generates a new config.h file; since config.h has a new time stamp, all files that includes it (directly or indirectly) end up being recompiled.
Now, if I alternatively switch between branch1 and branch2 in the source folder (cause I'm working on both branches every day), two consecutive make commands issued in the same build folder (either branch1_build or branch2_build) will trigger a full recompilation, since, although config.h has not changed in content, its time stamp has changed, so cmake flags it has changed.
My question is: what options do I have to avoid this? Or, better phrased, how can I avoid recompiling a source-build tree pair that is in fact unchanged since the last build, while also minimizing the changes to the source code?
The only solution I can think of is to execute CONFIGURE_FILE on config.h.in, with output config.h.tmp; compare config.h.tmp with config.h, and, only if different, copy config.h.tmp to config.h. However, this seems clumsy, and overcomplicated. I hoped cmake already had a mechanism for this, perhaps hidden under some options/variations of CONFIGURE_FILE...
Assuming this is not yet possible, I was wondering how complicated it would be for cmake to check the sha (rather than the timestamp) of a particular file when deciding whether it is outdated or not, and comparing it with the sha of a previous build (yes, the word outdated has date in it, but let's not get into enlish vocabulary discussions here). I imagine this is more expensive, so I would think that, if possible at all, this behavior should not be the default, and the user should use sparingly this feature, by explicitly tagging a file as check_sha_not_time kind of file. In the example above, the user would tag config.h as check_sha_not_time, and avoid recompilation of pretty much the whole library.
Note 0: I know little of how cmake internally works, so my suggestion of using sha rather than timestamp could be completely crazy and/or impossible given cmake implementation. I apologize for that. But that's why one asks things here, cause he/she doesn't know, right?
Note 1: I also tried using ccache, but unsuccessfully. Perhaps I need to use some particular flag or configuration option in ccache to trigger this capability.
Note 2: I want to avoid duplicating the source folder.
We’re currently upgrading our archaic build system from a bunch of batch scripts to a makefile system using NMake. It’s challenging as we use a custom intermediate language that ends up getting translated to C++ where some of our translators can generate 10’s of files what have a common parts in the file names. The other challenging thing is we use a bunch of CSV files to configure our interfaces and these files get passed through to our configuration tools which generate more source code files. Right now I am focusing on creating the simple rules for our configuration files but can’t seem to figure out a way associate a dependency with a rule if the dependency exists. I tried to use $(wildcard xxx.csv) but found out that this command doesn’t exist for NMake like it does for GNU Make.
So how can I create my rule so that it executes and runs my commands if I have two dependency csv files that will always exists and a third csv file that will exist only when my project calls for it?
[..] will exist only when my project calls for it?
This is a bit unclear. Assuming that there is a command that - depending on some external circumstances - might generate that third csv file, you could use a "stamp file" (I think they call it "pseudo target" in NMAKE):
stamp:
command_that_might_generate_csv3
touch stamp # updates timestamp of "stamp" (or creates it)
target: csv1 csv2 stamp
command_using_all_of csv1 csv2 csv3
I have a project consisting of a set of makefiles that CANNOT be run with make --jobs=N because the dependencies are not specified tightly enough for make to correctly execute the recipes in correct order (ie I get race conditions).
I am currently using Huddle, by Electric-Cloud.com, and it does exactly what I need: it parses the makefile and then executes the jobs in parallel and accounts for the unspecified dependencies.
Question: is there a free or free-er thing that does this?
Yes I know I could re-write the makefiles but project management says "no way".
UPDATE #1
I understand that I'll have to do some work to get functionality similar to Electric-Cloud's functionality.
I know that Electric-Cloud parses the makefile(s) to find the dependencies so wouldn't the same thing be accomplished using makedepend?
I'm thinking:
Run makedepend on existing makefiles
Feed in the output using include <makedepend.output>
make all --jobs=64
UPDATE 2
Turns out makedepend is specific to C/C++: it merely runs the pre-processor on source files and parses any #include statements; not what I need.
I need what this guy is asking for:
Build a makefile dependency / inheritance tree
UPDATE 3
The makefile "dependency graph generator" actually already exists
http://plindenbaum.blogspot.com/2012/11/visualizing-dependencies-of-makefile.html?m=1
but that's not going to help me.
Many of my recipes create directories which are used by other targets' recipes, effectively making them implicit prerequisites.
The graph dependency tool at above URL works by parsing the build log's statements but those statements don't indicate the implicit dependencies.
Even if I try to run my makefile with --dry-run, the build fails because some of the recipes that aren't executed - cause it's a dry run - create directories that other invocations of make need simply to 'pretend execute' a recipe.
UPDATE 4
Electric-Cloud has made Huddle - 4 local cores, non-clustered - free for anyone forever.
Furthermore, they output an .xml file that lists each job's dependencies so I can use it to fix my makefiles compatible so they're compatible with the --jobs option.
I am currently using Huddle, by Electric-Cloud.com, and it does exactly what I need: it parses the makefile and then executes the jobs in parallel and accounts for the unspecified dependencies.
I actually don't know about these tools, but can't you provide them with a super makefile under your control, that clarifies the inner dependencies of the various targets?
You probably just have to add some indirection level for these (imported?) projects directory structure and another Makefile.
I'm using SCons to build a very large project, with many buildable sub-projects. I can easily use keyword commands like scons group=ai to build the AI sub-projects with if statements (choosing the right SConscripts based on the keyword command), but I want to make it as easy as possible for others to use scons. Ideally, I'd like to use it like so: scons ai to build the AI components. However, the only single-word command functionality I've found in SCons so far is aliasing, and all the examples are about changing the target. This is not what I want. Since I have a very large project with multiple sub-SConscript files to build the subprojects, I want to call the SConscripts selectively. I've tried code like so:
env.Alias("ai", SConscript("ai/SConscript", 'env'))
but this calls the AI SConscript every time, regardless of whether I use the "ai" alias or a different one. Does anyone know if it is possible to use aliasing this way to selectively call SConscripts based on the alias?
As you mentioned, the Alias() function is only used for targets. I can think of 2 ways to solve this
Alias() can be called multiple times for the same alias with different targets, so you could call it for all targets in each SConscript, then you could build everything in a SConscript. Here's an example of what I mean:
ai/SConscript:
# targets, etc
env.Alias("ai", target1)
env.Alias("ai", target2)
...
env.Alias("ai", targetn)
Another option would be to put some logic in your root SConstruct so it only calls sub-project SConscript's based on a command line argument. This option would require you to use a command line argument of this form: group=ai