Pre-Processing C++ file using MSBuild and Visual Studio 2012+ - c++

Question

I have a series of standard cpp files and each of these files contain a file-specific #include statement. However, the content of those included files must be populated by a pre-processing tool prior to invoking the standard C++ compiler.
The tricky part is that I want this to be fully integrated into Visual Studio using MSBuild. Therefore, when I bring up Visual Studio's Property Window on a cpp file, I want to see all the standard C++ compiler options and, ideally, some Custom Properties controlling the pre-processor tool. As a OOP analogy, I want my build tool to inherit everything from the standard CL MSBuild Rule, and add some Custom Properties & Build Steps to it.
I have successfully done this through an extremely laborious process of basically creating a Custom MSBuild Rule and Copy/Paste most of the C++ Options into my Custom Rule. Finally, I then pass along the million C++ Options to the standard C++ compiler through the CommandLineTemplate entry in my MSBuild .props file. It's ridiculously complicated and the C++ options don't automatically get updated as I update Visual Studio.
I can find plenty of examples of Custom MSBuild Rules, but I haven't been able to find one where it piggybacks onto an existing one.

Answer 1

Not a lot of love for MSBuild, I take it...
Anyway, after years of going back and forth on that one, I finally found something, soon after I posted my question. The key was to search for "extending" existing Rule which, apparently, I hadn't tried before.
Usually, when you create a Build Customization in VS, you end up with 3 files:
MyCustomBuild.xml:
Contains the Properties & Switches, as shown on the VS's Property Sheet.
MyCustomBuild.props:
Contains default values for those Properties. They can be made conditional through the use of the Condition attribute.
MyCustomBuild.targers:
Contains a line to load up your xml and the Target/Task entries.
So the first part was to extend the existing C/C++ Properties as shown in Visual Studio. I found this link, which finally gave me something to work with:
https://github.com/Microsoft/VSProjectSystem/blob/master/doc/extensibility/extending_rules.md
Here's the xml bit.
<Rule
Name="RuleToExend"
DisplayName="File Properties"
PageTemplate="generic"
Description="File Properties"
OverrideMode="Extend"
xmlns="http://schemas.microsoft.com/build/2009/properties">
<!-- Add new properties, data source, categories, etc -->
</Rule>
Name attribute:
The Name attribute must match the rule being extended. In this case, I wanted to extend the CL rule, so I set that attribute = "CL".
DisplayName attribute:
This is optional. When provided, it will overwrite the tool's name seen on the Property Sheet. In this case, the tool name shown is "C/C++". I can change it to show "My C/C++" by setting this attribute.
PageTemplate attribute:
If this attribute is provided, it must match the overwritten rule's value. In this case, it would be "tool". Just leaving it out seems to work fine. I suspect this could be needed if 2 rules had the same name, but a different template. You could use this to clarify which one you wanted to extend.
Description attribute:
Optional. I don't know where that even shows up within the VS GUI. Maybe it's just for documenting the xml file.
OverrideMode attribute:
This is an important one! It can be set to either "Extend" or "Replace". In my case, I chose "Extend".
xmlns attribute:
Required. Doesn't work properly if not present.
As the link suggest, you can then provide the properties, data source and categories. Keep in mind that categories are usually displayed in the order they appear in the xml file. Since I was extending an existing Rule, my custom categories would all show up after the standard C/C++ categories. Given that my tool is for pre-processing the files, I would have preferred having my custom options at the top of the Property Sheet. But I couldn't find way around that.
Note that you do NOT need the ItemType/FileExtension or ContenType Properties, typically found for custom Rules.
So once I entered all of that, my custom pre-processing options showed up alongside the standard C/C++ properties on the Property Sheet. Note that all these new properties would be attached to the "ClCompile" Item list, with all the other C/C++ properties.
The next step was to update the .props file. I'm not going to get into it since it's pretty much standard when create these custom build Rules. Just know that you need to set them using the "ClCompile" Item, as mentioned above.
The final step was to get the .targets file to do what I wanted.
The first part was to "import" (not really an import entry) the custom Rule through the typical entry:
<ItemGroup>
<PropertyPageSchema Include="$(MSBuildThisFileDirectory)MyCustomBuild.xml" />
</ItemGroup>
Then I needed to pre-process every source file. Ideally, it would have been nicer to pre-process a file and then compile it - one file a time. I could have done this by overwriting the "ClCompile" target within my own .targets file. This Target is defined under the "Microsoft.CppCommon.targets" file (location under "C:\Program Files (x86)" varies, depending on the VS version). I basically could have Cut & Pasted the whole Target into my file, then add my pre-processing task code before the "CL" Task. I also would have needed to convert the Target into a Target Batch, by adding an "Outputs=%(ClCompile.Identity)" attribute to the "ClCompile" Target. Without this, my pre-processing task would have ran on all files before moving on to the "CL" task, bringing me back to square one. Finally, I would have needed to deal with Pre-Compiled Header files, since they need to be compiled first.
All of this was just too much of a pain. So I selected the simpler option of defining a Target which looks like this:
<Target Name="MyPreProcessingTarget"
Condition="'#(ClCompile)' != ''"
Outputs ="%(ClCompile.Identity)"
DependsOnTargets="_SelectedFiles"
BeforeTargets="ClCompile">
There are a number of attributes defined but the most important one is the BeforeTargets="ClCompile" attribute. This is what forces this target to execute before the cpp files are compiled.
I also chose to do a Target Batch processing here [Outputs ="%(ClCompile.Identity)"] because it was just easier to do what I wanted to do, if I assumed to have 1 file being processed at a time, in my Target.
The attribute DependsOnTargets="_SelectedFiles" is used to know if a GUI user has some selected file within VS Solution Explorer. If so, the files will be stored in the #(SelectedFiles) Item List (generated by the "_SelectedFiles" Target). Typically, when selecting specific files within the Solution Explorer and choosing to compile them, VS will forcefully compile them even if they are up-to-date. I wanted to preserve that functionality for the automatically-generated pre-processed include files, and forcefully regenerate them as well, for those selected files. So I added this block:
<ItemGroup Condition="'#(SelectedFiles)' != ''">
<IncFilesToDelete Include="%(ClCompile.Filename)_pp.h"/>
</ItemGroup>
<Delete
Condition="'#(IncFilesToDelete)' != ''"
Files="%(IncFilesToDelete.FullPath)" />
Note that the automatically-generated include files are named SourceFileName_pp.h. By deleting those files, my pre-processing Task will forcefully re-generate them.
Next, I build a new Item list from the "ClCompile" Item list, but with the "_pp.h" versions of the files. I do so with the following code:
<ItemGroup>
<PPIncFiles
Condition="'#(ClCompile)' != '' and '%(ClCompile.ExcludedFromBuild)' != 'true'"
Include="%(ClCompile.Filename)_pp.h" />
</ItemGroup>
The final part is a little uglier.
In order to run my pre-processing exe, I use the standard "Exec" Task. But I obviously only want to run it if the source file is newer than the generated file. I do so by storing the well-known metadata "ModifiedTime" of the source file, and the generated file into a couple of dynamic Properties. But I can't use the ModifiedTime metadata directly, as it's not an comparable value. So I used the following code, which I have found on StackOverflow here:
Comparing DateTime stamps in Msbuild
<PropertyGroup>
<SourceFileDate>$([System.DateTime]::Parse('%(ClCompile.ModifiedTime)').Ticks)</SourceFileDate>
<PPIncFileDate Condition="!Exists(%(PPIncFiles.Identity))">0</PPIncFileDate>
<PPIncFileDate Condition="Exists(%PPIncFiles.Identity))">$([System.DateTime]::Parse('%(PPIncFiles.ModifiedTime)').Ticks)</PPIncFileDate>
</PropertyGroup>
Note that I can store the timestamps in Properties, given that the Item Lists only contain one Item per Target Pass, because of Target Batching.
Finally, I can invoke my pre-processor using the "Exec" Task, as follows:
<Exec
Condition="'#(PPIncFiles)' != '' and $(SourceFileDate) > $(PPIncFileDate)"
Command="pptool.exe [options] %(ClCompile.Identity)" />
Supplying the options was yet, another headache.
Typically, the switches as defined under the xml file are just passed to a "CommandLineTemplate" metadata under the .props file using [OptionName]. This will pass the "Switch" attribute of the Property defined under the xml file. But that implies defining your own TaskName item, made from a TaskFactory, under the .targets file. But in my case, I was just using the existing "Exec" Task, which doesn't know anything about my custom Properties. I didn't know how to retrieve the "Switch" attribute in this case, and what seems to be available is just whatever the "Name" attribute contains. Luckily, a Property has both a Name and a DisplayName. The DisplayName is what the GUI user sees. So I just copied the "Switch" value into the "Name" value, when defining the Properties under the xml file. I could then pass the option to the Exec Task using something like:
<Exec
Condition="'#(PPIncFiles)' != '' and $(SourceFileDate) > $(PPIncFileDate)"
Command="pptool.exe %(ClCompile.Option1) %(ClCompile.Option2)... %(ClCompile.Identity)" />
Where I defined all my Properties as "EnumProperty", with an "EnumValue" having Name="" for disabled options, and other EnumValue having Name="switch" for the others. Not very elegant, but I didn't know a way around this.
Finally, it is recommended that when automatically generating files, the .targets file should also include a way to clean them up when the user Cleans up the Project. That's pretty standard but I'll include it here for convenience.
<PropertyGroup>
<CleanDependsOn>$(CleanDependsOn);PPIncCleanTarget</CleanDependsOn>
</PropertyGroup>
<Target Name="PPIncCleanTarget" Condition ="'#(ClCompile)' != ''">
<ItemGroup>
<PPIncFilesToDelete Include="%(ClCompile.Filename)_pp.h" />
</ItemGroup>
<Delete Files="%(PPIncFilesToDelete.FullPath)" Condition="'#(PPIncFilesToDelete)' != ''"/>
</Target>