A company I am working for, has some c binaries build with ant using cpptask. They use ivy to retrieve shared c libraries every time we start a build which wastes a significant amount of time comparing the revisions and downloading, when then only need to be download if the header files have changed. I have added a target which sets a var, which causes the build to skip over the ivy steps but I'd like a better solution. I see that cpptask creates a file history.xml and only rebuilds to binary if any of the sources have change. I'd like to know if there is way to independently test if the binary needs to build, and it does, I'd like it fire off the ivy targets. I'd also like for a variable to be set if the binary was rebuilt so that I can conditionally start an rpm generation task
<project name="conditional_compile" default="build">
<condition property="file.modified">
<isfileselected file="test.txt">
<modified/>
</isfileselected>
</condition>
<target name="build" if="file.modified">
<echo message="This is a compile step that depends on the modification of a file"/>
</target>
</project>
Related
I am working differentially building a huge monolithic solution that includes about 80 projects. In my build pipeline right now I include a step to build the entire solution. But what I'd like to do is to build the solution but provide conditions as msbuild arguments so that I can exclude some of the projects that might not have any changes associated with them. I already have scripts to go through my commits and realize what changed and which projects need to be built.
I just need a way to send that info to MSBuild so that it does not build all projects everytime. I tried building projects separately but that takes a whole lot more time than just building the solution together.
So, I'm looking for any solutions out there through which I can specify to MSBuild that skip a specific project would help a lot. Thanks much!
I already have scripts to go through my commits and realize what
changed and which projects need to be built.
Since I could get clearly know that which script are you using to realize what changed and which projects need to be built. I am assuming that you are using MSbuildTarget script which in the xx.csproj to do these judgement.
=If I did not have misunderstanding, you can get help from this similar issue (See ilya's answer).
See this document and you'll find the build action is performed by these three targets, BeforeBuild,CoreBuild and AfterBuild. So assuming you have a target to go through my commits and realize what changed and if a project need to be built, you can add script like below to xx.csproj:
<PropertyGroup>
<BuildWrapperDependsOn>$(BuildDependsOn)</BuildWrapperDependsOn>
<BuildDependsOn>CheckIfBuildIsNeeded;BuildWrapper</BuildDependsOn>
</PropertyGroup>
<Target Name="CheckIfBuildIsNeeded">
<!-- Execute command here that checks if proceed with the build and sets the exit code -->
<Exec Command="exit /b 1" WorkingDirectory="$(SourcesPath)" IgnoreExitCode="true">
<Output TaskParameter="ExitCode" PropertyName="ExecExitCode"/>
</Exec>
<Message Text="Exit Code: $(ExecExitCode)" Importance="high" />
<PropertyGroup Condition="'$(ExecExitCode)' == '1'">
<DoBuild>false</DoBuild>
</PropertyGroup>
</Target>
<Target Name="BuildWrapper" Condition=" '$(DoBuild)' != 'false' " DependsOnTargets="$(BuildWrapperDependsOn)" Returns="$(TargetPath)" />
Above is the script from ilys, and hope my description can help you understand it. With this script, when we start a build target, it will firstly run the targets it depends on, so it will run the CheckIfBuildIsNeeded target and BuildWrapper target. And only when the DoBuild property is true, the BuildWrapper will actually execute. And since buildwrapper depends on original $(BuildDependsOn), it will continue the real build process.
The total logic is: Run CheckIfBuildIsNeeded script and output value to indicates whether need to build=>Try to Run BuildWrapper=>IF need to build, then run the real build success(BeforeBuild, Corebuild,Afterbuild), if the value is false, finish the build process. So I think you can do some little changes to this script then it can work for your situation. (Not sure what your script looks like, I can't complete it for you)
And since you have many projects, you don't need to add this script to every project manually. You can create a Directory.Build.props file, copy the script into it, and place the file in solution folder, then it will work for all projects in the solution.
I have a VS 2015 C++ project with both PreBuild and PostBuild steps.
In addition I have a Custom Target added to the project by "Build Dependencies -> Build Customization". The Custom Target runs a Perl script which runs nmake building files with Intel Compiler. The custom target always runs. Specifically the Perl script always runs while nmake checks for changes and prevents building if input files have not changed.
Invoking the custom target causes the PreBuild and PostBuild to run even if the custom target did not produce and new output (it ran but did nothing but checks).
I want to prevent PreBuild and PostBuild to run if my Custom Target didn't produce any new output. So far I didn't find a way to do this.
Another option is to prevent the custom target from running if sources have not changed. Unfortunately the files built by the Intel compiler are marked as "Exclude From Build" and thus do not trigger the custom target. I tried to define Input & Output for the task run by the custom target with no luck.
Any help will be highly appreciated!
Make sure your custom targets have an Inputs and Outputs attribute which properly describes which files will be used as input and which one were the resulting output. MsBuild will use the timestamp on these files to decide whether you actually changed anything. The timestamp on these files must be older than the file that would be generated as output from the target, that's how MsBuild decides.
Example:
<Target Name="Custom"
Inputs="#(CSFile)"
Outputs="hello.exe">
<Csc
Sources="#(CSFile)"
OutputAssembly="hello.exe"/>
</Target>
See also:
https://msdn.microsoft.com/en-us/library/ms171483.aspx
You can use transforms to map input to output if there is a logical relationship between the two:
<Target Name="Convert"
Inputs="#(TXTFile)"
Outputs="#(TXTFile->'%(Filename).content')">
<GenerateContentFiles
Sources = "#(TXTFile)">
<Output TaskParameter = "OutputContentFiles"
ItemName = "ContentFiles"/>
</GenerateContentFiles>
</Target>
https://msdn.microsoft.com/en-us/library/ms171483.aspx
Do not rely on BeforeTargets and AfterTargets and never rely on PreBuildEvent, as that target itself doesn't have any inputs or outputs and thus always triggers, they're quite old constructs, stemming from the 2003 era, instead override BuildDependsOn and inject your target in the chain.
Example:
<PropertyGroup>
<BuildDependsOn>
Convert;
$(BuildDependsOn);
</BuildDependsOn>
</PropertyGroup>
See:
https://blogs.msdn.microsoft.com/msbuild/2006/02/10/how-to-add-custom-process-at-specific-points-during-build-method-2/
I've written a MSBuild project files which try to build in parallel all the configs of my VS2010 solution:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<ItemGroup>
<BuildFile Include="$(SourceRoot)MyProject.sln" />
<Config Include="Debug">
<Configuration>Debug</Configuration>
</Config>
<Config Include="Release">
<Configuration>Release</Configuration>
</Config>
</ItemGroup>
<Target Name="BuildAll" Outputs="%(Config.Configuration)">
<Message Text="Start building for configuration: %(Config.Configuration)" />
<MSBuild Projects="#(BuildFile)"
Properties="Configuration=%(Config.Configuration)"
Targets="Build" />
</Target>
</Project>
And I launch msbuild with:
msbuild /m /p:BuildInParallel=true /t:BuildAll buildall.proj
The problem is that my solution have many .Net projects which all have the same output folder. These projects use also the same external assemblies.
So very often, two output executables are generated at the same time and their dependencies copied at the same time. This leads to errors like:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(3001,9):
error MSB3021:
Unable to copy file "xxx\NLog.dll" to "D:\src\Blackbird\shared\bin\debug\NLog.xml". The process
cannot access the file 'xxx\NLog.dll because it is being used by another process.
which I think means: "2 different projects use NLog and try to copy its assembly in the output folder at the same time"...
Is there a way to get around that? I really would like to avoid to modify all the projects in the solution.
Looking at the task source code "C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(3001,9)", I've seen that it is possible to make msbuild retry the copy:
<Copy
SourceFiles="#(ReferenceCopyLocalPaths)"
DestinationFiles="#(ReferenceCopyLocalPaths->'$(OutDir)%(DestinationSubDirectory)%(Filename)%(Extension)')"
SkipUnchangedFiles="$(SkipCopyUnchangedFiles)"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForCopyLocalIfPossible)"
Condition="'$(UseCommonOutputDirectory)' != 'true'"
>
I've try to set the variables CopyRetryCount, CopyRetryDelayMilliseconds, ... I was hoping that if the copy fails, another copy done a few milliseconds later would succeed. But I've been unable to set these parameters. How can I change them?
Is there another solution?
I've found the solution
<MSBuild Projects="#(BuildFile)"
Properties="Configuration=%(Config.Configuration);Retries=10;RetryDelayMilliseconds=50"
Targets="Build" />
It works as expected but now it generates a warning before retrying the copy
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(3001,9):
warning MSB3026: Could not copy
"D:\src\xxx\System.Data.SQLite.pdb" to "..\Debug\System.Data.SQLite.pdb".
Beginning retry 1 in 50ms. The process cannot access the file
'..\Debug\System.Data.SQLite.pdb' because it is being used by another process.
During my last test, it generated 36 times this warning! Is there a way for suppressing the warning MSB0326?
In general, anything that runs during build must not acquire an exclusive lock on its inputs - only shared-read lock. It appears that something in your build process (probably NLog, whatever that is) violates this - it takes exclusive lock on the input "xxx\NLog.dll", so when another msbuild node tries to copy the same input it fails.
Retry is a reasonable workaround for the particular symptom you have - though it's not guaranteed to always succeed.
I had the same problem. I needed to add appropriate exception to my AntiVirus to avoid sacanning DLLs produced by MsBuild.
I've read about the use of "Custom.Before.Microsoft.Common.targets" and "Custom.After.Microsoft.Common.targets" in order to execute a custom target before/after every project build and I would like to use this technique in order to change version info while building on our TeamCity build server.
The problem is that although it works for C# projects, it doesn't seem to work for native C++ projects.
After some digging around in the Microsoft.Cpp.targets file I found out that for native C++ projects this seems to be implemented through setting $(ForceImportBeforeCppTargets) and $(ForceImportAfterCppTargets).
I can't seem to find a single piece of information on the web about this technique for native C++ apps though, so I'm asking if I'm looking in the right direction or not.
Any help is appreciated.
For VC++ projects it is a bit different. You define a file to be imported either at the beginning or at the end of the project. To use this approach you need to define values for the properties ForceImportBeforeCppTargets or ForceImportAfterCppTargets. For example if you want a file to be included at the beginning of the project you can pass in the value at the command line. For example I just created a dummy VC++ project named CppTets01. Then I created the two sample files below.
Before.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="CustomTargetInBefore" AfterTargets="Build">
<Message Text="From CustomTargetInBefore" Importance="high"/>
</Target>
</Project>
After.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="CustomTargetInAfter" AfterTargets="Build">
<Message Text="From CustomTargetInAfter" Importance="high"/>
</Target>
</Project>
Then I executed the following command:
msbuild CppTest01.vcxproj
/p:ForceImportBeforeCppTargets="C:\Temp\_NET\ThrowAway\CppTest01\CppTest01\Before.proj";
ForceImportAfterCppTargets="C:\Temp\_NET\ThrowAway\CppTest01\CppTest01\After.proj"
The result was
C:\Temp_NET\ThrowAway\CppTest01\CppTest01>msbuild CppTest01.vcxproj /p:ForceImportBeforeCppTargets="C:\Temp_NET\ThrowAway\CppTest01\C
ppTest01\Before.proj";ForceImportAfterCppTargets="C:\Temp_NET\ThrowAway\CppTest01\CppTest01\After.proj"
Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.1]
Copyright (C) Microsoft Corporation 2007. All rights reserved.
Build started 10/18/2010 8:32:55 AM.
Project "C:\Temp\_NET\ThrowAway\CppTest01\CppTest01\CppTest01.vcxproj" on node 1 (default targets).
InitializeBuildStatus:
Creating "Debug\CppTest01.unsuccessfulbuild" because "AlwaysCreate" was specified.
ClCompile:
All outputs are up-to-date.
All outputs are up-to-date.
ManifestResourceCompile:
All outputs are up-to-date.
Link:
All outputs are up-to-date.
Manifest:
All outputs are up-to-date.
FinalizeBuildStatus:
Deleting file "Debug\CppTest01.unsuccessfulbuild".
Touching "Debug\CppTest01.lastbuildstate".
CustomTargetInBefore:
From CustomTargetInBefore
CustomTargetInAfter:
From CustomTargetInAfter
Done Building Project "C:\Temp\_NET\ThrowAway\CppTest01\CppTest01\CppTest01.vcxproj" (default targets).
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:00.21
As you can see from the output the targets were successfully injected into the build process. If you want to relate this back to Custom.Before.Microsoft.Common.targets and Custom.Before.Microsoft.Common.targets then you should know that the technique used there is a bit different. Specifically if you create those files they are automatically imported into every C#/VB.NET project. In this case you have to set this property. You really have two options here:
You can set this property as an environment variable
You can use another technique, ImportBefore & ImportAfter which is specific to VC++
For #1 let me explain a bit. In MSBuild when you access a property with the syntax $(PropName) then if a property with the name PropName doesn't exist MSBuild will look up in the environment variables to see if such a value exists, if it does then that value is returned. So if you have a build server in which you want to include a file for each VC++ build, then just create those properties as environment variables. Now for the other technique.
ImportBefore/ImportAfter
In VC++ a new concept is introduced. In Microsoft.Cpp.Win32.targets you can see the declaration at the top of the .targets file.
<Import Project="$(VCTargetsPath)\Platforms\Win32\ImportBefore\*.targets"
Condition="Exists('$(VCTargetsPath)\Platforms\Win32\ImportBefore')" />
Then there is one towards the bottom
<Import Project="$(VCTargetsPath)\Platforms\Win32\ImportAfter\*.targets"
Condition="Exists('$(VCTargetsPath)\Platforms\Win32\ImportAfter')" />
A similar import declaration exists for the other target platforms as well. Take a look at the files at %ProgramFiles32%\MSBuild\Microsoft.Cpp\v4.0\Platforms\ for the specific names.
With this technique if you want a file to be imported then simply create a file that ends with .targets and place it into the appropriate folder. The advantage of this is that it will be imported into every VC++ build for that platform, and that you can create many different files. The drawback is that you have to place them in those specific folders. That's the main difference between both techniques. With this first technique you can specify the file location via property and its not automatically included for every build, but for the second approach it is but you cannot change the location
You can also add project content into one of *.props files from directory %LOCALAPPDATA%\Microsoft\MSBuild\v4.0\
It make same effect.
There are plenty of guides out there which help you mimic VS2008's "Custom Build Step" in VS2010 with MSBuild. However, I'd like my build to be smarter and make use of MSBuild. I've written a little MSBuild task which invokes the ANTLR parser generator. That build task works flawlessly when I run it in a simple test MSBuild file. However, when I try to add my task to a C++ project, I run into problems. Essentially I've added this to the top of my project file (Right after the <project> element):
<UsingTask TaskName="ANTLR.MSBuild.AntlrGrammar"
AssemblyName = "ANTLR.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d50cc80512acc876" />
<Target Name="BeforeBuild"
Inputs="ConfigurationParser.g"
Outputs="ConfigurationParserParser.h;ConfigurationParserParser.cpp;ConfigurationParserLexer.h;ConfigurationParserLexer.cpp">
<AntlrGrammar
AntlrLocation="$(MSBuildProjectDirectory)Antlr.jar"
Grammar="ConfigurationParser.g"
RenameToCpp="true" />
</Target>
However, my target is not being called before build.
How can I add my task to a C++ build?
Before reading this answer, you'll probably want to see:
General .vcxproj File Reference
The New Way of doing Build Extensibility in .NET 4
The old way of extending MSBuild, and the one mentioned by the reference book I have, essentially is based on overriding default-empty targets supplied by Microsoft. The new way, as specified in the second link above, is to define your own arbitrary target, and use the "BeforeTargets" and "AfterTargets" properties to force your target to run before or after your intended target.
In my specific case, I needed the ANTLR Grammars task to run before the CLCompile target, which actually builds the C++ files, because the ANTLR Grammars task builds .cpp files. Therefore, the XML looks like this:
<Project ...
<!-- Other things put in by VS2010 ... this is the bottom of the file -->
<UsingTask TaskName="ANTLR.MSBuild.AntlrGrammar"
AssemblyName = "ANTLR.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d50cc80512acc876" />
<Target Name="AntlrGrammars"
Inputs="Configuration.g"
Outputs="ConfigurationParser.h;ConfigurationParser.cpp;ConfigurationLexer.h;ConfigurationLexer.cpp"
BeforeTargets="ClCompile">
<AntlrGrammar
AntlrLocation="$(MSBuildProjectDirectory)\Antlr.jar"
Grammar="Configuration.g"
RenameToCpp="true" />
</Target>
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
</Project>
As for why this is superior to a PreBuildEvent and/or PostBuildEvent; this is smart enough to not rebuild the .cpps when the grammar itself is not updated. You'll get something like:
1>AntlrGrammars:
1>Skipping target "AntlrGrammars" because all output files are up-to-date with respect to the input files.
1>ClCompile:
1> All outputs are up-to-date.
1> All outputs are up-to-date.
This also silences Visual Studio's incessant complaining every time you run the program that it needs to rebuild things, like it does with plain pre- and post- build steps.
Hope this helps someone -- took me frickin forever to figure out.