ivy report for entire repo? - build

I'm working on an internal ivy repository with a decent number of projects under it, each with many revisions. I would like to make a dependency report for the entire repository showing which versions of which artifacts depend on which revisions of other artifacts. Obviously it isn't too difficult to make a script to parse the published ivy xml files, but if this functionality exists already I'll use that. Something like the repreport task would be nice, but for a whole repo.
My main goal here is to get a report of artifacts that are not referenced by any other artifacts so as to make a list of candidates for removal from the repo.
So, does ivy have any way to build a dependency report against and entire repository?
Edit: Working through this, it looks like ivy:repreport is the way to go.
Here is my build.xml file:
<project name="Report Build" xmlns:ivy="antlib:org.apache.ivy.ant" basedir=".">
<property name="ivy.version" value="2.2.0"/>
<property name="ivy.home" value="${user.home}/.ivy2"/>
<target name="fetch-ivy" unless="offline" description="Install Ivy if it doesn't already exist">
<mkdir dir="${ivy.home}"/>
<get
src="http://repo2.maven.org/maven2/org/apache/ivy/ivy/${ivy.version}/ivy-${ivy.version}.jar"
dest="${ivy.home}" usetimestamp="true"/>
</target>
<target name="init-ivy" depends="fetch-ivy" unless="ivy-initialized">
<path id="ivy.lib.path">
<fileset dir="${ivy.home}" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml" uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
<property name="ivy-initialized" value="yes"/>
</target>
<target name="report" depends="init-ivy">
<ivy:settings file="ivy-settings-report.xml" id="report.ivy.settings"/>
<ivy:repreport settingsref="report.ivy.settings"/>
</target>
</project>
And here is my ivy settings file:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<settings defaultResolver="main"/>
<resolvers>
<chain name="main">
<url name="internalartifacts" m2compatible="false">
<artifact
pattern="http://internalartifacts.local/[organization]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/>
<ivy pattern="http://internalartifacts.local/[organization]/[module]/[revision]/ivy-[revision].xml"/>
</url>
</chain>
</resolvers>
</ivy-settings>

The documentation for repreport says:
To generate a xml report for all the latest versions of all the
modules in your repository:
<ivy:repreport />
Limitation: this task requires to be able to browse the repository,
and is thus limited to resolvers supporting repository listing. In
particular, it means it doesn't work to report all organizations in a
repository using m2compatible mode. Moreover, to be able to list
organizations, this task requires an [organisation] token in the
resolver(s) used.
So this should totally work.
As mentioned in the comment:
It is important that your repository has ivy.xml files for the artifacts in it. Otherwise ivy cannot recognize the dependencies between the artifacts and your report will be empty.

Related

How to extend msdeploy/publishing pipeline for .NET Core apps

Old style/.NET Framework webapps in Visual Studio have a publish pipeline that could be customized. For example, a Parameters.xml file could be specified, like so:
<PropertyGroup>
<WebAppPackageConfigDir Condition=" '$(WebAppPackageConfigDir)' == '' ">$(MSBuildProjectDirectory)\Package</WebAppPackageConfigDir>
<ProjectParametersXMLFile>$(WebAppPackageConfigDir)\Parameters.xml</ProjectParametersXMLFile>
</PropertyGroup>
so that when the MSDeploy package is built, the MSDeploy parameters defined therein are incorporated into the package. The Web Processing Pipeline (mostly defined in C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\Web\Microsoft.Web.Publishing.targets) supports customizing the MSDeploy package in multiple ways - for example, package parameters can be defined in a file or in an MSBuild <ItemGroup>.
In SDK style webapps (eg ASP.NET Core webapps), a "new" publishing pipeline is used starting in eg C\Program Files\dotnet\sdk\2.1.2\Sdks\Microsoft.NET.Sdk\build\Microsoft.NET.Publish.targets. This pulls in additional targets files depending on the <PublishProtocol> value, eg C\Program Files\dotnet\sdk\2.1.2\Sdks\Microsoft.NET.Sdk.Publish\build\netstandard1.0\PublishTargets\Microsoft.NET.Sdk.Publish.MSDeploy.targets for PublishProtocol=MSDeploy or .\Microsoft.NET.Sdk.Publish.MSDeployPackage.targets for PublishProtocol=MSDeployPackage. Both files define a _CreateParametersFiles target as:
<!--
***********************************************************************************************
TARGET : _CreateParameterFiles
***********************************************************************************************
-->
<Target Name="_CreateParameterFiles">
<ItemGroup>
<MsDeployDeclareParameters Remove="#(MsDeployDeclareParameters)" />
<MsDeployDeclareParameters Include="IIS Web Application Name" >
<Kind>ProviderPath</Kind>
<Scope>IisApp</Scope>
<Match>$(PublishIntermediateOutputPath)</Match>
<Description></Description>
<DefaultValue>$(DeployIisAppPath)</DefaultValue>
<Value>$(DeployIisAppPath)</Value>
<Tags>IisApp</Tags>
<Priority></Priority>
<ExcludeFromSetParameter>false</ExcludeFromSetParameter>
</MsDeployDeclareParameters>
</ItemGroup>
<ItemGroup Condition="'#(_EFSQLScripts)' != ''">
<MsDeployDeclareParameters Include="%(_EFSQLScripts.DBContext)">
<Kind>ProviderPath</Kind>
<Scope>dbfullsql</Scope>
<Match>%(_EFSQLScripts.Identity)</Match>
<Description></Description>
<DefaultValue>%(_EFSQLScripts.ConnectionString)</DefaultValue>
<Value>%(_EFSQLScripts.ConnectionString)</Value>
<Tags>dbfullsql</Tags>
<Priority></Priority>
<ExcludeFromSetParameter>false</ExcludeFromSetParameter>
</MsDeployDeclareParameters>
</ItemGroup>
<CreateParameterFile
Parameters="#(MsDeployDeclareParameters)"
DeclareSetParameterFile="$(_MSDeployParametersFilePath)"
IncludeDefaultValue="True"
OptimisticParameterDefaultValue="$(EnableOptimisticParameterDefaultValue)"
SetParameterFile="$(_MSDeploySetParametersFilePath)"
GenerateFileEvenIfEmpty="True" />
</Target>
It appears that there's no support for customizing deploy parameters. Similarly, the <Target Name="_CreateManifestFiles"> target appears to prohibit customization.
The WPP (Web Publishing Pipeline - the previous implementation) is/was clunky, but there were articles/blog posts about how to customize it. With ASP.NET Core, there is no mention of customizing or extending the deploy process, that I can find. I did find a Channel 9 Video on ASP.NET Publishing, but it seems outdated - it applies to VS 2017 with the project.json and powershell scripts used for deployment, and it appears that was scrapped in favor of MSBuild targets.
I also did find https://github.com/aspnet/websdk/blob/dev/README.md#microsoftnetsdkpublish, which documents a few parts of the msdeploy settings that can be changed, but I would still state that there is no support for extending the pipeline.
I'm interpreting the lack of available information to mean "figure it out yourself", which is challenging when dealing with MSBuild and MSDeploy. Has anyone had any success customizing the deploy pipeline for ASP.NET Core webapps?

How to use the Web Publishing Pipeline and Web Deploy (MSDEPLOY) to Publish a Console Application?

I would like to use web deploy to publish a Visual Studio "Console" application to a folder on the target system.
I have had some luck, and have been able to produce something similar to what I need, but not quite.
I've added the following to the console .csproj:
added the following projectName.wpp.targets file
<Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" />
and I've added the following projectName.wpp.targets:
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<PropertyGroup>
<DeployAsIisApp>false</DeployAsIisApp>
<IncludeSetAclProviderOnDestination>false</IncludeSetAclProviderOnDestination>
</PropertyGroup>
<ItemGroup>
<FilesForPackagingFromProject Include="$(IntermediateOutputPath)$(TargetFileName).config">
<DestinationRelativePath>bin\%(RecursiveDir)%(FileName)%(Extension)</DestinationRelativePath>
<FromTarget>projectName.wpp.targets</FromTarget>
</FilesForPackagingFromProject>
</ItemGroup>
</Project>
I then edit the .SetParameters.xml file as follows:
<parameters>
<setParameter name="IIS Web Application Name" value="c:\company\project" />
</parameters>
When I then deploy using the generated .cmd file, I get all the files deployed to C:\company\project\bin.
That's not bad, but I'd like to do better. In particular, I'd like to omit the "bin" folder and put all files in the "C:\company\project" folder, and I'd like to be able to specify the ACLs
Has anybody been able to work around these problems?
Ok, so here's the way how to omit the 'bin' folder.
First of all, I'd like to emphasize that all this msdeploy-related stuff is for web apps deployment, and 'bin' folder seems for me to be almost hardcoded deeply inside. So if you want to get rid of it - you have to do some dirty things. Which I did.
We'll have to change $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets project a little bit, so it's better to change not it, but it's copy.
Steps:
1.Backup $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets(alternatively, you could install MSBuild.Microsoft.VisualStudio.Web.targets package, redirect your csproj file to Microsoft.WebApplication.targets file obtained from package and work with it).
2. In the $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplicaton.targets find the xml node which looks like <CopyPipelineFiles PipelineItems="#(FilesForPackagingFromProject)"(there are several ones of them, take the one from the line ~2570).
3. Comment the node out, replace with the custom one, so eventually it will look like:
<!--
<CopyPipelineFiles PipelineItems="#(FilesForPackagingFromProject)"
SourceDirectory="$(WebPublishPipelineProjectDirectory)"
TargetDirectory="$(WPPAllFilesInSingleFolder)"
SkipMetadataExcludeTrueItems="True"
UpdateItemSpec="True"
DeleteItemsMarkAsExcludeTrue ="True"
Condition="'#(FilesForPackagingFromProject)' != ''">
<Output TaskParameter="ResultPipelineItems" ItemName="_FilesForPackagingFromProjectTempory"/>
</CopyPipelineFiles>-->
<!-- Copying files to package folder in 'custom'(dirty) way -->
<CreateItem Include="$(OutputPath)\**\*.*">
<Output TaskParameter="Include" ItemName="YourFilesToCopy" />
</CreateItem>
<Copy SourceFiles="#(YourFilesToCopy)"
DestinationFiles="#(YourFilesToCopy->'$(WPPAllFilesInSingleFolder)\%(RecursiveDir)%(Filename)%(Extension)')" />
Then
4. Your projectName.wpp.targets don't have to have FilesForPackagingFromProject, so it will look like:
<!-- targets -->
<PropertyGroup>
<DeployAsIisApp>false</DeployAsIisApp>
<IncludeSetAclProviderOnDestination>false</IncludeSetAclProviderOnDestination>
</PropertyGroup>
<ItemGroup>
<!-- intentionally left blank -->
</ItemGroup>
</Project>
That's it. Worked for me(tm), tested. Let me be honest, I don't like this approach, but that was the only way I made it working in the needed way. It's up to you whether you'll use it in your project or not.
My opinion is not to use msdeploy here - it was not for you task.
Better to write msbuild-scripts from scratch or accept the 'bin' folder, and fight against the framework again once next customization is required.

Call Publish after Build in MSBuild

I am trying to call the Publish target every time I build my WPF app. I have tweaked the .csproj file to include this:
<Target Name="AfterBuild">
<Message Text="Running AfterBuild..." />
<MSBuild Projects="$(MSBuildProjectFullPath)" Properties="Configuration=$(Configuration); PublishDependsOn=" Targets="Publish" />
</Target>
When I run this from the command line, I see the message that it is 'Running AfterBuild...' but nothing happens. If I remove the '; PublishDependsOn=' from the Properties of the MSBuild task, I get a circular reference error.
What magic am I missing here?
OK, I figured out how to do what I want to do. Instead of trying to explicitly call Publish in AfterBuild, I just added it to the DefaultTargets of the project. Now it calls Build then Publish.
<Project ToolsVersion="4.0" DefaultTargets="Build;Publish" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

Create code coverage report starting at root folder

I have just created a new zend framework application to try out unit testing.
I have followed this tutorial and everything seems to be working correctly for testing. There is a problem with the display of the coverage report. It displays the correct information, but the report starts at the root of my hard drive and I need to traverse the tree to my project folder to see useful information.
This means that every time I ran the tests, I need to click 5 folders deep to get to the actual report.
How do I make the report start in my project folder? This is my phpunit config file:
<phpunit bootstrap="./bootstrap.php">
<testsuite name="Application Test Suite">
<directory>./application</directory>
</testsuite>
<testsuite name="Library Test Suite">
<directory>./library</directory>
</testsuite>
<filter>
<whitelist>
<directory>../../library/Zend</directory>
<exclude>
<directory suffix=".phtml">../application/</directory>
<file>../application/Bootstrap.php</file>
<file>../application/controllers/ErrorController.php</file>
</exclude>
</whitelist>
</filter>
<logging>
<log type="coverage-html" target="./log/report" charset="UTF-8" yui="true"
hightlight="true" lowupperbound="50" highlowerbound="80">
<log type="testdox" target="./log/testdox.html">
</log>
</log>
</logging>
</phpunit>
I fixed the problem...
I needed to explicitly specify my application folder in the whitelist. If it is empty, the code coverage report just starts from 'c:' and tries to find every '.php' file.
After adding the line in the whitelist section:
<directory>../application/</directory>
It works as expected.
Since I don't have any library tests in my test folder, including the Zend library folder probably had no effect and the report must have considered the whitelist empty. And because there is no blacklist, it just started from the root.
The code coverage starts at the most common path for all files included in the report. So if your web root is in /var/www and you include libraries in /usr/local/zend/ the most common path will be the root path.
The solution would be to exclude the library path because usually you don't want to measure the code coverage for external libraries anyway.

Running multiple JUnit Tests with ANT for Regression and Granularity

I'm trying to set up an ANT build script which compile code, compile tests, run unittests and then build. These are all done through separate targets with dependencies i.e.
<target name="compile">
<javac>...
</target>
<target name="compile-tests" depends="compile">
<javac>...
</target>
<target name="unittest" depends="compile-tests">
<junit...
<test ...
<fail if="tests.failed" ..
</target>
<target name="build" depends="compile, unittest">
</target>
Each 'test' inside the 'junit' task focuses on one part of the application, (typically package by package) and points to a Junit TestSuite. This set up allows for all tests to be run when a build is called but this isn't ideal for day-to-day development.
I would like to be able to do 2 things:
Run all the tests in a build (like the setup shown above)
Run tests individually from ant
My solution for (2) was to use multiple antcall tasks which isn't really best practice. During these calls different properties were set to run all the tests as they each required a different property:
<!-- test package p2 with ant unittest -Dtest.p2=true -->
<target name="unittest" depends="compile-tests">
<junit...
<test if="test.p1" ...
<test if="test.p2"
<fail if="tests.failed" ..
</target>
<target name="unittestall">
<property name="test.p1" value="true"/>
...
</target>
<target name="build" depends="compile, unittest">
<antcall target="unittestall" />
<antcall target="clean" />
<antcall target="compile" />
</target>
This gave the granularity I required but meant alot of work was duplicated and ant's dependency features weren't being used to their full.
So my question is:
How can I best set up ANT and Junit so that all tests can be run as part of a build AND so that individual tests can be run?
Thankyou :)
from Joshua England
p.s. ANT 1.8 and Junit 4.10 :)
Something like this?
<target name="unittest-p1"></target>
<target name="unittest-p2"></target>
<target name="unittest-p3"></target>
<target name="unittest" depends="unittest-p1, unittest-p2, unittest-p3/>
You could then run all the tests by passing the unittest target:
ant unittest
(or any target which depends on unittest)
And you could run any individual test of set of tests by invoking the appropriate target, e.g.
ant unittest-p1
If you would end up with a lot of duplication in multiple junit targets, you could tidy that up by putting all the common stuff into a macrodef.