Build sqlproj on Azure DevOps - build

I'm trying to use Azure DevOps Pipelines to build my .NET Core 2.1 solution from GitHub. It includes a SQL project that has a TargetFrameworkVersion of v4.6.2. This project always fails to build.
Build FAILED.
/home/vsts/work/1/s/MySolution/MyDatabase/MyDatabase.sqlproj : warning NU1503: Skipping restore for project '/home/vsts/work/1/s/MySolution/MyDatabase/MyDatabase.sqlproj'. The project file may be invalid or missing targets required for restore. [/home/vsts/work/1/s/MySolution/MySolution.sln]
/home/vsts/work/1/s/MySolution/MyDatabase/MyDatabase.sqlproj(57,3): error MSB4019: The imported project "/usr/share/dotnet/sdk/2.1.403/Microsoft/VisualStudio/v15.0/SSDT/Microsoft.Data.Tools.Schema.SqlTasks.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
1 Warning(s)
1 Error(s)
How do I reference or include those targets for the build server? It builds fine in VS2017. I've spent more than a day hunting and cannot find any information on this problem.

Thanks to Herman Cordes for directing my investigation.
The problem was the selected build server. SSDT is a Windows-only package, so I had to use a Windows VM instead of the default Ubuntu VM, and use the VSBuild#1 task instead of DotNetCoreCLI#2.
azure-pipelines.yml
pool:
vmImage: 'vs2017-win2016'
steps:
- task: VSBuild#1
displayName: 'vsbuild $(buildConfiguration)'
inputs:
configuration: $(buildConfiguration)
EDIT: The MSBuild#1 task also works.

Here is a solution for linux build agent
Easiest way to build DacPac file on a linux agent is done via MSBuild.Sdk.SqlProj
Go to your database project directory in parallel to .sqlproj file create a directory like DB.Build under it create DB.Build.csproj copy.pase the content as below
<Project Sdk="MSBuild.Sdk.SqlProj/1.1.0"> <!-- This will pull in the required tools and dependencies to build a .dacpac with .NET Core -->
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Content Include="..\src\DB\masterdata\**\*.sql" /> <!-- link in the new .csproj to the .sql scripts in your existing database project -->
</ItemGroup>
</Project>
After run you will see dacpac file appears under DB.Build/bin/Release/netstandard2.0/DB.Build.dacpac
Here's my build agent output (Ubuntu agent on Azure devops)
Starting: SQL DB build Release
==============================================================================
Task : .NET Core
Description : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version : 2.187.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
/opt/hostedtoolcache/dotnet/dotnet build /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/RecommenderAPI.DB.Build.csproj -dl:CentralLogger,"/home/vsts/work/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.187.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/home/vsts/work/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.187.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --configuration Release /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=/home/vsts/work/1/recommender-service-cicd/DacPac/
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
Restore completed in 51.72 ms for /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/RecommenderAPI.DB.Build.csproj.
Using package name RecommenderAPI.DB.Build and version 1.0.0
Using SQL Server version Sql150
Deleting existing file /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/obj/Release/netstandard2.0/RecommenderAPI.DB.Build.dacpac
Writing model to /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/obj/Release/netstandard2.0/RecommenderAPI.DB.Build.dacpac
RecommenderAPI.DB.Build -> /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/bin/Release/netstandard2.0/RecommenderAPI.DB.Build.dacpac
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:01.71
Finishing: SQL DB build Release
Note: Make sure to restore you NuGet packages in step prior to build
Your ADO pipeline manifest should look like this:
...
- task: DotNetCoreCLI#2
displayName: 'Restore SQL Project'
inputs:
command: 'restore'
projects: '**/*DB*/*.csproj'
feedsToUse: 'select'
vstsFeed: 'db-feed'
...
- task: DotNetCoreCLI#2
displayName: 'SQL DB build $(buildConfiguration)'
inputs:
command: build
projects: '**/*DB*/*.csproj'
platform: '$(buildPlatform)'
arguments: '--configuration $(buildConfiguration) /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=$(Pipeline.Workspace)/$(pipelineArtifactName)/DacPac/'

I was having the exact same issue building a SQL Server project on an Azure DevOps CI/CD pipeline. None of the pre-built build tasks would work for me!
I solved this by avoiding to add a SQL Server project to the solution.
I achieved this by using an MSBuild SDK, capable of producing a SQL Server Data-Tier Application package (.dacpac) from the set of SQL scripts. By adding this second project to the solution, I managed to continue taking advantage of linking the project to a live database through SQL Server Object Explorer on Visual Studio. I gave a more detailed explanation about my implementation in this answer.

Related

Project.UnitTests.dll not found when trying to run unit tests in Azure Devops

The error I am getting :
The test source file "D:\a\_work\3\s\MyProject.UnitTests\bin\Release\net6.0\MyProject.UnitTests.dll" provided was not found.
##[error]Error: The process 'C:\hostedtoolcache\windows\dotnet\dotnet.exe' failed with exit code 1
##[warning]No test result files were found.
##[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
Related yml script:
steps:
- task: DotNetCoreCLI#2
displayName: "Run unit tests"
inputs:
command: test
projects: "**/*MyProject.UnitTests/*.csproj"
arguments: --configuration $(projectBuildConfiguration) --no-build --no-restore
feedsToUse: 'config'
nugetConfigPath: 'nuget.config'
The tests are running all fine in my local machine.
Project build configuration is release
Since you don't have a separate build step you should not be passing the --no-build --no-restore flags.

The nuget command failed with exit code(1) The element <ProductVersion> beneath element <Project> is unrecognized

I'm setting a build in Azure DevOps Pipelines, I have in repos some solutions including a SQL Server Analysis Project. This project triggers a error in the build for the NuGetCommand:
[error]The nuget command failed with exit code(1) ...: error MSB4067: The element <ProductVersion> beneath element <Project> is unrecognized.)
[error]Packages failed to restore
[section]Finishing: NuGetCommand
Any ideas how to solve this error? Maybe run the NuGetCommand for all projects except this SSAS, how can I exclude this solution/project?
I tried to Manage NuGet Packages for this SQL Server Analysis Project, but is not supported.
I have this code in the azure pipeline, I need to run this command for every solutions except SQL Server Analysis Project.
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
The nuget command failed with exit code(1) The element beneath element is unrecognized
This issue is more related to the MSBuild instead of nuget.
Since NuGet is now fully integrated into MSBuild, when we invoke nuget to restore packages, it will call MSBuild auto-detection. There is a issue in MSBuild 15 about it.
Besides,
the latest version of Microsoft Reporting Services Projects for Visual
Studio (1.18) adds MSBuild support for SSRS projects. With this
installed, SSRS projects can be updated to a format supported by
MSBuild, which prevents this problem from happening.
So, to resolve this issue, please update your Visual Studio on the agent server to the latest version to check if you still have this issue.
Hope this helps.

SSDT Installation for VS2017 Team Explorer Fails

I am attempting to install SSDT v15.6.0 into an existing Visual Studio Team Explorer 2017 instance. When I run the SSDT installer, I can select Visual Studio Team Explorer 2017 from this list of existing Visual Studio instances. If I do not choose a any options (other than the required SQL Server Database option). The installation completes successfully.
If I however attempt to pick any/all of the other options the installation fails. From looking at the log files, it appears that the installation of the VSIX is failing due to a required component missing.
Below is the snippet of the SSDT-Setup installation log file which indicates that the VSIX failed to install.
[167C:1A6C][2018-05-16T15:40:41]i301: Applying execute package: Microsoft.DataTools.IntegrationServices, action: Install, path: C:\ProgramData\Package Cache\9864EE5369359DB622D0234B6C6B7640B949140C\VSIXBootstrapper.exe, arguments: '"C:\ProgramData\Package Cache\9864EE5369359DB622D0234B6C6B7640B949140C\VSIXBootstrapper.exe" /q /admin /instanceIds:"68f392fc" /logFile:"C:\Users\golaat\AppData\Local\Temp\SsdtSetup\SSDT-Setup-ENU_20180516153141_009_Microsoft.DataTools.IntegrationServices.log" "payload\Microsoft.DataTools.IntegrationServices.vsix"'
[167C:1A6C][2018-05-16T15:41:13]e000: Error 0x80131500: Process returned error: 0x80131500
[167C:1A6C][2018-05-16T15:41:13]e000: Error 0x80131500: Failed to execute EXE package.
[1D80:1858][2018-05-16T15:41:13]e000: Error 0x80131500: Failed to configure per-machine EXE package.
Looking at the log file for the installation of the VSIX it seems like a required component is missing:
5/16/2018 3:41:05 PM - BEGIN: Targetting Visual Studio Team Explorer 2017,15.0.27428.2043
5/16/2018 3:41:05 PM - BEGIN: Execute batch install
5/16/2018 3:41:08 PM - BEGIN: Preparing extension Microsoft Integration Services Projects
5/16/2018 3:41:08 PM - Loading extension: Microsoft Integration Services Projects
5/16/2018 3:41:10 PM - JSON file loaded
5/16/2018 3:41:10 PM - The dependent package of 'Component.D1B09713-C12E-43CC-9EF4-6562298285AB,version=1.4' cannot be found: Microsoft.VisualStudio.Component.SQL.SSDT,version=[15.0,16.0).
5/16/2018 3:41:10 PM - END: Execute batch install
5/16/2018 3:41:10 PM - END: Installing
5/16/2018 3:41:10 PM - Common folder deleted: C:\Users\golaat\AppData\Local\Temp\s0ggmkmo.cwz
5/16/2018 3:41:10 PM - Microsoft.VisualStudio.Setup.Dependencies.DependencyGraphConstructionException: The dependent package of 'Component.D1B09713-C12E-43CC-9EF4-6562298285AB,version=1.4' cannot be found: Microsoft.VisualStudio.Component.SQL.SSDT,version=[15.0,16.0).
Is there a way to identify this missing component an perhaps install it manually?
This is how I fixed it :
Run installer normally to the end and let it fail.
When it fails the window will have a link to the log file, open it.
In this log file, 1 or 2 lines before the error(Process returned error: 0x80131500) problem component location will be revealed, something like this:
[6748:674C][2020-07-29T09:44:03]i301: Applying execute package:
ISVsix, action: Install, path: C:\ProgramData\Package
Cache\15160B731819F56D87A626F9A2777550340022D7\VSIXBootstrapper.exe,
arguments: '"C:\ProgramData\Package
Cache\15160B731819F56D87A626F9A2777550340022D7\VSIXBootstrapper.exe"
/q /admin /instanceIds:"5403e431"
/logFile:"C:\Users\xxxx\AppData\Local\Temp\SsdtisSetup\Microsoft.DataTools.IntegrationServices_20200729093525_11_ISVsix.log"
"payload\Microsoft.DataTools.IntegrationServices.vsix"
You have to combine location of the VSIXBootstrapper with vsix component location. In my case I had vsix file under C:\ProgramData\Package Cache\15160B731819F56D87A626F9A2777550340022D7\payload
Copy failing component(.vsix), in this case Microsoft.DataTools.IntegrationServices.vsix to some folder before it gets deleted. If it already got deleted then restart the installer and search for this component in parent folders.
Restart your computer. I was getting "Another installation running .." error before I did.
Go to the folder where you copied the failing component and install the .vsix manually. This time it should install normally.
Repeat step 1 but this time hoping it will work normally.
I hope this at least gets you in the right direction.
This is currently broken due to a bug in Visual Studio - see this post on the Developer Community site:
SSDT cannot install onto 15.7 version of Visual Studio
It'll be fixed in a forthcoming Visual Studio update, but if you can't wait for that a workaround is to completely uninstall Visual Studio and install an older version. See my answer here for instructions:
https://stackoverflow.com/a/50349464/1184850
With the release of SSDT 15.7.1, Team Explorer no longer shows up as an existing Visual Studio install instance option within SSDT. At first I was concerned that the SSDT install would install a Visual Studio shell with no TFS connectivity, however this was not the case. It appears that the SSDT install of the Visual Studio shell picks up on the fact that Visual Studio Team Explorer 2017 is already installed on the machine and the SSDT Visual Studio shell instance has full TFS Connectivity!
I found a solution, follow these steps:
Download SQL Server and install it. In the Feature Selection step install Integration Service
Now download Visual Studio installer and install .NET desktop development and Data storage and processing
Now run Visual Studio and go to Extensions > Manage Extensions > Online, and search for SQL Server Integration Services Projects. Download and install that
You are now good to go.

How to create web deploy package in VSTS/VSO Build vNext?

I have an automated build in Visual Studio Team Services (formerly Visual Studio Online. The solution is building correctly. How do I configure it so it will also create a web deploy package?
We use the following MSBuild Arguments: /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"
With these you do not need to create a new publishing profile.
We enable web deploy package creation within the "Visual Studio Build" task. Simply pass the following MS Build arguments:
MSBuild Arguments: /p:DeployOnBuild=true;PublishProfile=<name of your profile>

Team Foundation Server Deployment error

I have setup my own Team Foundation Build server and I am using Visual Studio Online to host my project online. I have a django project which I am trying to deploy on windows azure cloud service. The project can be published from my dev machine without any errors however when the build server tries to build it, it gives the following error during uploading the package:
Exception Message: Could not load file or assembly 'Microsoft.WindowsAzure.Storage, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. (type FileNotFoundException)
I checked in .pyproj and found a reference to Microsoft.WindowsAzure.Storage but there the version is 2.1.0.0 and that is the version that is installed on the server as well.
Also there are no errors during the build process but only during the deployment stage. How do I make Msbuild use Storage version 2.1.0.0 instead of 2.0.0.0 . Also, which SDK version contains Storage version 2.0.0.0
I just had this problem. This is the same problem as TFS Build Controller : "Could not load file or assembly 'Microsoft.WindowsAzure.StorageClient, Version=1.7.0.0".
You'll need to copy Microsoft.WindowsAzure.Storage.dll version 2.0.0.0 to "C:\Program Files\Microsoft Team Foundation Server 12.0\Tools", and then restart the Visual Studio Team Foundation Build Service Host 2013 service.