I am building a pipeline using a template (and nextly will be more pipelines using the same template). My problem is when I change something in the template the pipeline is executed, that means when there be more pipelines using the template, all will be executed.
Is there anyway to indicate that when template changes I don't need to execute pipeline automatically?
this is my template:
parameters:
- name: repoName # name of the parameter; required
type: string # data type of the parameter; required
default: ''
jobs:
- job: BuildImage
displayName: Build
pool:
name: Azure Pipelines
demands: java
vmImage: ubuntu-20.04
steps:
- checkout: ${{ parameters.repoName }}
clean: true
fetchDepth: 1
- task: Gradle#2
displayName: 'gradlew clean'
inputs:
workingDirectory: ./
tasks: clean
publishJUnitResults: false
jdkVersionOption: 1.11
- task: Gradle#2
displayName: 'gradlew build'
inputs:
options: '-x test'
publishJUnitResults: false
jdkVersionOption: 1.11
##Fix Jacoco
- task: Gradle#2
displayName: 'gradlew test'
inputs:
tasks: test
publishJUnitResults: false
codeCoverageToolOption: JaCoCo
jdkVersionOption: 1.11
## Add Sonar
and here the template is using it:
resources:
repositories:
- repository: repository
type: git
name: CORE_tpaas-utilities
ref: 'refs/heads/development'
extends:
template: ../templates/ci.yml
parameters:
repoName: repository
You can use the path filter for your pipeline trigger. For example add below to your pipeline yaml file.
resources:
....
trigger:
paths:
exclude:
- path/to/template.yml
extends:
....
After you specify above path filter to exclude the template yaml file for the pipeline trigger. The pipeline will not be executed when changes are made to the template file.
See here for more information.
Related
Is it possible to add some conditions to this way of using templates?
I would like to use conditions based on branches.
If branch = main or development > deploy to dev
Only if branch > main, deploy to staging
The complete Pipeline:
trigger:
batch: true
branches:
include:
- main
pool: 'On-Prem Pool'
variables:
- group: ContainerRegistry
- group:xyz
- name: appName
value: 'xyz'
- name: dockerfilePath
value: '**/Dockerfile'
stages:
# Build App and push to container registry
- stage: Build
displayName: Build and push to container registry
jobs:
- job: Build
displayName: Build job
pool: 'On-Prem Pool'
steps:
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry: $(registryServiceConnection)
- task: Docker#2
displayName: Build container image
inputs:
command: build
repository: $(appName)
dockerfile: $(dockerfilePath)
containerRegistry: $(registryServiceConnection)
tags: $(Build.BuildId)
- task: Docker#2
displayName: Push container image
inputs:
command: push
repository: $(appName)
containerRegistry: $(registryServiceConnection)
condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest')) # DonĀ“t push to ACR on Pull Requests
# Infrastructure
# Lint the Bicep file.
- stage: Lint
jobs:
- template: pipeline-templates/lint.yml
# Deploy to the dev environment.
- template: pipeline-templates/deploy.yml
parameters:
environmentType: Development
resourceGroupName: rg-xx
appSuffix: xyz
costCenter: 'xyz'
serviceConnectionName: 'SVCxx'
dockerRegistryPassword: $(dockerRegistryPassword)
dockerRegistryUserName: $(dockerRegistryUserName)
containerTag: $(Build.BuildId)
registryUrl: $(registryUrl)
cpuCores: 0.5
memory: 1
minReplicas: 0
maxReplicas: 1
# Deploy to the staging environment.
- template: pipeline-templates/deploy.yml
parameters:
environmentType: Staging
resourceGroupName: rg-xxx-001
appSuffix: xyz
costCenter: 'xyz'
serviceConnectionName: 'SVCxxx'
dockerRegistryPassword: $(dockerRegistryPassword)
dockerRegistryUserName: $(dockerRegistryUserName)
containerTag: $(Build.BuildId)
registryUrl: $(registryUrl)
cpuCores: 0.5
memory: 1
minReplicas: 1
maxReplicas: 2
)
And it is the last two -templates I would like to apply some conditions on.
Many thanks!
During my test, I edited my build.yaml as below. And when I ran it, the second template could evaluate correctly but the third one not, and thus, the third template was skipped.
pool:
vmImage: windows-latest
stages:
- template: template.yml
parameters:
env: dev
- ${{ if eq(variables['Build.QueuedBy'], 'Ceeno') }}:
- template: template.yml
parameters:
env: qa
- ${{ if eq(variables['Build.QueuedBy'], 'Vacee') }}:
- template: template.yml
parameters:
env: prod
And if you are looking for the evaluation for other method or other scenario, you could share with me for further investigations.
Yes I solved it in a similar way;
- stage: BuildDeployProd
displayName: Build and deploy Prod env
jobs:
- template: pipeline-templates/deploy.yml
parameters:
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
I have a shared template that I am referencing in my azure pipeline to Build, Test, Pack, and Publish a C# nuget package.
DotNet-BuildTestPackPublish.yml
parameters:
- name: project
type: string
default: ''
- name: dotnetVersion
type: string
default: '5.0.x'
- name: isNugetPackage
type: boolean
default: false
- name: isFirstProjectInRepo
type: boolean
default: true
steps:
- task: UseDotNet#2
displayName: 'Add .NET ${{parameters.dotnetVersion}}'
condition: and(succeeded(), eq(${{parameters.isFirstProjectInRepo}}, true))
inputs:
packageType: 'sdk'
version: ${{parameters.dotnetVersion}}
- task: NuGetAuthenticate#0
displayName: 'NuGet Auth'
condition: and(succeeded(), eq(${{parameters.isFirstProjectInRepo}}, true))
- checkout: self
persistCredentials: true
displayName: "Git Checkout"
condition: and(succeeded(), eq(${{parameters.isFirstProjectInRepo}}, true))
- task: DotNetCoreCLI#2
displayName: "Build"
inputs:
command: 'build'
projects: '**/*.sln'
- task: DotNetCoreCLI#2
displayName: Unit Test
inputs:
projects: '**/*.sln'
command: 'test'
- task: PowerShell#2
displayName: "Extract Version"
condition: and(succeeded(), eq(${{parameters.isNugetPackage}}, true), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
inputs:
targetType: 'inline'
script: |
Write-Host "Locating Project Path..."
$input_path = (Get-ChildItem -Include ${{parameters.project}}.csproj -File -Recurse)
Write-Host "Project Path = $input_path"
Write-Host "Getting version from csproj..."
$regex = '(?<=<Version>).*?(?=<\/Version>)'
$version = Select-String -Path $input_path -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value }
Write-Host "Version = $version"
if([string]::IsNullOrWhiteSpace($version))
{
Write-Host "Version not in csproj. Defaulting to 1.0.0"
$version = "1.0.0"
}
Write-Host "Getting Build Number from Environment"
$buildNumber = '$(Build.BuildId)'
Write-Host "Build Number = $buildNumber"
$versionAndBuildNumber = "$version+$buildNumber"
Write-Host "Package Version = $versionAndBuildNumber"
$tagText = "${{parameters.project}}+$version+$buildNumber"
Write-Host "Tag Text = $tagText"
Write-Output "##vso[task.setvariable variable=gitTag]$tagText"
Write-Output "##vso[task.setvariable variable=versionNumber]$versionAndBuildNumber"
- task: PublishSymbols#2
inputs:
searchPattern: '**/bin/**/${{parameters.project}}.pdb'
symbolServerType: "teamServices"
- task: DotNetCoreCLI#2
displayName: "Nuget Pack"
condition: and(succeeded(), eq(${{parameters.isNugetPackage}}, true), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
inputs:
command: 'pack'
packagesToPack: '**/${{parameters.project}}.csproj'
versioningScheme: byEnvVar
versionEnvVar: versionNumber
includesymbols: true
includesource: true
- task: NuGetCommand#2
displayName: "Nuget Push"
condition: and(succeeded(), eq(${{parameters.isNugetPackage}}, true), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
inputs:
command: 'push'
packagesToPush: '$(Build.ArtifactStagingDirectory)/**/*.nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: 'Internal'
allowPackageConflicts: true
- script: |
echo Current Directory:
cd
dir /s
git tag $(gitTag)
git push --tags origin $(gitTag)
workingDirectory: $(Build.SourcesDirectory)
displayName: "Git Tag Repository"
condition: and(succeeded(), eq(${{parameters.isNugetPackage}}, true), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
This works well if there is only one project in my repository:
azure-pipelines.yml (only one project)
trigger:
branches:
include:
- '*'
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: BuildScripts
name: Feldryn/BuildScripts
type: git
ref: main
trigger:
- main
variables:
- group: 'Pipeline Shared Variables'
steps:
- template: DotNet-BuildTestPackPublish.yml#BuildScripts
parameters:
project: 'Daemon.Domain.RumbleRunners.Values'
isNugetPackage: true
# the following step is superfluous for this example, but it DOES work
- task: PowerShell#2
displayName: "Replace Project Reference with Nuget"
inputs:
targetType: 'inline'
script: |
cd Daemon.Domain.RumbleRunners
dotnet remove reference ..\Daemon.Domain.RumbleRunners.Values\Daemon.Domain.RumbleRunners.Values.csproj
dotnet add package Daemon.Domain.RumbleRunners.Values
cd..
As soon as I attempt to call the template again in the same pipeline for another project, the first will start failing.
This does not work:
azure-pipelines.yml (two projects)
trigger:
branches:
include:
- '*'
pool:
vmImage: 'windows-latest'
resources:
repositories:
- repository: BuildScripts
name: Feldryn/BuildScripts
type: git
ref: main
trigger:
- main
variables:
- group: 'Pipeline Shared Variables'
steps:
- template: DotNet-BuildTestPackPublish.yml#BuildScripts
parameters:
project: 'Daemon.Domain.RumbleRunners.Values'
isNugetPackage: true
- task: PowerShell#2
displayName: "Replace Project Reference with Nuget"
inputs:
targetType: 'inline'
script: |
cd Daemon.Domain.RumbleRunners
dotnet remove reference ..\Daemon.Domain.RumbleRunners.Values\Daemon.Domain.RumbleRunners.Values.csproj
dotnet add package Daemon.Domain.RumbleRunners.Values
cd..
# the inclusion of following step is the only change
- template: DotNet-BuildTestPackPublish.yml#BuildScripts
parameters:
project: 'Daemon.Domain.RumbleRunners'
isNugetPackage: true
isFirstProjectInRepo: false
It fails with 'fatal: not a git repository (or any of the parent directories): .git' when attempting to git tag the first project.
I've done some troubleshooting and determined:
It's not my git tag step that is at fault.
It works when there is one project
The failure moves to the following step if I comment the git tag step.
New error: Could not find any project in D:\a\1\s\Daemon.Domain.RumbleRunners\.
I've outputted the contents of my working directory and can see all expected files are present, including the .git folder
The working directory is not different between the working and non-working approaches.
I'm at a loss for how to troubleshoot further.
Why can't I call my template twice? What is happening here?
checkout task behaves differently when there are multiple checkouts:
When there's only a single checkout: self step, it will check out the repository to /home/vsts/work/1/s/.
When there are multiple checkout steps (even if they refer to the same repository), it will check out to /home/vsts/work/1/s/{repository_name}.
Even though you have a condition on the second checkout, the step still counts, thus point 2. applies.
To validate this, check logs of checkout task for both scenarios. You can also try a minimal repro pipeline:
steps:
- checkout: self # <-- /home/vsts/work/1/s/
vs
steps:
- checkout: self # <-- /home/vsts/work/1/s/{repo_name}/
- checkout: self # <-- /home/vsts/work/1/s/{repo_name}/
condition: false
I want to include a template, but it won't work. I know this is somehow a duplicate question, but the other answers don't solve my problem.
Template-File (base.yml)
trigger:
- master
pool: 'Etiscan Windows'
variables:
solution: '**/*.sln'
buildPlatform: 'x86'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/*.sln'
feedsToUse: 'select'
vstsFeed: '76b8ae4e-2263-4849-bfb3-e1f621bf5bd7'
...to be continued....
Azure-Pipelines.yml
resources:
repositories:
- repository: templates
type: git
name: YML-Templates/NugetCfBuilderTemplate
stages:
- stage: Job1
- template: base.yml#templates
Errors
/base.yml#templates (Line: 6, Col: 1): Unexpected value 'trigger'
/base.yml#templates (Line: 9, Col: 1): Unexpected value 'pool'
/base.yml#templates (Line: 11, Col: 1): Unexpected value 'variables'
/base.yml#templates (Line: 16, Col: 1): Unexpected value 'steps'
In the resource, we can also add the trigger, refer doc. Help you modified the yaml, hope this will help:
resources:
repositories:
- repository: templates
type: git
name: YML-Templates/NugetCfBuilderTemplate
trigger:
branches:
include:
- master
stages:
- stage: Job1
- template: base.yml#templates
Note: please remove the trigger in your base.yml file.
Update:
I help you modified your base yaml and here I use the Powershell task to do the test:
stages:
- stage: job1
pool: 'default'
variables:
solution: '**/*.sln'
buildPlatform: 'x86'
buildConfiguration: 'Release'
jobs:
- job: A1
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello World"
# - task: NuGetToolInstaller#1
# - task: NuGetCommand#2
# inputs:
# command: 'restore'
# restoreSolution: '**/*.sln'
# feedsToUse: 'select'
# vstsFeed: '76b8ae4e-2263-4849-bfb3-e1f621bf5bd7'
Note: Please remove the ' - stage: Job1' from your main yaml file.
Trigger can be defined only at the pipeline level, you cannot set it at the stage level. The stage can only have these children (see YAML schema reference):
stages:
- stage: string # name of the stage (A-Z, a-z, 0-9, and underscore)
displayName: string # friendly name to display in the UI
dependsOn: string | [ string ]
condition: string
variables: # several syntaxes, see specific section
jobs: [ job | templateReference]
You also need to insert a Job containing the steps and move the pool stanza down to Job level.
In my main pipeline in one stage, I call the same (deployment) template twice with just a bit different data:
//pipeline.yml
- stage: dev
condition: and(succeeded(), eq('${{ parameters.environment }}', 'dev'))
variables:
getCommitDate: $[ stageDependencies.prepare_date.set_date.outputs['setCommitDate.rollbackDate'] ]
jobs:
- template: mssql/jobs/liquibase.yml#templates
parameters:
command: update
username: $(username_dev)
password: $(password_dev)
environment: exampleEnv
databaseName: exampleDB
databaseIP: 123456789
context: dev
checkoutStep:
bash: git checkout ${{parameters.commitHash}} -- ./src/main/resources/objects
- template: mssql/jobs/liquibase.yml#templates
parameters:
command: rollbackToDate $(getCommitDate)
username: $(username_dev)
password: $(password_dev)
environment: exampleEnv
databaseName: exampleDB
databaseIP: 123456789
context: dev
//template.yml
parameters:
- name: command
type: string
- name: environment
type: string
- name: username
type: string
- name: password
type: string
- name: databaseName
type: string
- name: databaseIP
type: string
- name: context
type: string
- name: checkoutStep
type: step
default:
checkout: self
jobs:
- deployment: !MY PROBLEM!
pool:
name: exampleName
demands:
- agent.name -equals example
environment: ${{ parameters.environment }}
container: exampleContainer
strategy:
runOnce:
deploy:
steps:
...
My problem is that the deployment cannot have the same name twice.
It is not possible to use the ${{parameters.command}} to distinguish between deployments names, because it contains forbidden characters. Only ${{parameters.command}} differs between two calls.
My question is whether it is possible to distinguish the name of a deployment other way than passing another parameter (e.g. jobName: ). I have tried various conditions and predefined variables but without success.
Additionally, I should add DependsOn so that the second template is called for sure after the first.
It is not possible because getCommitDate and thus command parameter in your second stage contains runtime expression and job name needs compile time expression. So if you use command as job name at compile you have there rollbackToDate $(getCommitDate).
To solve this issue, the job identifier should be empty in a template:
- job: # Empty identifier
More informations available HERE
I am quite new to azure devops. I am trying to deploy my app from azure to aws ec2 using codedeploy. The build pipeline is giving "Error: Deployment failed Dev-TitasEcom-Frontend Resource is not in the state deploymentSuccessful" error. I have searched a lot but could not find any proper resource to resolve the issue. Below given the code of azure-pipelines.yaml
pr:
branches:
include:
- master
- dev
paths:
include:
- ./*
jobs:
- job: Build_Job
displayName: Build
pool:
vmImage: 'ubuntu-latest'
demands:
- npm
steps:
- checkout: self
clean: false
# - powershell: 'npm cache clean --force'
# displayName: 'PowerShell Script'
# env:
# APPDATA: npm-cache
# - task: CodeDeployDeployApplication#1
# inputs:
# awsCredentials: 'AWS Service Con'
# regionName: 'us-east-1'
# applicationName: 'Dev-TitasEcom-Frontend'
# deploymentGroupName: 'Dev-erp-Frontend'
# #deploymentRevisionSource: 'workspace'
#revisionBundle: 'Dev-erp-Frontend-Rev'
# bucketName: 'Dev-erp-Frontend'
# fileExistsBehavior: 'OVERWRITE'
# batch: "true"
- task: CodeDeployDeployApplication#1
inputs:
awsCredentials: 'AWS Service Con 1'
regionName: 'us-east-1'
applicationName: 'Dev-TitasEcom-Frontend-11'
deploymentGroupName: 'Dev-erp-Frontend'
environmentName: 'DevTitasecomFrontend-env-12'
applicationType: 'version'
bucketName: 'elasticbeanstalk-us-east-1-3067722226831765'
fileExistsBehavior: 'OVERWRITE'
#versionLabel: 'v.0.01'
batch: "true"
- task: Npm#1
displayName: 'Npm Install'
inputs:
workingDir: "./"
command: "ci"
- task: Npm#1
displayName: 'Lint Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run lint"
continueOnError: true
- task: Npm#1
displayName: 'Copy Assets'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run copy-files"
continueOnError: false
- task: Npm#1
displayName: 'Build Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run build:prod"
# Archive files
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip' # Options: zip, 7z, tar, wim
archiveFile: '$(Build.ArtifactStagingDirectory)/www.zip'
replaceExistingArchive: true
- task: CopyFiles#2
inputs:
Contents: 'www/**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: 'titas-ecom-erp'