Is it possible to add some conditions to this way of using templates?
I would like to use conditions based on branches.
If branch = main or development > deploy to dev
Only if branch > main, deploy to staging
The complete Pipeline:
trigger:
batch: true
branches:
include:
- main
pool: 'On-Prem Pool'
variables:
- group: ContainerRegistry
- group:xyz
- name: appName
value: 'xyz'
- name: dockerfilePath
value: '**/Dockerfile'
stages:
# Build App and push to container registry
- stage: Build
displayName: Build and push to container registry
jobs:
- job: Build
displayName: Build job
pool: 'On-Prem Pool'
steps:
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry: $(registryServiceConnection)
- task: Docker#2
displayName: Build container image
inputs:
command: build
repository: $(appName)
dockerfile: $(dockerfilePath)
containerRegistry: $(registryServiceConnection)
tags: $(Build.BuildId)
- task: Docker#2
displayName: Push container image
inputs:
command: push
repository: $(appName)
containerRegistry: $(registryServiceConnection)
condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest')) # Don´t push to ACR on Pull Requests
# Infrastructure
# Lint the Bicep file.
- stage: Lint
jobs:
- template: pipeline-templates/lint.yml
# Deploy to the dev environment.
- template: pipeline-templates/deploy.yml
parameters:
environmentType: Development
resourceGroupName: rg-xx
appSuffix: xyz
costCenter: 'xyz'
serviceConnectionName: 'SVCxx'
dockerRegistryPassword: $(dockerRegistryPassword)
dockerRegistryUserName: $(dockerRegistryUserName)
containerTag: $(Build.BuildId)
registryUrl: $(registryUrl)
cpuCores: 0.5
memory: 1
minReplicas: 0
maxReplicas: 1
# Deploy to the staging environment.
- template: pipeline-templates/deploy.yml
parameters:
environmentType: Staging
resourceGroupName: rg-xxx-001
appSuffix: xyz
costCenter: 'xyz'
serviceConnectionName: 'SVCxxx'
dockerRegistryPassword: $(dockerRegistryPassword)
dockerRegistryUserName: $(dockerRegistryUserName)
containerTag: $(Build.BuildId)
registryUrl: $(registryUrl)
cpuCores: 0.5
memory: 1
minReplicas: 1
maxReplicas: 2
)
And it is the last two -templates I would like to apply some conditions on.
Many thanks!
During my test, I edited my build.yaml as below. And when I ran it, the second template could evaluate correctly but the third one not, and thus, the third template was skipped.
pool:
vmImage: windows-latest
stages:
- template: template.yml
parameters:
env: dev
- ${{ if eq(variables['Build.QueuedBy'], 'Ceeno') }}:
- template: template.yml
parameters:
env: qa
- ${{ if eq(variables['Build.QueuedBy'], 'Vacee') }}:
- template: template.yml
parameters:
env: prod
And if you are looking for the evaluation for other method or other scenario, you could share with me for further investigations.
Yes I solved it in a similar way;
- stage: BuildDeployProd
displayName: Build and deploy Prod env
jobs:
- template: pipeline-templates/deploy.yml
parameters:
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
Related
I want to include a template, but it won't work. I know this is somehow a duplicate question, but the other answers don't solve my problem.
Template-File (base.yml)
trigger:
- master
pool: 'Etiscan Windows'
variables:
solution: '**/*.sln'
buildPlatform: 'x86'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/*.sln'
feedsToUse: 'select'
vstsFeed: '76b8ae4e-2263-4849-bfb3-e1f621bf5bd7'
...to be continued....
Azure-Pipelines.yml
resources:
repositories:
- repository: templates
type: git
name: YML-Templates/NugetCfBuilderTemplate
stages:
- stage: Job1
- template: base.yml#templates
Errors
/base.yml#templates (Line: 6, Col: 1): Unexpected value 'trigger'
/base.yml#templates (Line: 9, Col: 1): Unexpected value 'pool'
/base.yml#templates (Line: 11, Col: 1): Unexpected value 'variables'
/base.yml#templates (Line: 16, Col: 1): Unexpected value 'steps'
In the resource, we can also add the trigger, refer doc. Help you modified the yaml, hope this will help:
resources:
repositories:
- repository: templates
type: git
name: YML-Templates/NugetCfBuilderTemplate
trigger:
branches:
include:
- master
stages:
- stage: Job1
- template: base.yml#templates
Note: please remove the trigger in your base.yml file.
Update:
I help you modified your base yaml and here I use the Powershell task to do the test:
stages:
- stage: job1
pool: 'default'
variables:
solution: '**/*.sln'
buildPlatform: 'x86'
buildConfiguration: 'Release'
jobs:
- job: A1
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello World"
# - task: NuGetToolInstaller#1
# - task: NuGetCommand#2
# inputs:
# command: 'restore'
# restoreSolution: '**/*.sln'
# feedsToUse: 'select'
# vstsFeed: '76b8ae4e-2263-4849-bfb3-e1f621bf5bd7'
Note: Please remove the ' - stage: Job1' from your main yaml file.
Trigger can be defined only at the pipeline level, you cannot set it at the stage level. The stage can only have these children (see YAML schema reference):
stages:
- stage: string # name of the stage (A-Z, a-z, 0-9, and underscore)
displayName: string # friendly name to display in the UI
dependsOn: string | [ string ]
condition: string
variables: # several syntaxes, see specific section
jobs: [ job | templateReference]
You also need to insert a Job containing the steps and move the pool stanza down to Job level.
I am using Azure devops yml pipeline at code base.
I have created Variable Group at pipeline (pipeline > Library > variable group > called 'MY_VG')
In my pipeline (yml) file i want to send this variable group MY_VG to template my_template.yml as parameter.
But this parameter MY_VG is not being expanded when i use it under 'Variable' (though while printing it gives me value)
How to access the value of this MY_VG in the template here group: ${{parameters.variable_group}} shown below?
(I am calling a template file my_template_iterator.yml which iterates the environments and call the my_template.yml)
azure-pipelines.yml
resources:
repositories:
- repository: templates
type: git
name: MY_PROJECT/GIT_REPO_FOR_TEMPLATE
stages:
- stage: "CheckOut"
displayName: Checkout
jobs:
- job: Checkout
displayName: Checkout Application
pool:
name: $(my_pool_name)
workspace:
clean: all
steps:
- checkout: self
- template: folder_name/my_template_iterator.yml#templates
parameters:
agent_pool_name: $(my_pool)
db_resource_path: $(System.DefaultWorkingDirectory)/src/main/resources/db
envs:
- env:
env_name: 'dev'
variable_group: MY_VG
pipeline_environment_name: DEV_ENV
is_master: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
## Iterator: my_template_iterator.yml
parameters:
agent_pool_name: ''
db_resource_path: ''
envs: {}
stages:
- ${{ each env in parameters.envs }}:
- template: my_template.yml
parameters:
agent_pool_name: ${{ parameters.agent_pool_name }}
db_resource_path: ${{ parameters.db_resource_path }}
env_name: ${{ env.env_name }}
variable_group: ${{ env.variable_group }}
pipeline_environment_name: ${{ env.pipeline_environment_name }}
is_master: ${{ env.is_master }}
## my_template.yml
parameters:
- name: 'variable_group'
type: string
default: 'default_variable_group'
stages:
- stage:
displayName: Read Parameters
jobs:
- job: READ
displayName: Reading Parameters
steps:
- script: |
echo variable_group: ${{parameters.variable_group}}
- stage:
displayName: Deployment
variables:
group: ${{parameters.variable_group}}
condition: ${{parameters.is_master}}
I tested with above your yaml files. It seems fine. However i found a small mistake that you missed a - when you define a variable group in my_template.yml of yours. Maybe that is the reason the variable group didnot expand for you.
variables:
# a variable group
- group: myvariablegroup
I modified your my_template.yml file a little bit. And i got it working. See below:
parameters:
- name: 'variable_group'
type: string
default: 'default_variable_group'
- name: agent_pool_name
default: ""
- name: env_name
default: ""
- name: db_resource_path
default: ""
- name: pipeline_environment_name
default: ""
- name: is_master
default: ""
stages:
- stage:
displayName: ${{parameters.env_name}}
## i changed here add '-' to group
variables:
- group: ${{parameters.variable_group}}
jobs:
- job: READ
displayName: Reading Parameters
steps:
- script: |
echo variable_group: ${{parameters.variable_group}}
- powershell: echo "$(variableName-in-variablegroup)"
Ok. I'm not sure if this doable in a way how you are trying. But I tried another approach which seems to be working.
First let's define template for simple stage:
parameters:
- name: env_name
type: string
default: 'dev'
- name: variable_group
type: string
default: 'MY_VG'
- name: pipeline_environment_name
type: string
default: 'DEV_ENV'
stages:
- stage: ${{ parameters.env_name }}
jobs:
- job: angularinstall
steps:
- script: echo "${{ parameters.env_name }}"
- script: echo "${{ parameters.variable_group }}"
- script: echo "${{ parameters.pipeline_environment_name }}"
then template for main build:
parameters:
- name: stages # the name of the parameter is stageList
type: stageList # data type is stageList
default: [] # default value of stageList
stages: ${{ parameters.stages }}
and then we can combine them together in main build file:
extends:
template: template.yaml
parameters:
stages:
- template: stageTemplate.yaml
parameters:
env_name: 'dev'
variable_group: 'MY_VG'
pipeline_environment_name: 'DEV_ENV'
- template: stageTemplate.yaml
parameters:
env_name: 'stage'
variable_group: 'MY_STAGE'
pipeline_environment_name: 'STAGE_ENV'
And for that I got:
Does it sth what bring you closer to your solution?
I am building a pipeline using a template (and nextly will be more pipelines using the same template). My problem is when I change something in the template the pipeline is executed, that means when there be more pipelines using the template, all will be executed.
Is there anyway to indicate that when template changes I don't need to execute pipeline automatically?
this is my template:
parameters:
- name: repoName # name of the parameter; required
type: string # data type of the parameter; required
default: ''
jobs:
- job: BuildImage
displayName: Build
pool:
name: Azure Pipelines
demands: java
vmImage: ubuntu-20.04
steps:
- checkout: ${{ parameters.repoName }}
clean: true
fetchDepth: 1
- task: Gradle#2
displayName: 'gradlew clean'
inputs:
workingDirectory: ./
tasks: clean
publishJUnitResults: false
jdkVersionOption: 1.11
- task: Gradle#2
displayName: 'gradlew build'
inputs:
options: '-x test'
publishJUnitResults: false
jdkVersionOption: 1.11
##Fix Jacoco
- task: Gradle#2
displayName: 'gradlew test'
inputs:
tasks: test
publishJUnitResults: false
codeCoverageToolOption: JaCoCo
jdkVersionOption: 1.11
## Add Sonar
and here the template is using it:
resources:
repositories:
- repository: repository
type: git
name: CORE_tpaas-utilities
ref: 'refs/heads/development'
extends:
template: ../templates/ci.yml
parameters:
repoName: repository
You can use the path filter for your pipeline trigger. For example add below to your pipeline yaml file.
resources:
....
trigger:
paths:
exclude:
- path/to/template.yml
extends:
....
After you specify above path filter to exclude the template yaml file for the pipeline trigger. The pipeline will not be executed when changes are made to the template file.
See here for more information.
I am quite new to azure devops. I am trying to deploy my app from azure to aws ec2 using codedeploy. The build pipeline is giving "Error: Deployment failed Dev-TitasEcom-Frontend Resource is not in the state deploymentSuccessful" error. I have searched a lot but could not find any proper resource to resolve the issue. Below given the code of azure-pipelines.yaml
pr:
branches:
include:
- master
- dev
paths:
include:
- ./*
jobs:
- job: Build_Job
displayName: Build
pool:
vmImage: 'ubuntu-latest'
demands:
- npm
steps:
- checkout: self
clean: false
# - powershell: 'npm cache clean --force'
# displayName: 'PowerShell Script'
# env:
# APPDATA: npm-cache
# - task: CodeDeployDeployApplication#1
# inputs:
# awsCredentials: 'AWS Service Con'
# regionName: 'us-east-1'
# applicationName: 'Dev-TitasEcom-Frontend'
# deploymentGroupName: 'Dev-erp-Frontend'
# #deploymentRevisionSource: 'workspace'
#revisionBundle: 'Dev-erp-Frontend-Rev'
# bucketName: 'Dev-erp-Frontend'
# fileExistsBehavior: 'OVERWRITE'
# batch: "true"
- task: CodeDeployDeployApplication#1
inputs:
awsCredentials: 'AWS Service Con 1'
regionName: 'us-east-1'
applicationName: 'Dev-TitasEcom-Frontend-11'
deploymentGroupName: 'Dev-erp-Frontend'
environmentName: 'DevTitasecomFrontend-env-12'
applicationType: 'version'
bucketName: 'elasticbeanstalk-us-east-1-3067722226831765'
fileExistsBehavior: 'OVERWRITE'
#versionLabel: 'v.0.01'
batch: "true"
- task: Npm#1
displayName: 'Npm Install'
inputs:
workingDir: "./"
command: "ci"
- task: Npm#1
displayName: 'Lint Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run lint"
continueOnError: true
- task: Npm#1
displayName: 'Copy Assets'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run copy-files"
continueOnError: false
- task: Npm#1
displayName: 'Build Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run build:prod"
# Archive files
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip' # Options: zip, 7z, tar, wim
archiveFile: '$(Build.ArtifactStagingDirectory)/www.zip'
replaceExistingArchive: true
- task: CopyFiles#2
inputs:
Contents: 'www/**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: 'titas-ecom-erp'
The Problem
A GitHub trigger set up in Google Cloud Build doesn't actually substitute the configured values while running the Build (cloudbuild.yaml)
This is the Google Cloud Build config
cloudbuild.yaml
steps:
- name: 'gcr.io/cloud-builders/gcloud'
id: 'decrypt .npmrc'
args:
- kms
- decrypt
- --ciphertext-file=npmrc.enc
- --plaintext-file=/root/.npmrc
- --location=global
- --keyring=my-keyring
- --key=npm-key
- --project=${_CLOUD_KMS_PROJECT}
volumes:
- name: 'home'
path: /root/
- name: 'gcr.io/cloud-builders/npm'
id: 'install'
args: ['install']
env:
- HOME=/root/
volumes:
- name: 'home'
path: /root/
- name: 'gcr.io/cloud-builders/npm'
id: 'test'
args: ['run', 'test']
- name: gcr.io/$PROJECT_ID/skaffold:alpha
id: 'deploy'
args: ['run', '-f=${_SKAFFOLD_FILE}']
env:
- CLOUDSDK_COMPUTE_ZONE=${_CLOUDSDK_COMPUTE_ZONE}
- CLOUDSDK_CONTAINER_CLUSTER=${_CLOUDSDK_CONTAINER_CLUSTER}
substitutions:
_SKAFFOLD_FILE: dummy.yaml
_CLOUDSDK_COMPUTE_ZONE: us-west1-a
_CLOUDSDK_CONTAINER_CLUSTER: dummy
timeout: 1000s
Curiously, when triggering the build via a gcloud SDK call it works ✅ ex:
gcloud builds submit --config=cloudbuild.yaml --substitutions=_SKAFFOLD_FILE=skaffold.yaml,_CLOUDSDK_COMPUTE_ZONE=us-west1-a,_CLOUDSDK_CONTAINER_CLUSTER=skaffold .
Some more context
Cloud Build Trigger dashboard (img)
Trigger configuration with substitutions (img)
Missing substitutions in Build history from triggered build (img)
In contrast: Correctly substituted values when running gcloud builds submit (img)