GCP cloudbuild.yaml conditional step error - google-cloud-platform

This is my cloud build file
substitutions:
_CLOUDSDK_COMPUTE_ZONE: us-central1-a
_CLOUDSDK_CONTAINER_CLUSTER: $_CLOUDSDK_CONTAINER_CLUSTER
steps:
- name: gcr.io/$PROJECT_ID/sonar-scanner:latest
entrypoint: 'bash'
args:
- '-c'
- 'if [ $BRANCH_NAME != 'production' ]; then sonar-scanner -Dsonar.host.url=https://sonar.test.io -Dsonar.login=${_SONAR_LOGIN} -Dsonar.projectKey=service -Dsonar.sources=. ; fi'
- id: 'build test-service image'
name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME:$SHORT_SHA', '.']
- id: 'push test-service image'
name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME:$SHORT_SHA']
- id: 'set test-service image in yamls'
name: 'ubuntu'
args: ['bash','-c','sed -i "s,TEST_SERVICE,gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME:$SHORT_SHA," k8s/*.yaml']
- id: kubectl-apply
name: 'gcr.io/cloud-builders/kubectl'
args: ['apply', '-f', 'k8s/']
env:
- 'CLOUDSDK_COMPUTE_ZONE=${_CLOUDSDK_COMPUTE_ZONE}'
- 'CLOUDSDK_CONTAINER_CLUSTER=${_CLOUDSDK_CONTAINER_CLUSTER}'
images: ['gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME:$SHORT_SHA']
I am getting error
failed unmarshalling build config cloudbuild.yaml: yaml: line 17: did
not find expected key
Update 1
As per #cloudomation suggestion updated if condition
- 'if [ $BRANCH_NAME != "production" ]; then sonar-scanner -Dsonar.host.url=https://sonar.test.io -Dsonar.login=${_SONAR_LOGIN} -Dsonar.projectKey=service -Dsonar.sources=. ; fi'
Condition is working but when it's true getting this error
Step #1: Digest: sha256:ef0de1c8e48544b9693b9aab2222bf849028bb66881762bf77e055b0abbf7f2b Step #1: Status: Downloaded newer image for gcr.io/wotnot-235414/sonar-scanner:latest Step #1: gcr.io/project-235414/sonar-scanner:latest Step #1: /opt/sonar-scanner-3.2.0.1227-linux/bin/sonar-scanner: exec: line 59: /opt/sonar-scanner-3.2.0.1227-linux/jre/bin/java: not found Finished Step #1 ERROR ERROR: build step 1 "gcr.io/project-235414/sonar-scanner:latest" failed: exit status 127
but when my step is like this way it's working perfectly
- name: gcr.io/$PROJECT_ID/sonar-scanner:latest
args:
- '-Dsonar.host.url=https://sonar.test.io'
- '-Dsonar.login=XXXXXXXXXXXXXX'
- '-Dsonar.projectKey=service-name'
- '-Dsonar.sources=.'
also this is running that mean there no issue in builder image it's just issue of passing parameters
docker run gcr.io/$PROJECT_ID/sonar-scanner:latest bash -Dsonar.host.url=https://sonar.test.io -Dsonar.login=${_SONAR_LOGIN} -Dsonar.projectKey=service -Dsonar.sources=.
if condition working but after that then condition having some issue

You should escape the quotes:
- 'if [ $BRANCH_NAME != \'production\' ]; then sonar-scanner -Dsonar.host.url=https://sonar.test.io -Dsonar.login=${_SONAR_LOGIN} -Dsonar.projectKey=service -Dsonar.sources=. ; fi'
or use double quotes:
- 'if [ $BRANCH_NAME != "production" ]; then sonar-scanner -Dsonar.host.url=https://sonar.test.io -Dsonar.login=${_SONAR_LOGIN} -Dsonar.projectKey=service -Dsonar.sources=. ; fi'

Related

Google cloud - cloubduild and app variabile substitution in trigger

I'm using cloudbuild to deploy new version of my app when a new commit appears in github.
Everything is working good.
Now I'm trying to setup a variable substitution in the trigger configuration, because I want to put my version number in the trigger once, so that I can find the deployed correct version without modifying cloudbuild configuration file.
Variabile substitution works great in my cloudbuild file, for example:
(cloudbuild.yaml)
# TEST: PRINT VARIABLE IN LOG
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'echo', '${_VERSION}']
# DEPLOY APP
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "-v", "${_VERSION}", "app.yaml"]
dir: 'frontend'
timeout: "20m"
${_VERSION} is correctly replaced with the string I put into my trigger.
Now I want to obtain the same result in app.yaml file, substituting an env variabile, something like:
(app.yaml)
runtime: nodejs
env: flex
service: backend
env_variables:
VERSION: "${_VERSION}"
TEST_ENV: "read from google"
When I read TEST_ENV from my app, it works, but _VERSION is not replaced.
Any suggestion?
When you perform this step
# DEPLOY APP
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "-v", "${_VERSION}", "app.yaml"]
dir: 'frontend'
timeout: "20m"
The app.yaml is provided as-is to the gcloud command, and it's not evaluated. You have to update it manually. Something like this
# REPLACE: PUT THE CORRECT VALUE IN APP.YAML FILE
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'sed', "-i", "sed -i "s/\$${_VERSION}/${_VERSION}/g", 'app.yaml']
Of course if you let the
env_variables:
VERSION: "${_VERSION}"
as-is in your app.yaml file. You can change this replacement string
I want to add this solution in case someone has problems with the one proposed by giullade (in my case, cloudbuild gave me an error in executing the sed command).
I also changed my replacement string to one more readable and to avoid escaping the $ sign.
# Step 0: REPLACE variables in app.yaml file
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
dir: 'backend'
args:
- '-c'
- |
sed -i "s/__VERSION/${_VERSION}/g" app-staging.yaml
and in my app.yaml:
env_variables:
VERSION_ENV: "__VERSION"

Error: Deployment failed Resource is not in the state deploymentSuccessful Finishing: CodeDeployDeployApplication

I am quite new to azure devops. I am trying to deploy my app from azure to aws ec2 using codedeploy. The build pipeline is giving "Error: Deployment failed Dev-TitasEcom-Frontend Resource is not in the state deploymentSuccessful" error. I have searched a lot but could not find any proper resource to resolve the issue. Below given the code of azure-pipelines.yaml
pr:
branches:
include:
- master
- dev
paths:
include:
- ./*
jobs:
- job: Build_Job
displayName: Build
pool:
vmImage: 'ubuntu-latest'
demands:
- npm
steps:
- checkout: self
clean: false
# - powershell: 'npm cache clean --force'
# displayName: 'PowerShell Script'
# env:
# APPDATA: npm-cache
# - task: CodeDeployDeployApplication#1
# inputs:
# awsCredentials: 'AWS Service Con'
# regionName: 'us-east-1'
# applicationName: 'Dev-TitasEcom-Frontend'
# deploymentGroupName: 'Dev-erp-Frontend'
# #deploymentRevisionSource: 'workspace'
#revisionBundle: 'Dev-erp-Frontend-Rev'
# bucketName: 'Dev-erp-Frontend'
# fileExistsBehavior: 'OVERWRITE'
# batch: "true"
- task: CodeDeployDeployApplication#1
inputs:
awsCredentials: 'AWS Service Con 1'
regionName: 'us-east-1'
applicationName: 'Dev-TitasEcom-Frontend-11'
deploymentGroupName: 'Dev-erp-Frontend'
environmentName: 'DevTitasecomFrontend-env-12'
applicationType: 'version'
bucketName: 'elasticbeanstalk-us-east-1-3067722226831765'
fileExistsBehavior: 'OVERWRITE'
#versionLabel: 'v.0.01'
batch: "true"
- task: Npm#1
displayName: 'Npm Install'
inputs:
workingDir: "./"
command: "ci"
- task: Npm#1
displayName: 'Lint Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run lint"
continueOnError: true
- task: Npm#1
displayName: 'Copy Assets'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run copy-files"
continueOnError: false
- task: Npm#1
displayName: 'Build Client App'
inputs:
workingDir: "./"
command: "custom"
customCommand: "run build:prod"
# Archive files
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip' # Options: zip, 7z, tar, wim
archiveFile: '$(Build.ArtifactStagingDirectory)/www.zip'
replaceExistingArchive: true
- task: CopyFiles#2
inputs:
Contents: 'www/**'
TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: 'titas-ecom-erp'

CloudBuild Bash-style string manipulation

I need to extract a string from the TAG_NAME default variable. but i could not get this to work.
- name: 'gcr.io/cloud-builders/git'
id: find-folder-name
dir: ${_DIR}
entrypoint: 'bash'
args:
- '-c'
- |
if [ ${_STRATEGY} = "tag" ]; echo "tag name " $TAG_NAME; echo ${TAG_NAME%\.np\.v\.*};fi
volumes:
- name: 'ssh'
path: /root/.ssh
secretEnv: ['GCLOUD_SERVICE_KEY']
The regex works perfect fine if i just run it in gitbash locally.
The output is follow, i am expecting it will also print "test" as well, but it is empty. here is the output
tag name test.np.v.1.1.7
Fixed by assigning to a variable
release=$TAG_NAME;echo ${release%\.np\.v\.*}

GCP cloudbuild optional step

Having this config on cloudbuild.yaml (there are other similar fragments on the file):
- name: 'gcr.io/cloud-builders/gcloud'
id: 'step_1'
args: ['builds',
'submit',
'--config=path_to_sub_app_1/app_1_build.yaml',
'--substitutions=VAR_1=${ENV_VAR_1}']
waitFor: ['Docker push']
- name: 'gcr.io/cloud-builders/gcloud'
id: 'step_2'
args: ['builds',
'submit',
'--config=path_to_sub_app_2/app_2_build.yaml',
'--substitutions=VAR_1=${ENV_VAR_1}']
waitFor: ['Docker push']
Is it possible to skip step_1 and continue the execution normally (step_2)?
Use entrypoint: 'bash':
- name: 'gcr.io/cloud-builders/gcloud'
id: 'step_1'
entrypoint: 'bash'
args:
- '-c'
- |
if [ "$_SKIP_STEP" != "true" ]
then
gcloud builds submit --config=path_to_sub_app_1/app_1_build.yaml --substitutions=VAR_1=${ENV_VAR_1}
fi
waitFor: ['Docker push']
Define this var: _SKIP_STEP="false"
And now we can run the build and skip step_1:
gcloud builds submit --config=cloudbuild.yaml --substitutions=_SKIP_STEP=true

Why is my Container Builder build failing with "failed to find one or more images after execution of build steps"

I don't understand what this error message means. It happens at the end of my build, when the build is complete and the image is being tagged. Here's the tail end of the log:
Step 17/18 : WORKDIR /var/www
---> 0cb8de2acd8f
Removing intermediate container 7e7838eac6fb
Step 18/18 : CMD bundle exec puma -C config/puma.rb
---> Running in 9089eb79192b
---> 890a53af5964
Removing intermediate container 9089eb79192b
Successfully built 890a53af5964
Successfully tagged us.gcr.io/foo-staging/foobar:latest
ERROR
ERROR: failed to find one or more images after execution of build steps: ["us.gcr.io/foo-staging/foobar:a2122696c92f430529197dea8213c96b3eee8ee4"]
Here's my cloudbuild.yaml:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'us.gcr.io/$PROJECT_ID/foobar', '.' ]
images:
- 'us.gcr.io/$PROJECT_ID/foobar:$COMMIT_SHA'
- 'us.gcr.io/$PROJECT_ID/foobar:latest'
timeout: 3600s
I thought maybe it was a transient failure, but I retried the build and it happened again.
Ah I needed to tag the build in the build step:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'us.gcr.io/$PROJECT_ID/foobar:$COMMIT_SHA', '-t', 'us.gcr.io/$PROJECT_ID/foobar:latest', '.' ]
images:
- 'us.gcr.io/$PROJECT_ID/foobar:$COMMIT_SHA'
- 'us.gcr.io/$PROJECT_ID/foobar:latest'
timeout: 3600s